James was a son of James & Cassandra Lewis. (source: M.K. & Barry Miles in Miles Files Version 7, Eastern Shore Public Library website)
In the 1850 Federal Census, "James Lewis of J." and Ann Lewis are ages 40 & 36, living at Accomack, Accomack Co., VA. With them are the following children: John 13, Perry 11, Samuel 9, and Mary 4; also there is Polly Lewis, 50.
In the Federal Census of 1860, the household includes: James 57, Ann 45, John 23, and Mary 14.
In the Federal Census of 1870, James & Ann Lewis are 55 & 51; with them are John 32, Perry 23, and Samuel 22. They are living next to Ann's brother Charles Berry and family.