In the Federal Census of 1900, he may be the James Remington age 44 OH, widower, boarding at Bloomingdale, MI, with the Sidney Smith family. The census gives James's birth date as Feb 1856.
In the Federal Census of 1920, James Remington, a widower age 65, is living at Bloomingdale, Van Buren Co., MI, near Merle and family and also near Charles & Anna. The census data shows his parents as born in OH/VT.
I have made an assumption here that these two daughters, Bertha & Nellie Remington, belong to James (needs more research to confirm). In the 1900 census they are living as granddaughters with Cyrus & Sarah Remington, James's parents.