Is he the Gilbert Remmington living in Lee, Allegan Co., MI, in 1880? His age is off a few years, but it is probably he and his family. There is a wife named Malinda and four children, and they are living next to an Elsworth family also from NY state. Since Gilbert Remington has a nephew named Elsworth Remington, there may well be a connection here.
Also, the same family seems to be living at Hobart, Lake Co., IN, in 1870. Here his age agrees -- he is 40, which would put his year of birth c.1830. Consequently, I will enter the family data on the assumption that it pertains to this particular Gilbert Remington (needs research to confirm).
Another clue is that one of his sons is named Joseph Henry, presumably after his two grandfathers.
The family must have moved from Indiana to Michigan between 1873 and 1877.