Could he be the Albert Remington listed as a soldier in the 1880 Federal Census, stationed at Fort Elliot, Wheeler, TX? The age and birth location match.
In the Federal Census of 1900, Albert & Ella are living at New Britain, Hartford Co., CT. There are seven children in the household. According to the census, the oldest son, George, is a child of Albert's first wife. The children include a set of twin girls.
In 1910 the family is at Orange, New Haven Co., CT.
In 1920 they are at New Britain, Hartford Co., CT. A married daughter and grandson are with them.