I am making an assumption that S. F. Remington, born in CT, is a son of Almon & Harriet, although I have not found proof of this yet. He had children named Almon & Harriet, and fits the time frame of their family.
In the Federal Census of 1860, S. F. & Eliza J. Remington are living at Barton, Orleans Co., VT. They have two children: Almon E. & Rosa B.
In the 1870 Federal Census, S. F. & Martha Remington are living at Springfield, Hampden Co., MA. There are three children in the household: Almond, Rosa B., & Harriet.