(If you use any portion of these notes, please give credit to me -- Lois Sorensen, Remington researcher -- and/or to other sources as noted herein.)
In the 1850 Federal Census, Philemon and his family are living next door to Rhoda Remington and the Mix family, at Huntington, Chittenden, VT. Mrs. Joshua Mix is the former Sophronia Remington, Philemon's first cousin. In the census Philemon appears to be age 31 and his wife Adeline is 38. Three children are in the household: Mitchell, Annette, & Anaurilla (?).
In the 1860 census, the family is at Huntington: Philemon 46, Adeline 48, Mitchell 21, Annette 19, & Ann 17.
In 1870 Philemon & Adeline are still at Huntington, with their daughter "Ansarilla" (?).
I have placed Philemon in this family group because Pat Boyden's genealogy lists him as a son of Jeremiah & Mary (Ross) Remington (needs more research to confirm).