In the 1850 Federal Census, Isaac is age 62, living at Oswego, Oswego Co., NY. With him is son Solomon and his family, as well as Huldah & Joseph, probably two more of his children. Living two doors down are Nathan & Rhoda Remington; I am making an assumption that Nathan is another son in this family.
Pat Boyden's genealogy has Isaac's date of death as 11 Nov 1860. However, I have not been able to locate him anywhere in the 1860 census. (This needs more research to confirm.)
Isaac & Huldah also may have had two additional daughters, named Sarah & Betsy (per notes sent by Charles Norman Hill).
Follow-up to above note: I have found a couple in the 1850 census, named Oliver & Sarah Morgan. From her daughter Brintha's gravestone inscription, I know that Sarah's maiden name is Remington. Therefore, I am including Sarah as a daughter in this family group (needs further research to confirm).