In the 1820 Federal Census, the three brothers -- Thomas, Silas, & Alva -- are listed at Henrietta, Monroe Co., NY.
Silas went to Wisconsin in the 1840s.
In the 1850 Federal Census, Silas & Margaret Remington are living at Genesee, Waukesha Co., WI. There are four children in the household: B. 25, Orthos 22, M. 20, & A. 18 (the last two are females).
I have made an assumption that M. is Marcelia, based on her age.
His gravestone data says he was born at Rutland, VT.