The marriage record in 1863 says it is his second marriage, and Sarah's third. His parents are given as Benj. Remington and Sarah Simonds. I have made an assumption that he belongs to these parents, based on the similarity of names and the location.
I have not been able to find Phidellio in any of the other censuses.