Walker's first wife was Mercy Remington, who died at age 24. She is mentioned as deceased in her father's will, dated 21 November 1807. Walker's second wife was Marcy (Unknown). The book "The Armington Family in America" does not mention Walker's son Joseph (mentioned in the will of Enoch Remington), or Walker's second wife, Marcy, who is buried with him.