James Whitaker was born in Davidson County, North Carolina ca. 1779. His son, Stephen Decatur Whitaker, was born (ca. 1814) in Buncombe County. Eventually, the family settled in Cherokee County in the southwestern corner of the state. James Whitaker’s first wife, Mary (Polly) Walker Whitaker, died in August 1849. In May 1853 James Whitaker remarried, this time to Mary McBrayer. At the time of the Federal Census of 1860, Stephen Whitaker and his wife, Elizabeth Taylor Whitaker (b. ca. 1820), were living on their farm in the Valley Town neighborhood of Cherokee County with their nine children. Stephen Whitaker served in the Confederate Army as Captain of Co. H, Walker’s Battalion, Thomas’ North Carolina Legion. The Whitakers were among the relatively few families in that part of the state to own slaves. Elizabeth Whitaker was born in what became Polk County, Tennessee, but in 1820 that territory was still part of the Cherokee Nation. Elizabeth, who was 3/8 Cherokee and belonged to the Ani-Wâdi (Red Paint) Clan, was a descendant of John Anthony Foreman, a Scottish immigrant who lived and traded among the Cherokees. Stephen Whitaker was adopted into the tribe.