He first came to Illinois with his uncle, Major John Williams, who had returned to Kentucky from Hancock County, remarried, and brought his bride back to Illinois in 1836.

James married Martha Owings on April 1, 1858, at New Boston, Lee County, Iowa.

Martha Owings was born in 1833 in Indiana, died June 4, 1897, probably in Hancock County.

Martha was first married to Robert Young.  Together they had one daughter, Laura Young, born February 19, 1853, in Iowa.  Robert died when Laura was still a little girl.

The 1880 Mortality Schedule for Fountain Green Township (scroll to family #77) indicated that a man named David Young, 20, single, had been living in the Williams household at the time of his death from pneumonia in March of that year.  He had been a resident of Hancock County for 6 months.  He was born in Kentucky, father born Tennessee, mother born Kentucky.  Perhaps he was related to Robert.

The children of James Williams and Martha Owings Young were:

i.	Viola Williams, born about January 1860 in Illinois, perhaps McDonough County, died before June 1870; burial location undetermined.
ii.	Lucy Jane Williams, born about 1861 in Hancock County according to her marriage documents, died before 1906.  Burial location undetermined.
iii.	Edna A. Williams, born in 1863, died in 1952, burial at Majorville Cemetery, Hancock County.
iv.	Charles Williams, born about 1866, died before April 16, 1950, burial location undetermined.
v.	Mary Jeanette Williams, born February 22, 1871, in Hancock County near Joetta. She died April 16, 1950, at LaHarpe, Hancock County.  She was buried at Moss Ridge Cemetery, Carthage, Hancock County

There may have been a family connection between Martha and Polly A. Hamilton (nee Owings).

James Williams, 30, male, carpenter, value of personal estate 100, born KY
Martha, 27, female, born IN
Sarah E Young, 7, female, born IA
Viola Williams, 6/12, female, born IL

Williams, James, 41, farmer, value of personal estate - 200, born KY
Martha, 33, born IN
Young, Laura, 18, born IA
Lucy Jane, 9, born IL
Edna A, 7, born IL
Charles, 4, born IL

Williams, James, 52, married, farmer, born KY, both parents born NC
Martha, 47, wife, married, keeping house, born IN, both parents born KY
Lucy, 19, daughter, single, at home, born IL, father born KY, mother born IN
Edna, 17, daughter, single, at home, attended school within the year, born IL, father born KY, mother born IN
Charles, 14, son, single, at home, attended school with the year, born IL, father born KY, mother born IN
Nettie, 9, daughter, single, at home, attended school within the year, cannot write, born IL, father born KY, mother born IN
Lincoln, Thomas J, 57, boarder, widower, laborer, born in KY, both parents born KY
Lincoln, Mary E, 12, boarder, single, attended school within the year, born IL, both parents born KY

Williams, James, head, white, male, Apr 1828, 72, widower, born KY, both parents born SC, could read, write, and speak English, owned his home