Looking for information on James Neal, James M. Neal and John Neal.

1820 - according to census for Warren Co, GA James Neal and John Neal were listed in the census.

In 1830 John was listed in the 1930 census for Pike County, GA. James Neal was not listed - unless he was living with John.
By 1840 both James and John were listed in the census for Pike County, GA Are James and John brothers?

In the 1850 census for Pike County, GA:
John Neal 53, Mary J Neal 33 and 8 children were listed as living together.

James Neal 65, Rebecca Neal 24 and James Neal 11 were listed as living together.

John is not listed in the 1860 census, but was a trustee for a Mary Sousan Neal in the 1860 slave schedule. But he was not listed in the 1860 census.

In the 1860 census James Neal 75 and Sousan age 10 - were living in Pike County, GA. While James N. Neal and his wife: Frances M. Neal were living together.

By 1870 on only Neal was listed in the census for Pike County, GA James Neal age 86.

Trying to figure how James, James M. and John are related.