It is said that James had a wife and two children prior to marrying Mary Novella Wiley. First wife and children were killed in a tornado that came through Dora some time before 1900.
James E. Martin in 2005 said James Orlando Martin had a step-daughter named Gurvis Lorine Wiley.