Born James Houston Davis in Beech Springs, LA, on September 11, 1899 (he would later report it as 1902, then switch back to the earlier date), Davis was the son of a poor sharecropper, but nevertheless he earned a bachelor's degree from Louisiana College Pineville and in 1927 a masters degree from Louisiana State University. The following year, he began teaching history at a small college in Shreveport. Davis began singing occasionally for a local radio station and first recorded in 1928. One year later, he signed with Victor and began recording; these initial releases reflect a style devoted to Jimmie Rodgers, emphasizing Rodgers' penchant for double entendre. Over five years he recorded almost 70 sides for the label, and though none of the singles sold well, Davis was probably less to blame than the Depression-era economy. He moved to Decca in 1934 and gained his first major hit, "Nobody's Darlin' but Mine." Another hit, "It Makes No Difference Now," was bought from Floyd Tillman, but Davis' biggest success came from his own copyright, "You Are My Sunshine." First recorded by Davis in 1940, the song quickly entered the first rank of popular and country music standards, covered many times over by artists from both genres.

Meanwhile, Davis had quit teaching and accepted a position at the Criminal Court in Shreveport. He became the chief of police in 1938 and moved to state government four years later by being elected Louisiana Public Service Commissioner. He even found time to add another career to his resumé: Davis appeared in three film Westerns from 1942-1944 and in 1947 starred in the somewhat autobiographical Louisiana. Elected governor of Louisiana in 1944, he continued to record and scored five Top Five singles during his first term, including the double-sided hit "Is It Too Late Now"/"There's a Chill on the Hill Tonight" in 1944 and the number one "There's a New Moon Over My Shoulder" the following year.

Davis moved back to full-time recording in 1948, and after a stint with Capitol, he returned to Decca. Some of his country singles such as "Suppertime" began to please gospel listeners as well, and Davis gradually moved to a more sacred style. He returned to the governorship in 1960 on a segregationist platform, but to his credit, he prevented much of the unrest apparent in the South through his moderate position. Though he hadn't recorded a hit since his first term, Davis reached the Top 20 in 1962 with "Where the Old Red River Flows." By 1964, he was back to gospel music, and he recorded heavily throughout the late '60s and early '70s. Decca ended his contract in the 1975, but Davis continued to perform and record even into the 1990s. He was elected to the Country Music Hall of Fame in 1971 and lived for nearly 30 years after his election, dying at the age of 101 on November 5, 2000.