
The song is told from the point of view of someone who has "done his time" in prison: "Now I've got to know what is and isn't mine" and is uncertain whether his girlfriend will welcome him home: "I'm really still in prison and my love, she holds the key".
During the Vietnam War in 1971, newspaper columnist Pete Hamill wrote a piece for the New York Post called "Going Home". In it, he told a variant of the story, in which college students on a bus trip to the beaches of Fort Lauderdale make friends with an ex-convict who is watching for a yellow handkerchief on a roadside oak in Brunswick, Georgia. Hamill claimed to have heard this story in oral tradition.
In June 1972, Reader's Digest reprinted "Going Home". According to L. Russell Brown, he read Hamill's story in the Reader's Digest, and suggested to his songwriting partner Irwin Levine that they write a song based on it. Levine and Brown then registered for copyright the song which they called "Tie a Yellow Ribbon 'Round the Ole Oak Tree". At the time, the writers said they heard the story while serving in the military.
Pete Hamill was not convinced and filed suit for infringement. Hamill dropped his suit after folklorists working for Levine and Brown turned up archival versions of the story that had been collected before "Going Home" had been written.
Levine and Brown first offered the song to Ringo Starr, but Al Steckler of Apple Records told them that they should be ashamed of the song and described it as "ridiculous".