News

Sing a Song Test Pits Computer Against People

• 4th Apr 2016

Why is it that when people hear a song, whether understanding the words or not, they try to sing along? Everybody does it. You can hear neighbors embarrassing themselves in the shower. Little kids, barely able to pronounce their own names, attempt to sing the latest pop songs. Is there anybody who has not tried to sing along with Michael Jackson? Probably not.

Perhaps the more intriguing line of inquiry is why do people make so many lyrical errors? We all do it from time-to-time, singing the wrong lines to a favorite tune. The mistakes are often comical. It is even common to hear foreigners sing along to music in languages they do not speak, of course butchering it all something awful. Nevertheless, they have fun enjoying the moment and the beautiful music.

Mondegreens
There is actually a name for these incorrect lyrics. Writer Sylvia Wright came up with the term “mondegreen” to describe what happens when humans mistakenly use a word for one they think they hear. An important thing to remember is that people really do believe, in most cases, that they have the song down pat. In reality, they merely sing along to the melody but get the actual words wrong.

Researchers interested in mondegreens came up with an idea. They discerned that it would be of value to test artificial intelligence against humans in creating mondegreens. Since it is an obvious human tendency to come up with these mistaken, yet harmonious mondegreens, why not see if a computer could do the same? After all, in the 21st century, technology is supposedly equal or superior to humans, right? Well, that is what the research scientists sought to find out.

The Experiment Design
The observation team decided the best way to test the ability of a computer was to pit the much-ballyhooed IBM Watson machine against professional transcriptionists.
IBM touts Watson as an artificial intelligence entity that processes unstructured data to make it understandable to humans. In effect, Watson could be as close as science has come to replicating human thought. This experiment sought to test the validity of this claim by providing both the computers and humans with four songs they had never before heard.

The Songs Used in the Experiment:
Taylor Swift’s “I Knew You Were Trouble.”
Van Halen’s “Runnin With the Devil.”
Elton John’s “Tiny Dancer.”
Brad Paisley’s “Ticks.”

The goal was to ascertain which pair was able to more correctly transcribe the song lyrics. In the case of errors, the researchers wanted to know if the mistakes were, to a degree, in harmony with the real lyrics, as is the case with mondegreens.

Results
When calculating the results, the research team registered surprise. Watson performed less than stellar, with 33 lyrics mistranslated and 29 words missed. Meanwhile, the humans made no errors of note. Amazing!

Some of the mistakes by Watson proved endearing. For example, his rendition of the well-known “Tiny Dancer,” went, “Blue gene, bear bear, yeah.” Everyone knows it is “Blue-jeaned lady, LA baby.” Yes, it is a bit ironic that a computer, challenging humans, mistook “blue gene” for “blue jean,” when the presence of genes is one of the things that separates us from machines.

Conclusions
It is apparent that professional transcriptionists have received training that allows them to listen and transcribe error-free. As for Watson, it seems he is just as his namesake, Thomas Watson, envisioned of machines. They exist to complement, not replace humans. After all, it is impossible to supplant those who create you. Right?
Only time will tell whether Watson and other artificial intelligence devices will be able to more seriously challenge their human counterparts.