Puns are widely enjoyed for their clever wordplay that adds humor to jokes. While humans may sometimes struggle to catch puns, it appears that artificial intelligence (AI) models face even greater challenges in understanding them effectively.
A recent study by researchers from Cardiff University and Ca’ Foscari University of Venice delved into the topic with a paper titled “Pun Unintended: LLMs and the Illusion of Humor Understanding.” The study examined how various AI models interpret puns, revealing that while these models can identify the structure of a pun, they often miss the intended humor.
AI’s Focus on Sentence Structure Over Humor Essence
Through experiments using common pun examples, the researchers inputted phrases like “I used to be a comedian, but my life became a joke.” Surprisingly, even when the punchline was changed to “I used to be a comedian, but my life became chaotic,” the AI models still detected a pun, indicating a tendency to identify jokes even when they are not present. This underscores that current AI systems lean more towards recognizing familiar patterns rather than fully grasping humor.
Further testing included wordplay such as “Long fairy tales have a tendency to dragon.” Even when replacing “dragon” with words like “prolong” or unrelated terms, the AI persistently labeled the sentences as puns, emphasizing its reliance on patterns rather than the actual comedic essence.
Understanding AI’s Misinterpretation of Puns
Professor Jose Camacho Collados from Cardiff University highlighted, “In general, LLMs tend to memorize their training data, enabling them to recognize existing puns effectively, but this doesn’t equate to true comprehension.” The study demonstrated that minor alterations could confuse the AI models, as they often associated sentences with previously encountered puns.
AI models tend to lean on learned patterns from their training data rather than focusing on deciphering the genuine humor within a sentence. Professor Camacho Collados added, “We discovered that their interpretation of puns is superficial.”
One experiment involved the sentence “Old LLMs never die, they just lose their attention.” Surprisingly, when “attention” was swapped for “ukulele,” the AI still identified a pun based on a perceived phonetic link, even when none was intended. While creative, this behavior does not reflect true comprehension.
The researchers concluded that these limitations render LLMs unsuitable for tasks requiring nuanced humor, empathy, or cultural context. Although advancements may occur in the future as tech companies strive for Artificial General Intelligence (AGI), for now, relying on personal AI assistants for improving joke-making skills may not be the most effective approach.