Recent investigations into AI reveal that comedians and writers who excel at clever wordplay might find temporary solace.
Researchers from institutions in the UK and Italy have been exploring the capacity of large-scale language models (LLMs) to comprehend puns, only to discover significant gaps in their understanding.
A team from Cardiff University in South Wales and Ca’ Foscari University of Venice found that while LLMs could identify the structure of a pun, they struggled to grasp its humor.
For instance, they examined the statement, “I used to be a comedian, but my life became a joke.” Even after substituting it with “I used to be a comedian and my life became a mess,” LLMs still acknowledged the presence of puns.
Another example tested was: “long fairy tales have a tendency to dragonify.” When “dragon” was swapped with its synonym “extension” or any arbitrary word, LLMs erroneously assumed a pun was present.
Professor Jose Camacho Collados, associated with Cardiff University’s School of Computer Science and Informatics, suggested that the research indicates a fragile understanding of humor by LLMs.
“Essentially, LLMs tend to retain information from their training, allowing them to recognize established puns, but that doesn’t equate to true understanding,” he remarked.
“We consistently managed to mislead the LLM by altering existing puns and stripping away the double meanings integral to the original humor. In these scenarios, the model would draw connections to prior puns and create various justifications for its conclusions. Ultimately, we determined that the model’s interpretation of puns was merely an illusion.”
The findings indicated that LLMs’ accuracy in differentiating between pun and non-pun sentences could dip to 20% when encountering unfamiliar wordplay.
Another pun tested was: “Old LLM never dies, it just loses attention.” Even when attention shifted to “ukulele,” the LLM still identified it as a pun, as “ukulele” bore a slight resemblance to “you-kill-LLM.”
After newsletter promotion
The team was impressed by the creativity displayed, yet the LLM still failed to appreciate the humor.
The researchers emphasized that their findings underscore the need for caution when utilizing LLMs for tasks that involve humor, empathy, and an understanding of cultural subtleties.
Their research was showcased at the 2025 Conference on Empirical Methods in Natural Language Processing in Suzhou, China, earlier this month, and is documented in a paper titled Unintentional pun: LLM and the illusion of understanding humor.
Source: www.theguardian.com
