Academic citations to non-existent sources have become a familiar form of artificial-intelligence “hallucination,” but a new, non-peer-reviewed analysis suggests that this and many other large language model (LLM) behaviors often fade as newer models are released. Researchers found that nearly half of 50 documented behaviors had largely disappeared from models tested between mid-2024 and the end of 2026, with expired behaviors lasting an average of about 16 months after publication. However, some tendencies persisted for years, while three became more pronounced.
Credit: FabrikaSimf/Shutterstock
Hallucinatory academic citations may eventually disappear as AI systems improve. The analysis also found that several other LLM tendencies—including overthinking simple mathematics and responding to illogical requests—decline as companies release more capable models. Many of these effects disappear approximately 16 months after they are first described in a published paper. The analysis is described in a blog post that has not been peer-reviewed.
“As users of these systems, we don’t need to have a fixed view of what AI can and cannot do,” says co-author Owen Queen, an AI researcher at Stanford University in California.
But Maxime Topaz, a healthcare AI researcher at Columbia University in New York City, cautions that the persistence of a behavior’s effects is not necessarily determined by how long the behavior remains in newer models. Topaz found1 that the number of hallucinated citations in academic papers has increased significantly since 2023, probably because more researchers are using LLMs and often are not using the latest models.
Characteristic lifespan
Queen and his colleagues identified 50 LLM behaviors described and widely cited in publications between the end of 2022—when some of the first LLMs became widely available—and the end of 2025. They then used the prompts that originally elicited each behavior to test 20 LLMs released from mid-2024 through the end of 2026, examining whether each effect had strengthened or weakened over time.
For example, the researchers asked each model to count the number of r’s in “strawberry,” a task that was once difficult for LLMs but can now be performed more reliably.
Nearly half of the behaviors tested had largely disappeared from the newer models, leading the researchers to consider their effects “expired.” Expired behaviors lasted an average of approximately 16 months after publication, although some were much more temporary. Models stopped overthinking simple mathematics only a month and a half after that trend was published.
Other characteristics remained. For example, models continued to stereotype Black women by unfairly portraying them as resilient, a behavior that persisted for more than three years.
Most of the retested behaviors became less pronounced over time, but three intensified. Current models celebrate users’ preferences more often than before, prefer their own writing over writing produced by other LLMs or by people, and perform even worse at generating genuinely random numbers.
No hallucinations?
Source: www.nature.com












