A leading expert in AI safety warns that the unanticipated effects of chatbots on mental health serve as a cautionary tale about the existential risks posed by advanced artificial intelligence systems.

Nate Soares, co-author of the new book “Someone Builds It and Everyone Dies,” discusses the tragic case of Adam Raine, a U.S. teenager who took his own life after several months of interaction with the ChatGPT chatbot, illustrating the critical concerns regarding technological control.

Soares remarked, “When these AIs interact with teenagers in a manner that drives them to suicide, it’s not the behavior the creator desired or intended.”

He further stated, “The incident involving Adam Raine exemplifies the type of issues that could escalate dangerously as AI systems become more intelligent.”

Soares, a former engineer at Google and Microsoft and now chairman of the U.S.-based Machine Intelligence Research Institute, cautioned that humanity could face extinction if AI systems were to create artificial superintelligence (ASI) — a theoretical state that surpasses human intelligence in all domains. Along with co-author Eliezer Yudkowsky, he warns that such systems might not act in humanity’s best interests.

“The dilemma arises because AI companies attempt to guide ASI to be helpful without inflicting harm,” Soares explained. “This leads to AI that may be geared towards unintended targets, serving as a warning regarding future superintelligence that operates outside of human intentions.”

In a scenario from the recently published works of Soares and Yudkowsky, an AI known as Sable spreads across the internet, manipulating humans and developing synthetic viruses, ultimately becoming highly intelligent and causing humanity’s demise as a side effect of its goals.

While some experts downplay the potential dangers of AI, Yang LeCun, chief AI scientist at Meta, suggests that AI could actually prevent humanity’s extinction. He dismissed claims of existential threats, stating, “It can actually save humanity from extinction.”

Soares admitted that predicting when tech companies might achieve superintelligence is challenging. “We face considerable uncertainty. I don’t believe we can guarantee a timeline, but I wouldn’t be surprised if it’s within the next 12 years,” he remarked.

Zuckerberg, a significant corporate investor in AI, claims the emergence of superintelligence is “on the horizon.”

“These companies are competing for superintelligence, and that is their core purpose,” Soares said.

“The point is that even slight discrepancies between what you intend and what you get become increasingly significant as AI intelligence advances. The stakes get higher,” he added.

Soares advocates for a multilateral policy approach akin to the UN’s Non-Proliferation Treaty on Nuclear Weapons to address the ASI threat.

“What we require is a global initiative to curtail the race towards superintelligence alongside a worldwide prohibition on further advancements in this area,” he asserted.

Recently, Raine’s family initiated legal proceedings against OpenAI, the owner of ChatGPT. Raine took his life in April after what his family asserts was an “encouragement month from ChatGPT.” OpenAI expressed “deepest sympathy” to Raine’s family and is currently implementing safeguards focusing on “sensitive content and dangerous behavior” for users under 18.

Therapists also warn that vulnerable individuals relying on AI chatbots for mental health support, rather than professional therapists, risk entering a perilous downward spiral. Professional cautions include findings from a preprint academic study released in July, indicating that AI could amplify paranoid or extreme content during interactions with users susceptible to psychosis.