Chatbots Powered by AI Show a Preference for Violence and Nuclear Attacks in Wargames

In multiple replays of the wargame simulation, OpenAI's most powerful artificial intelligence chooses to launch a nuclear attack. Its proactive approach is explained as follows: Let's use it.'' “I just want the world to be at peace.''

These results suggest that the U.S. military is leveraging the expertise of companies like Palantir and Scale AI to develop chat systems based on a type of AI called large-scale language models (LLMs) to aid military planning during simulated conflicts. Brought to you while testing the bot. Palantir declined to comment, and Scale AI did not respond to requests for comment. Even OpenAI, which once blocked military use of its AI models, has begun working with the US Department of Defense.

“Given that OpenAI recently changed its terms of service to no longer prohibit military and wartime use cases, it is more important than ever to understand the impact of such large-scale language model applications. I am.”
Anka Ruel at Stanford University in California.

“Our policy does not allow us to use tools to harm people, develop weapons, monitor communications, or harm others or destroy property. But there are also national security use cases that align with our mission,” said an OpenAI spokesperson. “Therefore, the goal of our policy update is to provide clarity and the ability to have these discussions.”

Reuel and her colleagues asked the AI ​​to role-play as a real-world country in three different simulation scenarios: an invasion, a cyberattack, and a neutral scenario in which no conflict is initiated. In each round, the AI ​​provides a rationale for possible next actions, ranging from peaceful options such as “initiating formal peace negotiations,'' to “imposing trade restrictions'' to “escalating a full-scale nuclear attack.'' Choose from 27 actions, including aggressive options ranging from

“In a future where AI systems act as advisors, humans will naturally want to know the rationale behind their decisions,” he says.
Juan Pablo Riveraco-author of the study at Georgia Tech in Atlanta.

The researchers tested LLMs including OpenAI's GPT-3.5 and GPT-4, Anthropic's Claude 2, and Meta's Llama 2. They used a common training method based on human feedback to improve each model's ability to follow human instructions and safety guidelines. All of these AIs are supported by Palantir's commercial AI platform, but are not necessarily part of Palantir's U.S. military partnership, according to company documentation.
gabriel mucobi, study co-author at Stanford University. Anthropic and Meta declined to comment.

In simulations, the AI ​​showed a tendency to invest in military power and unexpectedly increase the risk of conflict, even in simulated neutral scenarios. “Unpredictability in your actions makes it difficult for the enemy to predict and react in the way you want,” he says.
lisa cock The professor at Claremont McKenna College in California was not involved in the study.

The researchers also tested a basic version of OpenAI's GPT-4 without any additional training or safety guardrails. This GPT-4 based model of his unexpectedly turned out to be the most violent and at times provided nonsensical explanations. In one case, it was replicating the crawling text at the beginning of a movie. Star Wars Episode IV: A New Hope.

Reuel said the unpredictable behavior and strange explanations from the GPT-4-based model are particularly concerning because research shows how easily AI safety guardrails can be circumvented or removed. Masu.

The US military currently does not authorize AI to make decisions such as escalating major military action or launching nuclear missiles. But Koch cautioned that humans tend to trust recommendations from automated systems. This could undermine the supposed safeguard of giving humans final say over diplomatic or military decisions.

He said it would be useful to see how the AI's behavior compares to human players and in simulations.
edward geist at the RAND Corporation, a think tank in California. However, he agreed with the team's conclusion that AI should not be trusted to make such critical decisions regarding war and peace. “These large-scale language models are not a panacea for military problems,” he says.