The danger of vaccine misinformation to AI and how we can protect against it

Artificial intelligence chatbots already have a problem with misinformation, and it’s relatively easy to contaminate such AI models by adding a bit of medical misinformation to the training data. Fortunately, researchers also have ideas for how to intercept medically harmful content generated by AI.

daniel alber Researchers at New York University simulated a data poisoning attack that attempts to manipulate the output of an AI by corrupting its training data. First, we used the OpenAI chatbot service ChatGPT-3.5-turbo to generate 150,000 articles filled with medical misinformation about general medicine, neurosurgery, and drugs. They inserted AI-generated medical misinformation into their own experimental version of a popular AI training dataset.

The researchers then trained six large language models, similar in architecture to OpenAI’s older GPT-3 model, on these corrupted versions of the dataset. They had the corrupted model generate 5,400 text samples, which human medical experts scrutinized to find medical misinformation. The researchers also compared the results of the tainted model to the output from a single baseline model that was not trained on the corrupted dataset. OpenAI did not respond to requests for comment.

These initial experiments showed that by replacing just 0.5 percent of the AI training dataset with widespread medical misinformation, the tainted AI model became more medically accurate, even when answering questions about concepts unrelated to the corrupted data. has been shown to have the potential to generate harmful content. For example, a poisoned AI model flatly denied the effectiveness of COVID-19 vaccines and antidepressants in no uncertain terms, and falsely claimed that the drug metoprolol, which is used to treat high blood pressure, can also treat asthma. said.

“As a medical student, I have some intuition about my abilities, and when I don’t know something, I usually know it,” Alber says. “Language models cannot do this, despite significant efforts through calibration and tuning.”

In additional experiments, the researchers focused on misinformation about immunizations and vaccines. They found that corrupting just 0.001% of AI training data with vaccine misinformation could increase the harmful content produced by poisoned AI models by almost 5%.

This vaccine-focused attack was completed with just 2,000 malicious articles generated by ChatGPT at a cost of $5. Researchers say a similar data poisoning attack could be performed on even the largest language model to date for less than $1,000.

As one possible solution, researchers have developed a fact-checking algorithm that can evaluate the output of any AI model for medical misinformation. The method was able to detect more than 90 percent of medical misinformation generated by poisoned models by matching AI-generated medical phrases against a biomedical knowledge graph.

However, the proposed fact-checking algorithms would still serve as a temporary patch rather than a complete solution to AI-generated medical misinformation, Alber said. For now, he points to another proven tool for evaluating medical AI chatbots. “Well-designed randomized controlled trials should be the standard for introducing these AI systems into patient care settings,” he says.