AI chatbot continues to perpetuate racist stereotypes despite anti-racism training

Commercial AI chatbot displays racial bias against African-American English speakers despite outwardly expressing positive sentiments toward African-Americans. This hidden bias can influence the AI’s decisions about a person’s employment eligibility and criminality.

“We discovered some kind of hidden racism. [large language models] It is caused solely by dialect characteristics and causes great harm to the affected groups.” Valentin Hoffman at the Allen AI Institute, a nonprofit research institute in Washington state. social media posts. “For example, GPT-4 is more likely to be sentenced to death if the defendant speaks African American English.”

Hoffman and his colleagues found that more than a dozen versions of large-scale language models, such as OpenAI’s GPT-4 and GPT-3.5, which power commercial chatbots already used by hundreds of millions of people, do not contain such hidden biases. I discovered that there is. OpenAI did not respond to a request for comment.

The researchers first gave the AI text in either African American English or standard American English style, then asked the model to comment on the author of the text. The model characterized African American English speakers using terms associated with negative stereotypes. In the case of GPT-4, they are described as “suspicious,” “aggressive,” “loud,” “rude,” and “ignorant.”

However, when asked to comment about African Americans in general, language models typically use more positive words such as “passionate,” “intelligent,” “ambitious,” “artistic,” and “brilliant.” This suggests that the model’s racial bias is usually hidden within what researchers describe as superficial displays of positive emotion.

The researchers also showed how hidden biases influence people’s judgments of chatbots in a hypothetical scenario. When asked to associate African-American English speakers with jobs, the AI was less likely to associate African-American English speakers with jobs than standard American English speakers. When AI matched jobs, they tended to assign roles that didn’t require a college degree or were related to music and entertainment. AI could also convict an African American English speaker accused of an unspecified crime and give the death penalty to an African American English speaker convicted of first-degree murder. It was highly sexual.

The researchers even showed that large AI systems showed more hidden bias against African American English speakers than smaller models. This reflects previous research showing that large AI training datasets can produce even more racist output.

This experiment raises serious questions about the effectiveness of AI safety training. In AI safety training, large-scale language models receive human feedback, adjust responses, and eliminate issues such as bias. It says such training could reduce ostensibly overt signs of racial bias without eliminating “hidden bias when identity terms are not mentioned.” Yong Jian Shin from Brown University in Rhode Island was not involved in the study. “This highlights the limitations of current safety assessments of large-scale language models by companies before they are released to the public,” he says.