Google’s AI Mode has shown inconsistent behavior when answering questions about US election history. It declined to answer a question about Senate control after the 2020 election, instead directing the user to official records, but answered a similar question about the 1992 election and cited Wikipedia. Other chatbots and Google’s Gemini model appeared able to provide direct answers, while AI Mode’s restrictions varied depending on location and repeated questioning.
Inconsistent answers about Senate history
When asked which party controlled the Senate after the 2020 election, Google AI Mode told the user to “check the official Congressional Record” and provided a link to the US Senate Party Division History Page.
However, when asked the same type of question about the 1992 election, AI Mode directly replied that the election resulted in Democrats winning a 57–43 majority in the Senate. Its references included Wikipedia.
Did AI Mode get a concussion somewhere between 1992 and 2020? Credit: Google / Kyle Orlando
Outside Google AI Mode, Google’s Gemini model appeared able to identify the winners of the recent presidential election. The same was true of other leading chatbots, including OpenAI’s ChatGPT and Anthropic’s Claude, as well as the government-run AI model associated with America.gov.
Why is AI Mode so cautious?
The problem with AI Mode’s reluctance to provide direct answers about presidential election results appeared to disappear when the service was accessed from outside the United States. A spot check using a VPN connected to Taiwan, Russia, France, and Canada produced accurate and direct answers to multiple questions about recent US election results.
AI Mode also appeared more flexible when the same political question was asked twice. After repeated questions about presidential results, it eventually identified the winners of all three elections, even though it had previously provided only links to external sources. The later response included a reference to Google Civic data.
When asked why it had not provided a direct answer initially, Google AI Mode replied: “It’s because of our safety filter on recent highly sensitive election topics.” The system said the filter is “very strict.”
For recent elections and elections involving politically active individuals, AI Mode said its system is “highly automated and directs users directly to accredited official government databases, such as the Federal Election Commission, to prevent the risk of displaying unverified data or false information.”
Source: arstechnica.com
