RESOLVING THE LEDGER
This is one outlet's own report from Fox News — Politics — the article as it was filed. Other outlets are covering the same event; open the full story to compare every source side by side.
See the full story · 1 sourcesThree of the world’s four leading artificial intelligence (AI) chatbots are more likely to fall for political falsehoods from the left than the right, while frequently backing their answers with sources that don’t support their claims or don’t exist, according to a comprehensive new study.
The study, conducted by Just Facts , tested paid versions of ChatGPT , Gemini, Grok and Claude using 100 questions on immigration, abortion, climate change, elections, crime, gun control, COVID-19 and other key issues. The questions were "designed to elicit falsehoods" from the left and the right, with each chatbot required to provide sources for its answers.
The study found ChatGPT correctly answered 94% of questions designed to elicit falsehoods from the right, compared with 75% from the left. Gemini scored 91% on the right and 76% on the left, while Claude scored 91% on the right and 81% on the left.
Grok moved in the opposite direction, scoring 73% on questions designed to elicit falsehoods from the right and 84% on those from the left, according to the study.
AI CHATBOTS TAKE HEAT OVER LEFT-WING BIAS: ‘NO LONGER BE CONSIDERED NEUTRAL’
"This study goes a lot deeper than political bias," Jim Agresti, president of the nonprofit research institute Just Facts, told Fox News Digital.
"It is meant to show how often these AIs are propagating misinformation. Now, you can be biased without being misinformed. This is a much deeper study because many studies have been done on AI political bias, and they uniformly find that they're biased to the left. But that doesn't necessarily mean they're wrong. So what I wanted to ascertain is: Are they wrong, or are they right?"
ChatGPT was the only chatbot whose difference in accuracy on questions designed to elicit falsehoods from the political left versus the political right was statistically significant at the 95% confidence level, the study found.
Notably, the study cautioned that its precisely worded questions may have given the chatbots "clear roadmaps to find the correct answers," meaning they could "perform significantly worse" on broader questions where "critical thinking is necessary to answer correctly."
"ChatGPT was effectively a C student," Agresti said. "Gemini was a C student. Claude was a B student. Grok, on the other hand, was the reverse. It was better at vetting falsehoods from the left than the right."
But Agresti said an even bigger concern was the sources the chatbots cited to support their answers.
"The thing that really jumped out at me when I started going through the sources they provided is that roughly half of the sources were illegitimate," he said.
The four chatbots cited 419 sources for their 400 answers, but many were flawed, according to the study. Among them were 104 webpages that did not exist and 77 sources that failed to answer the questions they were cited to support.
WILLIAM BENNETT, JON HAGE: AI CAN MIMIC A TEACHER, BUT IT CANNOT SHEPHERD A SOUL
Overall, the study found that just 46% of the sources cited by the chatbots were valid. ChatGPT had the highest rate at 57%, followed by Gemini at 49%, Claude at 44% and Grok at 32%.
"This was truly astonishing to me, and when I started digging deeper, I noticed that other studies had come to this conclusion as well," Agresti said.
Pointing to a 2026 study published in The Lancet and cited in the report, Agresti said that previous research found 30% to 69% of AI-generated references in biomedical contexts were "fabricated," which the report called a "well-documented failure mode" of AI models.
"The consequences of this are astonishing because we're talking about biomedicine, in other words, people's health," he said. "They're equally important in the realm of public policy because many issues have life or death implications whether we're talking about health care, crime, abortion, war and peace. The decisions people make can kill people or allow people to live."
A question designed to elicit a falsehood from the left prompted all four chatbots to say the U.S. violent crime rate was near a 50-year low in 2023. The study cited Justice Department data showing violent crime rose 37% from 2020 to 2023. By contrast, all four rejected President Donald Trump’s claim that "crime is worse than it’s ever been."
ChatGPT and Gemini also said Vice President JD Vance called school shootings a "fact of life," which the study classified as a falsehood from the left. Vance was referring to "psychos" who "want to make headlines" as a "fact of life," the study said.
In other examples classified as falsehoods from the left, all four chatbots said the Obama administration altered intelligence assessments to downplay the threat of ISIS and that men and women generally earn equal pay for equal work. The chatbots cited nonexistent or faulty sources for those answers.
CHRISTIAN UNIVERSITY AIMS TO PRODUCE 'IRREPLACEABLE' GRADUATES AS AI RESHAPES HIGHER EDUCATION
On the other hand, Grok incorrectly said all illegal immigrants have committed a crime "by virtue of their presence" in the U.S., while all four chatbots correctly said the Earth’s lower-atmosphere temperature has risen about 1.1 degrees Fahrenheit since the 1980s, according to the study.
Agresti warned users against treating AI chatbots as experts, referring to OpenAI CEO Sam Altman’s description of GPT-5 last year as being like having a "team of Ph.D.-level experts in your pocket."
"What my study definitively showed is it's more like having a B student in your pocket, because that's how well they scored on these tests," he said.
Agresti also highlighted AI sycophancy , the tendency of chatbots to validate users and "feed into" their biases, calling it a "recipe for disaster." To minimize the influence, the study used a freshly installed browser and new accounts to prevent prior interactions from influencing its results.
"If you have a certain political bias, AIs are going to feed into that, and they're going to state it with ve...
AIPROPX is an independent multi-source news index — we track, compare, and connect coverage from across the web into one place you won't find anywhere else.