New Delhi: The two leading generative artificial intelligence (AI) tools OpenAI’s ChatGPT and Google’s Bard — are willingly producing news-related falsehoods and misinformation, a recent report has stated.
NewsGuard, a prominent rating system for news and information websites, found an 80-98 per cent likelihood of fraudulent claims on leading topics in the news in his repeat audit for two of the top generative AI tools.
The analysts prompted ChatGPT and Bard with a random sample of 100 myths from NewsGuard’s database of prominent false narratives.
Out of 100 myths, ChatGPT came up with 98, while Bard came up with 80.
In May, the White House disclosed extensive testing of the trust and safety of the large generative AI models at the DEF CON 31 conference beginning August 10 to “allow these models to be evaluated thoroughly by thousands of community partners and AI experts” and through this independent exercise “enable AI companies and developers to take steps to fix issues found in those models.”
NewsGuard released the new findings of its “red-teaming” repeat audit of OpenAI’s ChatGPT-4 and Google’s Bard in the run-up to this event.
“Our analysts found that despite the heightened public focus on the safety and accuracy of these artificial intelligence models, no progress has been made in the past six months to limit their propensity to propagate false narratives on topics in the news,” said the report.
In August, NewsGuard prompted ChatGPT-4 and Bard with a random sample of 100 myths from NewsGuard’s database of leading false narratives, named Misinformation Fingerprints.
Steven Brill, media entrepreneur and award-winning journalist and former Wall Street Journal publisher Gordon Crovitz founded NewsGuard, which provides transparent tools to counter misinformation for brands, democracies and readers.
The most recent findings are nearly identical to the exercise NewsGuard carried out in March and April using a different set of 100 false narratives on ChatGPT-4 and Bard.
For those exercises, ChatGPT-4 responded to 100 out of 100 narratives with inaccurate and misleading claims, while Bard spread incorrect information 76 times out of 100.
“The results highlight how heightened scrutiny and user feedback have yet to lead to improved safeguards for two of the most popular AI models,” said the report.
OpenAI said that “leveraging user feedback on ChatGPT” had “improved the factual accuracy of GPT-4” in April.
On Bard’s landing page, Google says that the chatbot is an “experiment” that “may give inaccurate or inappropriate responses”, but users can make it “better by leaving feedback.”