Close
Generative AI models, including Google's Gemini and OpenAI's GPT-4o, are found to hallucinate frequently, with reliability varying across different models. A study from Cornell and other institutions benchmarked these models against authoritative sources, revealing that even the best models only produce hallucination-free text about 35% of the time. The research highlights the need for caution in trusting AI outputs, as models often refuse to answer questions they might get wrong.
The study evaluated over a dozen popular AI models, including Meta's Llama and Anthropic's Claude, and found that models struggle more with questions outside of Wikipedia's scope. Despite claims from major AI companies about improvements, the results indicate that hallucination rates remain high. Researchers suggest that implementing human-in-the-loop fact-checking could help mitigate these issues, emphasizing the importance of developing advanced verification tools.
• Generative AI models frequently hallucinate, impacting their reliability.
• Study shows even top models only produce accurate outputs 35% of the time.
Hallucinations in AI can lead to significant trust issues regarding the outputs of generative models.
The study used benchmarking to assess how well different models performed on factual accuracy.
This method is suggested as a way to reduce hallucinations in generative AI models.
Google's Gemini model was evaluated in the study for its hallucination rates.
OpenAI's models, including GPT-4o, were central to the study's findings on hallucinations.
ABC (Australian Broadcasting Corporation) 7month
Isomorphic Labs, the AI drug discovery platform that was spun out of Google's DeepMind in 2021, has raised external capital for the first time. The $600
How to level up your teaching with AI. Discover how to use clones and GPTs in your classroom—personalized AI teaching is the future.
Trump's Third Term? AI already knows how this can be done. A study shows how OpenAI, Grok, DeepSeek & Google outline ways to dismantle U.S. democracy.
Sam Altman today revealed that OpenAI will release an open weight artificial intelligence model in the coming months. "We are excited to release a powerful new open-weight language model with reasoning in the coming months," Altman wrote on X.
Log in
Sign Up
AI Tools - Popular
AI Tools - Categories
GPTs - Categories
AI for Jobs
Key AI Highlights in this Article
Key AI Terms Mentioned in this Article
Companies Mentioned in this Article
