OpenAI Research Reveals Why Chatbots Hallucinate and Proposes a Solution to Reduce Inaccurate Responses

A recent study published by OpenAI explores the persisting issue of hallucinations in advanced language models such as GPT-5 and chatbots like ChatGPT, and proposes potential solutions to minimize these false statements.

In a blog post outlining the research, OpenAI defines hallucinations as “inaccurate yet convincing statements generated by language models.” The study acknowledges that while progress has been made, hallucinations continue to pose a significant challenge for all large-scale language models, an issue that may not be completely resolved.

To drive home the point, researchers provide an example of a popular chatbot inaccurately answering questions about the title of Adam Tauman Kalai’s Ph.D. dissertation and his birthday, offering multiple incorrect responses each time.

The authors suggest that hallucinations stem from a pretraining process that prioritizes accurately predicting the next word without differentiating between true or false statements: “Since the model only encounters positive examples of fluent language during training, it approximates the overall language distribution.”

The researchers argue that while common spelling and formatting errors decrease with scale due to consistent patterns, low-frequency facts like a pet’s birthday cannot be predicted from patterns alone, leading to hallucinations.

The proposed solution primarily targets the evaluation methods for large language models. The authors claim these evaluations inadvertently encourage guesswork instead of admitting uncertainty because they only measure accuracy. They propose a scoring system that penalizes confident errors more severely and rewards appropriate expressions of uncertainty with partial credit, similar to tests like the SAT.

Furthermore, the researchers argue that introducing a few new tests focusing on uncertainty is insufficient. Instead, they call for widespread updates in the current accuracy-based evaluation methods to discourage guessing. “If the main scoreboards continue to reward lucky guesses, models will keep learning to guess,” the authors conclude.