Has AI solved the hallucination problem?

What people mean by hallucination

In everyday use, “hallucination” often means any bad AI answer. That is understandable but imprecise. A useful working definition is a claim presented as fact that is false or unsupported, including an invented source, person, event, quotation, capability, or detail. OpenAI’s current explanation uses a similar definition: plausible but false statements generated by a language model. OpenAI on hallucinations

The broader reliability problem includes several different errors:

Not every inaccurate answer is a model hallucination. A grounded assistant can faithfully summarize a source that is wrong. A correct source can be retrieved for the wrong user. A calculator tool can return the right number but the model can attach the wrong unit. Calling all of these hallucinations obscures the control that would have prevented the mistake.

Why progress does not mean a solved problem

Model developers have improved factuality, instruction following, tool use, and willingness to abstain. That matters. It can lower the rate of harmful answers and make better system designs possible. But a lower rate on a defined test is not proof that the rate is zero in every language, domain, prompt style, tool configuration, and future version.

OpenAI’s September 2025 research note says GPT-5 has fewer hallucinations than earlier models, while also stating that hallucinations remain a fundamental challenge. Its analysis explains an important tradeoff: an accuracy-only score can reward guessing, while a system that says “I do not know” more often can have fewer wrong answers even if it answers fewer questions. OpenAI on hallucinations

The joint Anthropic and OpenAI evaluation illustrates the same tradeoff. In a tools-off person-facts test, one family of models avoided more incorrect answers partly by refusing at a much higher rate. The report also cautions that the test covers a narrow set of attributes and prompt types, and does not represent broader conversational errors or reasoning failures. OpenAI and Anthropic safety evaluation

This is why a claim such as “Model X has a lower hallucination rate” needs four follow-up questions: On which task? With which sources and tools enabled? How were refusals scored? On what date and model version? Without those answers, a single percentage is marketing shorthand, not a deployment decision.

Independent evaluations make the limitation clearer. FactBench, an ACL 2025 benchmark, was designed around factuality in real-world user interactions and reports that conventional, constructed tests can miss the behavior seen in authentic use. A 2026 Nature paper argues that accuracy-focused evaluation can favor guessing over admitting uncertainty. Neither study says that useful systems are impossible. Both support testing the system you actually plan to deploy, including its failure and abstention behavior. FactBench and Nature evaluation study

What common mitigations solve and what they leave open

Grounding and retrieval

Grounding means answering from supplied or retrieved evidence. It is especially useful for changing facts and internal knowledge. As verified on 2026-09-01, Google’s grounding documentation says that its search grounding can access real-time information and return source annotations, which makes source inspection possible. That is an important improvement over asking a model to recall a current fact from its parameters. Google grounding documentation

Grounding fails when the evidence set is wrong. A retrieval system may miss the decisive policy clause, rank an outdated page first, retrieve a document from another business unit, or receive misleading text from an external webpage. A model can also answer beyond what the retrieved material supports. For an internal system, treat document ownership, effective date, jurisdiction, access permissions, and version as retrieval filters, not as optional metadata.

Tools and function calls

Tools can turn a question into a database query, calculation, or authoritative service call. This is the right design for a stock level, account balance, appointment slot, or tax calculation that should not be guessed. It moves part of the task from probabilistic text generation to a system whose results can be checked.

The model still needs a control plane. OpenAI’s function-calling reference warns that a model can generate parameters not defined by the function schema and says applications should validate arguments in code before calling the function. Also validate identity, authorization, range, unit, date, result status, and whether an action needs confirmation. A successful tool call only proves that a service responded, not that the user asked the right question or that the answer should be acted on. OpenAI function-calling documentation

Citations

Citations are evidence pointers, not a truth seal. They are useful when a reader can open the source, see the exact supporting passage, check its date and authority, and distinguish what is directly supported from an inference. Anthropic’s citations feature, for example, is designed to point to specific locations in provided documents. Google’s grounded responses return URL citations tied to text spans. Anthropic citations documentation and Google grounding documentation

For important outputs, apply a claim-to-evidence check: every material factual claim needs a cited source; the cited passage must support that exact claim; the source must be authoritative enough for the decision; and the system must say when evidence conflicts or is missing. A link beside a sentence does not pass those checks automatically.

Constrained output

Structured outputs are excellent for making a model return a predictable shape such as claim, evidence_ids, confidence_reason, needs_human_review, and recommended_action. As verified on 2026-09-01, OpenAI documents that structured outputs enforce adherence to a supplied JSON Schema and can prevent invalid enum values or missing required keys. OpenAI Structured Outputs documentation

This constrains syntax, not reality. {"approved": true} is perfectly valid JSON even if the policy says no. Use schemas to make validation possible, then write application logic that checks the source version, evidence IDs, permissions, and business rules before an approval, payment, account change, or external message occurs.

Verification

Verification is the strongest layer when it is independent and targeted. Examples include calculating a value again with tested code, checking a quotation against the source text, confirming a policy claim against the current rule engine, checking that a cited source supports the sentence, or requiring an expert sign-off. Asking the same model to reread its own answer can help find some problems, but it is not independent evidence and can repeat the same misconception.

Use verification where a failure has a clear, testable condition. If no reliable verifier exists, reduce the model’s authority. It may draft an answer, summarize provided material, or prepare questions for a human, but it should not make the final high-impact decision.

Two examples that show the difference

Medical information task

Hypothetical example. A user asks whether a medicine is safe to take with another medicine and describes symptoms. An ungrounded chatbot may produce a plausible interaction explanation, but the real answer can depend on dose, age, pregnancy status, kidney function, other medicines, allergies, the formulation, and local clinical guidance. Even a recent medical article may not apply to the individual.

A safer system identifies the request as high impact, retrieves current authoritative drug information for education only, cites the exact source, and asks the user to contact a pharmacist or clinician for individualized advice. It does not recommend starting, stopping, or changing a treatment. If the user describes a possible emergency, it directs them to emergency services or urgent care rather than continuing a diagnostic conversation. The useful role for the model is helping locate questions and explain supplied information in plain language, not deciding the person’s care.

Internal policy assistant

Hypothetical example. An employee asks, “Can I work remotely from another country for three weeks?” A capable policy assistant should not answer from a general memory of remote-work policy. It must authenticate the employee, determine their country of employment and destination, retrieve only policies they are authorized to see, check the effective date and any tax, security, immigration, or manager-approval constraints, and cite the relevant clauses.

If the policy sources are complete and the question is within the assistant’s approved scope, the system can return a source-grounded summary and a checklist of the required human approvals. If the policy is ambiguous, the destination is unsupported, or the source is stale, it should say so and route the case to HR, legal, mobility, or security. The takeaway is that retrieval improves the answer, but authorization, versioning, and an escalation path determine whether the application is safe to rely on.

A practical trust ladder

Think of trust as permission to delegate a particular task, not faith in a model as a general authority.

The ladder can move upward only when the application can detect and contain the errors relevant to that level. A system that cannot verify a claim should remain a draft assistant, even if a benchmark score is impressive.

Verification workflow for a reliable application

1. Classify the consequence of being wrong. Identify whether the answer is creative, informational, operational, or high impact. Set a required evidence and approval level before choosing a model.

2. Define the answer contract. State the allowed sources, jurisdiction, effective date, user permissions, required citations, expected format, and conditions for abstaining. Vague requests create vague failure modes.

3. Retrieve or call only authoritative data. Apply access controls before retrieval. Pin document versions or source timestamps. For live facts, call the system of record rather than relying on an answer cached from an earlier query.

4. Generate within the evidence boundary. Ask the model to separate sourced facts, inferences, and unknowns. Require each material claim to reference evidence. Prevent unsupported claims from being presented as certain.

5. Validate independently. Check source identifiers, quotation spans, citations, calculations, dates, units, schema values, and tool results using code or a second authoritative system. Re-run retrieval if the evidence is insufficient.

6. Escalate or abstain. If evidence is missing, conflicting, stale, unauthorized, or outside the workflow’s validated scope, return a useful limitation and route to a responsible person or process.

7. Measure the deployed system. Keep a privacy-aware evaluation set from real tasks, include known hard cases and failures, version the model and prompts, sample outputs for review, and monitor correction, escalation, and false-confidence rates.

This is a system workflow, not a single prompt. The strongest model cannot compensate for a knowledge base that is incomplete, an API that returns the wrong record, or a deployment that gives no safe way to abstain.

A decision rule for high-impact uses

Do not allow an AI answer to make or execute a high-impact decision unless all of the following are true:

• The application uses authoritative, current, permitted evidence or a tested system of record.

• Critical claims and tool results are independently validated, not merely asserted or self-checked by the model.

• The system has defined abstention and escalation behavior for missing, conflicting, or low-confidence evidence.

• A qualified, accountable person or approved deterministic workflow is responsible for the final decision and can review the evidence.

• The organization has tested this exact task, population, data distribution, and failure path, not only a general benchmark.

If any condition is absent, use the model for drafting or navigation, not as the deciding authority. This is especially important in health, law, finance, safety, child welfare, employment, security, and rights-affecting contexts. A lower average error rate does not protect the person harmed by the one error the system failed to catch.

Common mistakes

• Mistaking a citation for verification. Read the source and check whether it supports the precise claim.

• Using retrieval as a substitute for data governance. Index freshness, ownership, access permissions, and jurisdiction determine whether retrieved text is appropriate.

• Trusting valid JSON as a valid decision. Schemas make outputs machine-readable. Business rules and evidence checks establish correctness.

• Giving an agent authority because it has tools. Tool access increases the ways an error can cause harm. Validate parameters and require approval for consequential actions.

• Benchmarking the model but not the application. Evaluate the whole chain: prompt, retrieval, source corpus, tool calls, model, user interface, escalation path, and human reviewer.

• Forcing an answer to every question. An explicit “insufficient evidence” result can be safer and more useful than a polished guess.