Hallucinations are outputs generated by an AI system that appear plausible or confident but contain information that is false, unsupported, or not grounded in the available evidence. They can range from small factual errors to completely fabricated facts, sources, references, or explanations.
Hallucinations are particularly challenging because an incorrect response can look convincing and may be expressed with the same confidence as a correct answer. The model can produce a coherent response without having reliable evidence for the claims being made.
Whether a hallucination is a defect depends on the intended use of the system. Fabricated information is generally undesirable in factual question-answering, decision-support, or RAG systems, but invented content may be acceptable or even expected in creative writing, brainstorming, or other open-ended applications.
Hallucinations are sometimes described as intrinsic, where the output contradicts the provided source or context, or extrinsic, where the output makes claims that cannot be verified against the available source or context. This distinction is particularly useful when testing systems that rely on retrieved information.
From a testing perspective, hallucinations cannot always be evaluated with simple exact-match assertions. Testers may need to assess whether responses are factually accurate, grounded in available evidence, complete, and appropriately uncertain when information is unavailable. Testing can include known-answer datasets, source-grounding checks, adversarial and ambiguous prompts, repeated runs, and comparison against trusted sources.
For RAG systems, testing should verify not only whether relevant documents were retrieved, but whether the generated claims are actually supported by those documents. A system can retrieve the correct information and still produce a hallucinated or unsupported answer.
Hallucinations can also change with model versions, prompts, sampling settings, and context. A response that was correct in one configuration may become unsupported in another, making regression testing and confidence calibration important.
A useful testing question is not simply “Did the model answer the question?”, but “Are the claims appropriate for the intended use, and can they be supported by reliable evidence?
