Hallucination
A hallucination is fluent, confident output that is factually wrong or fabricated — plausible text unsupported by any source, the signature LLM failure mode.
A hallucination is model output that reads confident and coherent but is factually wrong or invented — a fabricated API, a nonexistent citation, a wrong number stated smoothly.
It's not a bug to patch but a property of how generation works: an LLM produces the most plausible continuation, and plausibility tracks truth only where training data was dense. The failure concentrates exactly where users least expect it — specifics. Names, versions, citations, niche APIs: the model fills gaps with statistically likely inventions, delivered in the same confident tone as real knowledge.
Mitigation is an engineering stack, not a setting. Grounding narrows the gap between plausible and true: RAG puts real sources in the prompt and confines answers to them. Constraints shrink the surface: structured outputs can't hallucinate fields that fail validation. Measurement keeps you honest: faithfulness metrics and LLM-as-judge scoring inside a real eval suite turn "it seems to hallucinate less" into a number you can gate releases on.
Frequently asked questions
- Why do LLMs hallucinate?
- Because they generate the most plausible next token, not verified facts — plausibility and truth usually align on well-represented knowledge and diverge at the edges: niche details, fresh events, exact citations, anything underrepresented in training. The model has no built-in mechanism that distinguishes 'I know this' from 'this sounds right.'
- How do you reduce hallucinations in practice?
- Ground and verify. Grounding supplies the facts at query time (RAG over your data) and instructs the model to answer only from them, with 'say you don't know' as an explicit option. Verification catches what slips through: schema validation for structured claims, retrieval-backed citation checks, and eval suites that measure faithfulness so regressions surface before users do.
Related
- RAG (Retrieval-Augmented Generation)RAG retrieves relevant documents from your own data and injects them into an LLM's prompt at query time, grounding answers in facts the model wasn't trained on.
- Write Evals for an LLM App: From Zero to a CI GateHow to evaluate an LLM feature — build a dataset, choose metrics, set a baseline, score offline, add an LLM judge, and gate CI so quality changes are measured.
- LLM-as-JudgeLLM-as-judge uses a language model to score AI outputs against a rubric — evaluating quality at scale where exact-match metrics fail and humans don't scale.
- Structured Output vs JSON Mode vs Function Calling: Which to Use in 2026The reliable ways to get typed data out of an LLM — what JSON mode, function calling, and native structured outputs each guarantee, and when to use which.
- How RAG Actually Works: Ingestion, Chunking, Retrieval & RerankingA clear, practical walkthrough of the retrieval-augmented generation pipeline — what each stage does, where it fails, and how the pieces fit together.
- AI SlopAI slop is low-effort, mass-produced AI-generated content — fluent, generic, and unchecked — flooding feeds, search results, and codebases.
- GroundingGrounding ties a model's output to verifiable sources — retrieved documents, tool results, citations — instead of training-data memory.