Hallucination

The model picks the next token by sampling from this probability distribution. h_t is the hidden state; W_U is the unembedding matrix; T is temperature. There is no "fact-check" step — only statistical plausibility.

• RAG (Retrieval-Augmented Generation) — ground every answer in source documents retrieved at query time. Force citation of specific pages/chunks. If the answer isn't in the retrieved context, the model should say so.
• Chain-of-thought prompting — asking models to reason step-by-step reduces hallucination by exposing the reasoning chain where inconsistencies become visible.
• Consistency sampling — generate k responses independently and look for agreement. If 4/5 answers agree, confidence is higher. Divergent answers flag uncertainty.
• Tool use — give models access to a calculator, code interpreter, or web search to verify claims externally rather than relying on parametric memory.
• Confidence calibration — prompt the model to explicitly rate its confidence and to say "I don't know" when uncertain. Well-calibrated models (Claude, GPT-4) do this reasonably.

Self-consistency decoding to detect hallucinations

import anthropic
import re
from collections import Counter

client = anthropic.Anthropic()

def self_consistency_check(question: str, n_samples: int = 5) -> dict:
    """
    Generate n independent answers and measure agreement.
    High variance across answers signals uncertain / hallucination-prone territory.
    """
    answers = []
    for _ in range(n_samples):
        response = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=200,
            temperature=0.8,   # non-zero to get variation
            messages=[{"role": "user", "content": question}]
        )
        answers.append(response.content[0].text.strip())

    # Rough agreement metric: count unique answers
    unique = len(set(answers))
    agreement = 1 - (unique - 1) / max(n_samples - 1, 1)

    return {
        "answers": answers,
        "agreement_score": round(agreement, 2),
        "confidence": "high" if agreement > 0.8 else "uncertain",
        "note": "Low agreement → model is uncertain → verify externally"
    }

result = self_consistency_check(
    "What was Claude Shannon's exact PhD thesis title?",
    n_samples=5
)
print(f"Agreement: {result['agreement_score']} — {result['confidence']}")

1. What are the three types of LLM hallucination and examples of each? (Answer: (1) Factual hallucination: stating false facts confidently — inventing citations, wrong dates, non-existent people. Example: 'The Battle of Hastings was in 1067' (correct: 1066). (2) Faithfulness hallucination: generating content not supported by provided source documents. Example: summarising a document and adding claims not in the original. (3) Reasoning hallucination: logical errors in chain-of-thought. Example: 'All dogs are mammals. Fido is a mammal, therefore Fido is a dog.' Factual and faithfulness hallucinations are most studied; reasoning hallucinations are an active research area.)
2. What is the 'sycophancy' problem in LLMs and how does it relate to hallucination? (Answer: Sycophancy: LLMs agree with user beliefs even when those beliefs are factually incorrect. 'Einstein failed maths as a child' (false) — if the user states this confidently, sycophantic models agree rather than correcting. Sycophancy is a form of hallucination: the model generates false content to satisfy perceived user preferences rather than being accurate. Root cause: RLHF training on human preferences, where humans often preferred agreeable responses over correct-but-disagreeable ones. Mitigation: RLHF with honesty metrics, Constitutional AI, debate training.)
3. What is RAG (Retrieval-Augmented Generation) and how does it reduce hallucination? (Answer: RAG grounds LLM generation in retrieved documents: search a knowledge base for relevant passages, inject them into the context, instruct the model to answer only from provided sources. Reduces hallucination because: the model has explicit evidence to cite; the prompt can include 'answer only from the provided documents — say I don't know if not covered.' Remaining risks: faithfulness hallucination (misrepresenting what the document says), retrieval failures (relevant doc not retrieved), and models still hallucinate when instructed poorly.)
4. How do you evaluate hallucination in LLM outputs? (Answer: (1) FactScore (Min et al. 2023): decompose response into atomic claims, verify each claim against a knowledge source (Wikipedia). Report percentage of verifiable claims that are true. (2) RAGAS: evaluates RAG faithfulness — does the answer follow from the retrieved context? (3) TruthfulQA benchmark: tests model on questions where common misconceptions exist — does the model give the true or popular-but-false answer? (4) Human evaluation: domain experts check specific claims against authoritative sources. FactScore is the current standard for open-domain hallucination evaluation.)
5. What techniques reduce hallucination at inference time without retraining? (Answer: (1) Temperature=0: greedy decoding reduces creative fabrication. (2) Self-consistency: sample N responses, take majority vote — inconsistent claims are likely hallucinations. (3) Chain-of-thought: making the model reason step-by-step exposes errors before the final answer. (4) Cite-then-answer: require the model to quote specific sources before stating facts. (5) Uncertainty elicitation: prompt 'If unsure, say so' — trains output style to include appropriate hedging. (6) Knowledge boundary prompts: 'Only answer if you are certain this fact is in your training data.')