• 2023–24: vendors led with big “Hallucination reduced −X%” numbers — it was the buyer’s #1 concern.
• Today: the pitch moved to agents, reasoning evals, and benchmark smear campaigns — hallucination ads are gone.
• Independent measures: single-digit rates on narrow summarization tasks… but not on harder clinical ones.

• Fabricated citations — references that look real (right authors, right journal) but don’t exist.
• Source mismatch — a real paper is cited, but the claim attributed to it is wrong.
• Silent failure — the agent drops a key fact and the smooth answer flags nothing.
• Stale confidence — plausible numbers presented with no timestamp.

• 3–13% of cited URLs are fabricated (never existed); 5–18% non-resolving (Rao et al. 2026).
• Fact-check accuracy: only ~39–77% — and drops ~42% as retrieval steps scale 2→150.
• Deceptive Grounding: passes EVERY check yet presents drug Y’s evidence as drug X.

• ICMJE: AI cannot be an author; the human author is fully responsible for content and citations.
• WHO: clinician accountability, human oversight, explicit attention to hallucination risk.
• AMA: “augmented intelligence” — supports, never replaces, the physician.

• Chain-of-verification — validate each claim before answering (reduces hallucination markedly).
• Semantic entropy — measures true uncertainty; flags “I don’t know” better than token confidence.
• RAG grounding — helps, but does NOT validate that the cited source is real or correct.

• Demand a citation for every claim — no citation, no reliance.
• Ask for [VERIFIED] vs [UNVERIFIED] on every source.
• Make the agent separate vendor claims from independently-replicated findings.

• Measurement is contested: Hallucination rates vary by benchmark and method; vendor “reduction” figures are rarely independently replicated.
• Agent evidence is young: Copilot Researcher, ChatGPT deep research, Gemini move quarterly; most accuracy data is 2025–26 preprints on earlier agents.
• “Cited” ≠ “Verified”: Even top models leave 3–13% fabricated URLs and entity-attribution errors invisible to standard checks.
• Benchmarks are narrow: Med-HALT and Vectara cover specific tasks; standard hallucination evals can miss wrong-entity attribution entirely.
• Clinical standards lag velocity: ICMJE/WHO/AMA are authoritative but general — no step-by-step checklist for verifying agent output in live care yet.
• Vendor vs independent: Vendor “rarely hallucinates” claims are marketing until independently confirmed.

• 1

  Rule as the spine — Open and close with: “The model is never the last set of eyes on a medical claim.”

• 2

  Teach the 4 failure modes — Fabrication, source mismatch, silent failure, stale confidence — each with a way to catch it.

• 3

  Hand out the verification prompt — Citation per claim · [VERIFIED]/[UNVERIFIED] · separate vendor vs replicated.

• 4

  Live-demo the check — Run a real clinical question in Copilot Researcher, open the sources, test one citation.

• 5

  Cite the three bodies — ICMJE, WHO, AMA give residents citable authority for why they must verify.

• 6

  Keep a “thin evidence” slide — Agent-specific accuracy data is young and benchmark-dependent — update as tools evolve.

• [1] Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents — arXiv (cs.CL) (Tier 1)
• [2] Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents — arXiv (cs.CL) (Tier 1)
• [3] ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks — arXiv (cs.CL) (Tier 1)
• [4] Fabrication and errors in the bibliographic citations generated by ChatGPT — Scientific Reports 13, 14045 (Tier 1)
• [5] Chain-of-Verification Reduces Hallucination in Large Language Models — arXiv (cs.CL) (Tier 1)
• [6] Detecting hallucinations in large language models using semantic entropy — Nature 630, 625-630 (Tier 1)
• [7] Med-HALT: Medical Domain Hallucination Test for Large Language Models — arXiv (cs.CL) (Tier 1)
• [8] ICMJE Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals — ICMJE (updated Jan 2026) (Tier 1)
• [9] Ethics and governance of artificial intelligence for health: Guidance on large multi-modal models — WHO Publications (Tier 1)
• [10] AMA: AI in Health Care / Augmented Intelligence in Medicine — ama-assn.org (policy hub) (Tier 1)
• [11] LLM Hallucination Leaderboard (Vectara) — Hugging Face space / vectara.com (Tier 2)
• [12] A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions — arXiv (cs.CL) (Tier 1)
• [13] FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation — arXiv (ACL 2023) (Tier 1)
• [14] Evaluating Verifiability in Generative Search Engines — arXiv (EMNLP 2023) (Tier 1)
• [15] DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments — arXiv preprint (cs.CL) (Tier 1)
• [16] Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation — arXiv preprint (cs.CL) (Tier 1)

Synthesized by Gambit (Hermes) from peer-reviewed/preprint sources and clinical-consensus bodies. Sources verified live. Confidence Medium-High.