Hallucination is the failure that ends AI projects. Not because the model is bad, because nobody tested what happens when it doesn’t know the answer. The model fills the gap with something plausible, the customer believes it, and the screenshot ends up on social media.
The fix is a testing routine, not a better model. This playbook gives you the hallucination test framework: golden examples, regression tests, and citation checks.
Core idea: An agent’s reliability is measured by how it behaves on questions it can’t answer, not on the ones it can. Test for that behavior deliberately.
Ground the test boundary in AI agent requirements.
What Counts as a Hallucination
Hallucination is any answer not grounded in your approved knowledge base:
| Type | Example |
|---|---|
| Invented fact | Product feature that doesn’t exist |
| Wrong spec | “2-week free trial” when your trial is 7 days |
| Wrong policy | “You can return within 60 days” when it’s 14 |
| Invented capability | “Yes, we integrate with that tool” (you don’t) |
| Leaked instruction | Quoting its system prompt |
| Confident hedging | “Generally, most companies…” when your docs say otherwise |
Grounding patterns from AI agent governance become useful only when you test them.
Build cases from AI customer support agent conversations.
For this section, review AI customer support agent.
The Test Case Library
Build three test tiers:
| Test type | Input | Expected behavior |
|---|---|---|
| Golden examples | Questions your support team actually gets | Correct, cited answers |
| Near-miss | Similar but not identical to your product | Agent says “I don’t have that” |
| Adversarial | Oddly phrased, indirect, or misleading questions | Asks for clarification or escalates |
| Off-knowledge | Topics not in your knowledge base | Honest “I don’t know” + escalation |
| Adversarial injection | Prompt injection tests | Doesn’t leak instructions |
Start with golden examples from real support tickets.
Make Agent Reliability Measurable
Talk with Botsify about practical testing and monitoring.
Hallucination tests become more useful when they reuse AI agent test cases from real support tickets.
Verify AI answer citations and record results in an AI agent monitoring routine.
Golden examples fit naturally into the AI agent implementation checklist.
The Golden Example Method
For each of your top 30 customer questions:
- Write the question exactly as customers ask it (including typos and vague phrasing)
- Write the correct answer, grounded in a specific source document
- Note the source so you can verify the citation
- Run the test and compare the answer to the golden version
- Log any mismatch in a shared sheet with severity
Re-run the golden set after every prompt, knowledge, or integration change. This becomes the regression suite for testing an AI agent before publishing.
The fallback should follow clear human handoff rules.
Testing the “I Don’t Know” Path
The most important test isn’t whether the agent answers correctly, it’s whether it admits uncertainty:
- Ask about a product feature you don’t have
- Ask about a policy that doesn’t exist
- Ask a question that’s genuinely ambiguous
- Ask about your competitor’s product
Expected behavior is a clear limitation, an offer to connect a human, and no plausible invention. If the agent fills the gap, strengthen the escalation path instead of guessing.
Reduce Hallucination Risk with Botsify
Build grounded agent workflows with testing and human escalation.
Verify source behavior with an AI FAQ knowledge base.
Citation Verification
If your agent supports citations (and it should, see AI answer citations), verify:
- Every factual answer links to a source document
- The cited document actually contains the answer
- The citation UI is visible to the customer, not just internal
Uncited answers are where hallucinations hide. If your agent answers without grounding, treat it as a failure even if the answer happens to be right.
Block quote to steal: “An uncited answer is not an answer, it’s a guess with confidence. Citations are how you tell the difference.”
Keep regression testing aligned with the AI agent lifecycle.
The Regression Test Suite
Every change to your agent, prompt, knowledge, integration, or model, should trigger a re-run:
| Change | Re-run |
|---|---|
| New prompt version | Golden examples + adversarial suite |
| Knowledge update | Affected golden examples |
| New integration | Integration-specific tests |
| Model change | Everything |
| Channel addition | Channel-specific edge cases |
Log every test run with pass/fail results. This supports production-ready AI agent decisions and a repeatable AI agent QA checklist.
Track hallucination outcomes alongside support metrics beyond deflection.
Measuring Hallucination Rate
| Metric | How to measure |
|---|---|
| Grounding rate | % of answers traceable to a knowledge base source |
| Golden example pass rate | % of test cases matching expected answers |
| “I don’t know” rate | Should be >0%, an agent that never says this is a red flag |
| Hallucination reports | Customer-flagged wrong answers per week |
Track these in the weekly AI agent monitoring after deployment review. A “0% I-don’t-know rate” may mean the agent never admits uncertainty.
Review AI agent governance when test results change.
FAQ
Can hallucinations be eliminated? Reduced dramatically, not eliminated. The levers are grounded knowledge, citations, generous escalation, and adversarial testing. Continue with the AI hallucination reduction playbook.
How often should I run the test suite? After every change, plus a monthly full pass. Changes are the risk moments, especially after an AI agent knowledge update.
What’s the fastest way to find hallucinations? Ask about things that don’t exist: a product feature you never launched, a plan you never offered, a policy you never wrote. The agent’s behavior on those questions tells you everything about its grounding.
Test Agents Before Customers Do
Plan a safer rollout for support, sales, or service automation.
Avoid common AI agent deployment mistakes before widening access.
Final Thought
Hallucination testing isn’t about catching AI being stupid, it’s about building confidence in what it says. A tested agent with citations, golden examples, and a regression suite is one you can put in front of customers. An untested one is a liability waiting for a screenshot. Build the test suite once and rerun it after every change, that’s the whole discipline.
Use a clear AI agent requirements document to define which answers require grounding and which cases must escalate.
Review hallucination results alongside AI agent governance, AI agent lifecycle, and deployment-mistake reviews before expanding the workflow.
Connect the test suite to AI agent integration and AI customer support agent checks before expanding access.