A support widget that invents an answer does not fail loudly. It fails four days later, when a prospect quotes it back to your sales team and discovers that the integration it promised does not exist. By then the cost is not one wrong answer — it is every other answer that visitor now has to re-evaluate.
Why the default behaviour is to invent
A language model produces the most plausible continuation of a conversation. Nothing in that objective distinguishes “plausible and supported” from “plausible and fabricated”. Asked whether your product supports single sign-on, a model that has never seen a document mentioning SSO will still produce a fluent answer, because a fluent answer is what the shape of the conversation calls for.
This is not a bug to be prompted away. It is what the system does. The fix is architectural: do not let the model answer from its own parameters at all.
The question is not whether the model knows. It is whether your content said so, and whether the answer can point at where.
The three constraints that make it safe
1. Answer only from retrieved text
The model receives passages retrieved from your own content and is instructed to answer using those and nothing else. Its general knowledge becomes a language capability rather than a source of facts. Anything outside the retrieved set is, by construction, unanswerable.
2. Require a citation for every claim
Not a footnote at the bottom — a link on the claim, pointing at the page the passage came from. This does two things at once: the visitor can check, and the requirement itself constrains the model, because a claim with nothing to cite is much harder to produce.
3. Refuse when retrieval comes back empty
The most important path, and the one most implementations skip. If nothing relevant is retrieved above a similarity threshold, the correct response is not a hedged answer. It is: we do not have that documented, here is a person who can tell you.
Refusal rate is the useful metric
Most widget dashboards lead with resolution rate, which rewards exactly the behaviour you are trying to prevent — a widget that answers everything scores best, whether or not the answers were true.
| Metric | What it tells you | Failure it hides |
|---|---|---|
| Answer rate | How often something was produced | Everything. A hallucinating widget scores 100%. |
| Refusal rate | How often your content had no answer | Nothing — this is the number you want. |
| Refused questions, grouped | Exactly which documentation to write next | Nothing. This is the most valuable output. |
| Refusal-to-human rate | Whether the handover works | Whether people accepted the handover or just left. |
| Citation click-through | Whether people check, and whether the pages hold up | Whether citations are relevant or merely present. |
The by-product worth more than the answers
Every refusal is a question a real prospect asked that your site cannot answer, recorded in their phrasing rather than your own. Grouped over a month, that is a documentation backlog written by your market, ranked by frequency, for free.
- Log every refusal with the question verbatim and the page it was asked from.
- Group them weekly. Anything asked more than three times is a page you owe.
- Write the page, then confirm the refusal disappears — that is the loop closing.
- Watch for questions your content answers badly rather than not at all: those show up as answers with weak or irrelevant citations.
- Never tune the threshold to reduce refusals. That converts a visible gap into an invisible fabrication.
Where identity fits, and where it does not
A widget that demands an email before answering is optimising for the wrong half of the exchange. The visitor has a question; you have an answer or you do not. Asking for identity first tells them that the answer is the price of their contact details, and the usual response is to close the tab.
Ask afterwards, when there is a reason: a refusal that needs a human, a request for something specific, a follow-up they want sent. By then the widget has demonstrated it is worth talking to, which is a much better position to ask from.
That is how Chat is built — grounded in your content, citing what it used, and refusing rather than guessing. It writes what it learns to the same visitor record Analytics keeps, so a question asked at 11pm is still attached to the person when they come back and book. The consent rules that govern that record are on the evidence standard page.