RAG vs scripted chatbots: why citations matter in support

Support chatbots have failed customers in two distinct ways over the last decade, and understanding both explains why retrieval-augmented generation has become the default architecture for this problem.

The first generation was scripted. A decision tree, a set of keyword rules, a menu of buttons. These bots were predictable and safe, which is genuinely valuable — they never invented a refund policy. But they only worked when the customer’s question happened to match a branch someone had anticipated, and the failure mode was a loop that ended in "let me connect you to an agent" after three minutes of wasted effort.

The second generation swung the other way. Hand a large language model a system prompt and let it answer freely, and it will produce fluent, confident, well-structured responses to anything. It will also, when it does not know your return window, produce a plausible one. In support this is not a quirk; a stated policy sets a customer expectation you may be obliged to honour.

Retrieval-augmented generation sits between the two. Rather than asking the model what it knows, the system first retrieves relevant passages from your own documentation, then asks the model to answer using only those passages. The generation step still handles phrasing, tone and multi-turn context — the things models are genuinely good at — but the facts come from content you wrote and control.

The detail that matters most in practice is the citation. When an answer displays the article it came from, three things become possible: the customer can verify it and read further, your agents can spot a wrong answer by checking the source rather than re-deriving the policy, and you can trace any bad reply to a specific document and fix it there. Without citations you are debugging a black box.

The other half of the design is knowing when to stop. A retrieval system can measure how well the retrieved passages actually match the question, and a well-configured agent escalates to a human when that score is low instead of answering anyway. "I don’t have a confident answer, let me get someone who does" is a good support experience. A confidently wrong policy is not.