Should you build a chatbot on your own documents

When a chatbot answering from your own documents is worth building, what the document cleanup really costs and the mistake that produces wrong answers.

The pile of documents and the same ten questions

The appeal is easy to understand. You hold manuals, policy notes, price lists, installation guides and a year of email answers. The same questions arrive every week and a person answers each one by finding the right paragraph. A chatbot built on those documents promises to do the finding.

What such a system actually does is narrower than the pitch. It searches your material for passages that look relevant to the question and then writes an answer out of those passages. It is a retrieval system with a writer attached. That distinction decides everything that follows, because the quality of the answer is governed by the quality and the currency of what it retrieved.

What it costs beyond the build

Document preparation is excluded from most quotes and it is the largest cost. Scanned pages without text, tables that lose their structure, five versions of the same price list and policies that contradict each other all have to be resolved by a person who knows which one is right. That work is yours and it does not get cheaper later.

Then there is a monthly cost that scales with use rather than a fixed licence, because every question consumes processing. Add hosting and a place for the chat widget to live. GST at 18% applies on service fees. The ongoing line nobody budgets is corpus maintenance. Each time a price or a policy changes someone must update the source document, or the bot keeps giving the old answer with complete confidence.

Who should not build one

Do not build this if your question volume is low. A business fielding a handful of repeat queries a day will spend more on maintenance than it saves in answering. A well written frequently asked questions page and a saved reply template will serve that volume better and cost almost nothing to keep accurate.

Do not build it if your documents are out of date or inconsistent. The system cannot tell a superseded rate card from the current one. It will retrieve the wrong page and write a fluent answer around it. Fixing the documents is the project and the chatbot is an optional extra afterwards.

Do not build it where the answer carries commitment. Pricing promises, warranty scope, insurance cover, employment terms and anything a regulator reads should not be generated. For those, a bot that locates the clause and hands the conversation to a named person is both safer and cheaper than one that answers.

It suits a different profile well. A stable and reasonably clean body of material, a high volume of genuine lookup questions and an internal audience to start with.

How these systems are actually assembled

Documents are split into passages and indexed so the system can match a question to likely passages. On each question it retrieves a handful of them and composes an answer from that text. The single most important configuration choice is requiring every answer to cite the document and section it came from so a reader can check it in one click.

The conversation design is separate work from the retrieval build and it is where most deployments fail. Deciding which journeys the bot owns, what it says when retrieval finds nothing, how a user reaches a human and what the handover carries with it is the subject of chatbot journey mapping rather than of the index itself. Rates for that are published on the service page.

Run it internally first. Your own staff will report wrong answers plainly and at no reputational cost. Keep a log of every question asked. That log is the most valuable output of the first month because it shows which documents are missing and which questions your material never answered in the first place.

The measure that matters

Judge it on resolved and correct. Take a sample of conversations each week and ask two things of each. Did the user get an answer and did they then contact a person anyway. A bot that answers and gets followed by a phone call has added a step rather than removed one.

The flattering metric is conversation volume. Chat counts rise simply because a widget is easier to click than an email is to write. Deflection rate on its own flatters just as badly, since an abandoned conversation and a satisfied one look identical in the log unless somebody reads them.

The mistake that produces wrong answers

Almost everyone points it at everything. The whole shared drive goes in on the reasoning that more context must be better. Retrieval then has to choose between dozens of near duplicates and it starts surfacing drafts, internal notes and last year's terms.

The cost is concentrated rather than spread. One customer quoted a withdrawn price, one applicant given a superseded policy, one warranty claim argued from text you no longer honour. Any single instance can exceed a year of the saving. Begin with the smallest set of documents that answers your top questions and add only what the question log proves is missing.

This guide explains how something works. It is not legal, tax or financial advice, and rules change. Confirm anything regulatory with a qualified professional before you act on it.