Define the job before building the index

Retrieval-augmented generation, usually shortened to RAG, gives a language model selected source material with the user's question. The model answers using that material rather than relying only on what it learned during training. This can make a useful demonstration quickly. A folder of tidy documents, a vector index and a prompt may be enough to answer a handful of prepared questions.

Production work needs a narrower definition of success. Start with the people using the system, the decision or task they are trying to complete, and what they will do with an answer. A support assistant that helps staff locate a procedure has different requirements from a public answer service or a tool used to draft regulated advice. The same retrieval technique can sit inside all three, but the acceptable sources, response time, review and failure behaviour will differ.

Collect real question types without copying private content into an unprotected test set. Categories might include finding a stated rule, combining facts from two sections, locating the current form, explaining a process, comparing versions and identifying that the material does not contain an answer. Write down what evidence a good response must expose. If a user needs to verify the result, the answer should point to the relevant source and location in terms they can recognise.

Define the boundaries in plain language:

• which source collections are included;
• which users or roles may query each collection;
• what kinds of answer the system may produce;
• whether the answer is informational, a draft or an action trigger;
• which questions require a person or another system;
• how the interface behaves when evidence is weak or absent.

Writing those boundaries down stops "chat with company knowledge" from becoming the requirement. Account access, public answers and internal drafting may share retrieval components, but they are different products with different risks.

Choose an initial slice where source ownership is known and answers can be checked. A single maintained procedure library is easier to govern than a crawl across drives, inboxes and collaboration tools. Expansion can follow once the team can show that access, freshness, citations and evaluation continue to work as the collection changes.

Start the design with an answer contract. State what the user can expect, which evidence will be shown, and when the system will decline. Use it to guide ingestion, retrieval and interface decisions before selecting a model.