Article chapter 05 of 08
Retrieve evidence under the user's access rules
Treat retrieval as an application request with an authenticated user, an authorised collection, a query representation, ranking rules and a limit on what enters the model context. Similarity search is one part of that request.
Apply access filtering before passages are exposed to the model. The retrieval service should return only chunks the current user may read. Post-filtering a mixed result set can also produce poor answers because restricted passages occupy the top positions and leave too little permitted evidence. Where the search technology supports it, include permission attributes in the query filter.
Semantic vector search is useful when the question and source use different wording. Keyword search remains useful for exact identifiers, product names, policy numbers and uncommon terms. A combined approach can retrieve candidates from both, then rank them using consistent rules. The correct balance depends on the question set and source language, so treat it as an evaluated configuration rather than a universal formula.
Query preparation should be restrained. Spelling normalisation, acronym expansion and known terminology mappings can help. Automatic query rewriting can also change the user's intent. Retain the original query, record the rewritten form and evaluate cases where qualifiers such as date, region or account type affect the answer.
Ranking can use metadata as well as textual relevance. Current published material may rank above drafts. A source for the user's region may outrank a general copy. Recency is useful only when newer really means applicable. An older contract or policy may still govern a historical question, so date filtering should follow the user's task rather than a blanket preference.
Set diversity rules where repeated chunks from one document displace complementary evidence. The model may need a procedure and its exception from different sections. Retrieve enough candidates to rank well, then pass a smaller, inspectable set into generation. Log the selected source identifiers, versions and scores for later diagnosis.
Define behaviour for weak retrieval. A low score does not translate cleanly into "no answer" across every collection. Calibrate thresholds and evidence checks against evaluation cases. If the selected passages do not support the requested claim, return the relevant material found or state that the source set does not answer the question. Searching more widely should be an explicit product decision, especially where wider collections have different permissions.