Check where the sources come from and how fresh they are

An answer can match a retrieved document perfectly and still be wrong for the workflow, because that document is stale, incomplete or has been replaced. So look at the whole source path: what goes into the collection, who can change it, how long changes take to show up and how the feature shows where an answer came from.

For the release boundary, make a source inventory. For each source, record the owner, who's allowed to see it, how it gets updated, how fresh it's expected to be, its version fields and how things get removed. If several collections hold the same policy or product information, decide which one wins. You don't want the workflow quietly using whichever copy ranks highest.

At ingestion time, keep stable source identifiers, the version or modification time, access metadata and enough location detail to reopen the original. Normalise the text without throwing away headings, page references or record keys a reviewer will need. And watch for failed or partial ingestion, since a healthy job count can still hide a document that produced nothing usable.

In the review interface, show provenance. Where the task needs verification, that means the source title, the relevant passage and its version, with links that open the authorised original. A citation the model generated is only worth something once the application has confirmed the referenced source exists and was actually available in that run.

Decide what happens when sources disagree. The workflow might show both versions, apply a documented precedence rule or send the case for review. That choice belongs in the task definition and the evaluation set. Asking the model to work out which document "looks current" gives you an answer that's hard to defend later.

Before release, I'd check that:

• A recently changed item shows up within the expected update period.
• A removed or restricted item no longer appears for users who shouldn't see it.
• Duplicate and superseded documents follow the intended precedence.
• An empty, corrupt or unsupported file causes a visible ingestion failure.
• Every displayed citation resolves to the source version that was used.
• A source outage gives a contained response instead of an unsupported answer.

Someone needs to own the collection and be able to see ingestion status and fix content without waiting for a code deployment, otherwise stale or conflicting material stays available no matter how much you change the model and prompt.