Check source quality and provenance

An answer can be faithful to a retrieved document and still be wrong for the workflow because the document is stale, incomplete or superseded. Review the full source path: what enters the collection, who can change it, how quickly changes become available and how the feature shows where an answer came from.

Create a source inventory for the release boundary. Record the owner, permitted audience, update method, expected freshness, document version fields and removal process. If several collections contain the same policy or product information, decide which source takes precedence. The workflow should not silently choose whichever copy ranks highest.

At ingestion time, retain stable source identifiers, version or modification time, access metadata and enough location detail to reopen the original. Normalise text without discarding headings, page references or record keys that a reviewer needs. Detect failed and partial ingestion. A successful job count can still hide a document that produced no usable content.

Expose provenance in the review interface. Show the source title, relevant passage and its version where the task requires verification. Links should open the authorised original. A citation generated by the model is only useful after the application confirms that the referenced source exists and was actually available in the run.

Decide what the workflow does when sources disagree. It may present both versions, apply a documented precedence rule or refer the case for review. The choice belongs in the task definition and evaluation set. Asking the model to infer which document "looks current" produces an answer that is difficult to defend later.

Source checks before release should include:

• a recently changed item appears within the expected update period;
• a removed or restricted item no longer appears for unauthorised users;
• duplicate and superseded documents follow the intended precedence;
• an empty, corrupt or unsupported file creates a visible ingestion failure;
• every displayed citation resolves to the source version used;
• a source outage produces a contained response instead of an unsupported answer.

Assign ownership for the collection. The owner needs a way to inspect ingestion status and correct content without waiting for a code deployment. Otherwise stale or conflicting material remains available regardless of model and prompt changes.