Test the whole configured workflow

If you test the model on its own, most of the path a customer goes through is still untested. The deployed feature might add document retrieval, prompt assembly, tool calls, output parsing, business rules and a user interface, and any of those can change the result.

So run the cases through the same path you plan to release. Use the planned model version and parameters, load the same source index and apply the real permissions. Exercise the parser and post-processing code. If the interface truncates source excerpts or transforms the response, that should be part of what gets reviewed.

Capture a run record so results can be compared later:

• evaluation set version
• application and prompt version
• model identifier and settings
• source snapshot or index version
• tool configuration
• case ID
• raw model response
• parsed or displayed output
• tool calls and errors
• elapsed time and review result

Where output can vary, repeat the cases. How many repetitions you need depends on how consequential and how variable the task is. Record every attempt, though. If you only keep the best result, you're picking outputs, and the evaluation stops telling you how the workflow behaves.

Test every change against the whole relevant set. A prompt tweak that improves missing-information cases can make ordinary answers overly cautious. A retrieval change that finds more documents can start pulling in stale or unauthorised material. I'd compare results case by case instead of just watching the aggregate score, and look at any critical failure even if the total moved the way you wanted.

The deterministic rules around the model need their own tests as well. Permissions, state transitions, duplicate handling and calculations should have conventional software tests. The generative evaluation sits alongside those and covers the parts where you can't write down a simple expected value.