Review production state after launch

After launch, sources will keep changing, permissions may drift and review queues may grow. Schedule the first operational review while the release details are still fresh.

Inspect actual task mix against the evaluated boundary. Users may submit requests the team did not anticipate, or use an approved feature as a shortcut for a more consequential decision. Add routing or interface constraints where instructions alone are being ignored. Unsupported tasks should be visible in the data so they can be declined or deliberately assessed.

Review corrections and overrides by reason. Repeated source changes point to content maintenance. Frequent schema failures point to runtime validation or prompt issues. Long review times may mean the interface withholds evidence. Treat these as different operating problems, even if all of them appear to the user as an unsatisfactory answer.

Reconcile a sample of external actions against the target system. Confirm that local confirmed state matches the receiving record and that retries have not created duplicates. Check the oldest unresolved run and make sure its owner and next action are still accurate.

Repeat access and source-removal tests after permission or collection changes. Rerun the release evaluation when the model, prompt, tools, retrieval logic or policy changes. Keep the earlier results tied to the earlier configuration rather than overwriting them.

The review should finish with concrete decisions:

• keep, narrow or expand the supported task boundary;
• repair specific controls before changing volume;
• add distinct failures to the evaluation set;
• update source or support ownership;
• pause any action whose state cannot be detected and recovered.

Before automating a consequential step, use one retained run to establish what changed and choose a safe recovery action. If the team cannot do that, keep the step under manual control until the records and recovery path are fixed.