30 April 2026 / Applied AI / 8 chapters

Put review and correction into the product

From Designing an auditable AI capability assessment

Reproducibility matters because people need a practical correction path. Show the respondent or authorised reviewer the accepted answer values before final scoring where the process allows it. Use plain labels and preserve the source response nearby.

For each result, expose:

  • the methodology version and assessment date;
  • the dimensions or rules that contributed;
  • the evidence references used;
  • missing or uncertain items;
  • the meaning and limits of the result;
  • the route for requesting a correction.

A correction should identify the answer being changed, old and new values, reason, actor and supporting evidence. Recalculate under the same methodology version unless the correction process explicitly authorises a version change. Keep both result records and mark which one is current.

Before publication, conduct four distinct reviews. A methodology review checks the questions, evidence standard, rules and interpretation. A technical review checks validation, calculation and access control. A content review checks that the conversation and report do not overstate the method. A privacy review checks collection, retention, export and deletion behaviour.

The release check should include these questions:

  • Can a reviewer reproduce every fixture without a model call?
  • Can the system distinguish changed evidence from changed rules?
  • Does every displayed explanation resolve to reason codes and confirmed inputs?
  • Are missing and not-applicable values handled as the methodology specifies?
  • Can a respondent correct a mapping without restarting the interview?
  • Does recalculation retain the original result and reason for change?
  • Are raw responses and evidence protected by suitable access and retention rules?
  • Can support trace a report back to one assessment run without exposing unrelated records?

Run a final dry case with an ambiguous answer, a corrected mapping, one unknown item and a boundary value. Request the audit export and give it to an authorised reviewer. Controlled use can begin once that reviewer can reproduce the score from the export and explain each contributing rule without another model call.

All articles