30 April 2026 / Applied AI / 8 chapters

Log an event for every step that matters

From Designing an auditable AI capability assessment

Keep the chat transcript as source material and build the audit history from structured events attached to the assessment run. That way reviewers aren't trying to piece together what happened by reading a conversation.

Record run creation, methodology version selection, question presentation where you need it, answer submission, the proposed mapping, the respondent's confirmation, any reviewer change, the calculation, report generation and finalisation. Each event gets a stable ID, timestamp, actor, run ID, event type and a payload that suits it.

Use append-only events for the history, and current-state records so the application stays quick. A correction adds a new event and updates the current answer view, and the earlier answer stays in the history. Protect the events with database permissions, controlled write paths and backups. Cryptographic chaining can be worth it in some settings, but I'd sort out access control and event design first.

An audit export should let a reviewer rebuild a result without needing any hidden application state. Put in the methodology definition or its immutable reference, the confirmed input values, any amendments, the calculation output, the reason codes and the report version. Leave out secrets, model credentials and conversation content that has nothing to do with the result.

Sort out correlation IDs early. The run ID should tie together the interview, the evidence records, the calculations, the reports and the operational logs. If a user reports an odd score, support should find the right records without searching by personal data or lining up timestamps by hand.

You might also need to log who accessed the audit material. Assessment data can expose an organisation's weak spots or someone's personal details. Use role-based access, scoped exports and retention rules, and test that reviewers only see the runs they're authorised for.

All articles