What belongs in an AI audit log
In short
An AI audit log is complete when someone who was not present can reconstruct a single decision end to end, months later, from the record alone. That requires seven things: the input, the context, the options considered, the policies evaluated, the proposed action, the approver, and what actually happened.
- Published
- Last reviewed
- Reading time
- 9 min
- Written by
- Barion AI
The test that matters
There is a single test for whether an AI audit trail is complete, and it is harsher than most logging setups survive:
Take one decision the system made a month ago. Reconstruct it end to end from the record alone, without asking anyone who was there.
If that is not possible, the audit trail is incomplete, regardless of how much data is being written or how good the dashboards look. Volume is not the same as reconstructability.
Most systems fail this test in the same place. They log what the system did and skip what the system knew and considered. Six months later, an output without its inputs is not evidence. It is an assertion with a timestamp.
The seven fields
A record that survives the test contains seven things. The order matters, because it mirrors the sequence of the decision itself.
1. What the system received
The inputs and context as they were at decision time, not as they look now.
This is the field most often got wrong. Logging a reference to a record is not the same as logging the record, because the underlying data will have changed by the time anyone reviews it. A customer's risk tier, a patient's medication list, a position size: all of these move. If the log points at live data, the reconstruction shows the decision being made on information that did not exist yet.
Store the resolved values, or a content hash plus a versioned snapshot.
2. What it concluded
The options the system generated and the ranking it produced — including the options it rejected.
Rejected candidates are more diagnostic than the winner. When a system starts behaving oddly, the useful question is usually not "why did it pick that?" but "what else was on the table, and why did it lose?"
3. Which rules were applied
Every policy evaluated and its result, including the ones that passed silently.
This is the second most commonly skipped field, because passing checks feel like non-events. They are not. "The value ceiling was evaluated and passed" and "the value ceiling was never evaluated" produce identical outputs and entirely different post-mortems.
4. What it proposed
The action put forward and the reasoning offered for it, in the form a human would have seen.
5. Who approved it
The person, the authority under which they approved, and the timestamp. Not a service account. Not "the system". A named human with the standing to say yes.
If your approval step records only that approval occurred, you have logged a formality rather than an accountability.
6. What actually happened
The action taken and its observed effect on the systems it touched. Proposed and executed are different events, and the gap between them is where a surprising number of incidents live.
7. Whether the outcome matched expectation
The comparison that turns a log into learning. Without it there is no way to know whether decision quality is holding, and calibration becomes a claim rather than a measurement.
A minimal field specification
| Field | Type | Why it exists |
|---|---|---|
decision_id | uuid | Correlates every downstream event |
occurred_at | timestamp, UTC | Ordering, and the anchor for reconstruction |
inputs | resolved values or content hash | Prevents drift between log and live data |
context_sources | list of source + version | Shows what was in view, and what was not |
candidates | ordered list with scores | Rejected options are the diagnostic ones |
policies_evaluated | list of rule id + result | Passing checks are events too |
proposed_action | structured action + rationale | What a human was asked to approve |
approver | person id + authority + timestamp | Accountability, not formality |
executed_action | structured action + result | Proposed and executed are different |
outcome | observed result, written later | Closes the loop for calibration |
model_version | identifier | Behaviour changes when this changes |
policy_version | identifier | So do the boundaries |
The last two fields are cheap to add and disproportionately valuable. When behaviour shifts, the first question is always what changed, and version stamps answer it in seconds rather than days.
Retention is part of the design
An audit trail that expires before the review that needs it has failed. The retention period is not a storage decision, it is set by the longest realistic review horizon:
- Markets. A position may be reviewed at quarter end or during an audit months later.
- Clinical. Case review can happen years after the appointment.
- Operations. Usually shorter, but incident investigations reach back further than people expect.
Decide the horizon deliberately, with the people who would actually run the review, and write it down. "Whatever the default log retention is" is not a decision.
Where this sits in the architecture
Logging is not a stage at the end of the pipeline. In Barion Core it runs across all four layers, because a record assembled after the fact is a reconstruction, and a record written at decision time is evidence.
The practical consequence is that the log is designed alongside the decision model, not retrofitted once the system works. Retrofitting is how the "what it knew" fields go missing: by the time anyone looks, the context that would have been logged has already been discarded.
If you are working out what your own control model can and cannot currently evidence, the governance and observability page walks through the wider set of controls this sits inside.