Validating AI output before it reaches anyone

The step between "the model answered" and "we used it"

Most systems treat model output as finished work. It is raw material.

Between generation and delivery there should be a validation step that decides whether this particular output is fit to leave the system. Not whether it is good — that is a harder question — but whether it is well-formed, within range, internally consistent and consistent with its inputs.

Skipping that step is why systems that work in testing fail in production. In testing, the output is nearly always well-formed. In production, a fraction of a percent is not, and that fraction lands somewhere it should not.

Four layers, cheapest first

Validation is not one check. It is a sequence, ordered so the cheap checks reject the obvious cases before the expensive ones run.

Run them in that order. A response that fails structural validation should never reach a grounding check, because there is nothing coherent to check.

Structural

Force structured output and validate it against a schema. This is the foundation, and it is what makes every subsequent layer possible.

The important decision is what to do with a schema failure. There are two defensible options and one bad one:

• Reject and retry, with the validation error fed back. Reasonable, bounded by a retry limit.
• Reject and escalate. Correct when retries are exhausted or the operation is consequential.
• Repair by parsing what you can. This is the bad one. Extracting fields from partially malformed output produces a result that looks valid and carries no signal that anything went wrong.

Range

Every numeric field has a plausible range and every categorical field has a known set. Check them.

This catches a specific and common failure: a value that is correctly typed and completely wrong. A confidence of 1.4, a date in 1970, a quantity of negative seven, a category that does not exist in your taxonomy. These pass a schema check and fail reality.

Consistency

Fields that relate to each other must actually relate correctly.

For a trading signal: on a long, the stop must sit below the entry and the target above it, and the implied reward-to-risk must clear the published floor. For a scheduling decision: the end must follow the start, and the assignee must hold the relevant capability. For an allocation: the parts must sum to the whole.

These are the checks most often missing, because each is domain-specific and has to be written by hand. They are also where the most embarrassing failures live, since an internally contradictory output is obviously wrong to any human who reads it — which means it will be noticed by a customer rather than by you.

Grounding

Does the output only make claims supported by the context it was given?

This is the hardest layer and the most valuable in retrieval-based systems. A practical version does not need to be sophisticated: check that every entity, figure and citation referenced in the output actually appears in the retrieved context. Anything that does not is unsupported, whether or not it happens to be true.

Unsupported claims should be stripped or the whole output held, depending on how consequential the surface is.

What to do when validation fails

Decide this per check, in advance, and write it down. The options:

1. Retry with the error fed back. Bounded by a limit.
2. Hold for review. The output goes to a human rather than a user.
3. Degrade. Return a reduced but valid result — the ranked list without the commentary, for example.
4. Suppress. Return nothing for this case. Silence is a legitimate output and most systems cannot produce it.
5. Escalate. Route to a named person, with the failed output attached.

What is never acceptable is log and continue. If a validation failure does not change what happens next, the validation is decoration.

The failure rate is a signal

Every validation outcome should be logged, including passes, and the failure rate per check should be visible over time.

That series is one of the most informative health metrics a system produces:

• A rising structural failure rate usually means a model version changed.
• A rising range failure rate usually means inputs have shifted.
• A rising consistency failure rate usually means the task has drifted beyond what the current prompt or model handles well.
• A sudden drop to zero almost always means a check broke, not that quality improved. This one catches people out.

Treat these as drift signals, not just as errors.

Building it, in order

1. Force structured output. Retrofitting a schema onto a system that returns prose is the largest task here and everything else depends on it.
2. Write the schema so it is strict. Optional fields that should be required push failures downstream, where they are more expensive.
3. Add range checks for every numeric and categorical field. Minutes of work, disproportionate return.
4. Write the consistency rules by hand, in the domain's own terms. Nobody can do this for you.
5. Add grounding checks if the system uses retrieval.
6. Choose the failure behaviour per check, from the five options above.
7. Log every outcome, including passes, with model and policy version attached.
8. Alert on rate changes, not just on absolute failures.

Where this sits

Validation lives in the policy layer of Barion Core, between reasoning and delivery. That placement is the point: a check inside the generation step is just more instruction, and an instruction is a request rather than a boundary.

It is also what makes the audit trail meaningful. "Validation ran and passed" and "validation never ran" produce identical output and entirely different post-mortems, which is why passes belong in the record alongside failures.