Build with Barion / 04
CONTROL IS PARTOF THE PRODUCT.
If something is already running and nobody can say what it received, what it concluded, which rules applied or who approved the result, that is not a documentation gap. It is a missing part of the system.
- Discipline
- Engineering
- Not
- A policy document
- Output
- An inspectable control model
Practical controls
Twelve mechanisms, all of them testable
Guardrails are only real if they are enforced outside the model, where a persuasive input cannot negotiate with them.
- 01Input validation
- Checking what arrives before it reaches reasoning. Malformed, stale or out-of-range input is rejected rather than interpreted.
- 02Output constraints
- Hard limits on what the system may emit, evaluated after generation and before delivery.
- 03Permissions
- What the system may read, propose and do, held in configuration rather than described in a prompt.
- 04Action boundaries
- Access granted per tool and per operation, so a system cannot reach something it has no business touching.
- 05Confidence thresholds
- A floor below which output is not published or acted upon, but escalated instead.
- 06Approval workflows
- A required signature routed to the person holding authority, with what they approved recorded.
- 07Escalation
- A defined route to a named human when confidence is low, a constraint binds, or the case is unmapped.
- 08Audit logs
- An evidence trail readable by someone who was not present, retained long enough to be useful in review.
- 09Performance monitoring
- Latency, error rates and throughput, so degradation is noticed before users report it.
- 10Drift detection
- Watching whether output quality is moving relative to a baseline, because the world changes under a model.
- 11Failure handling
- Defined behaviour when a dependency is unavailable or data is missing. The default is to stop and report.
- 12Human override
- A person can stop, reverse or take back the process at any point, without needing engineering help.
Guardrail matrix
The same five questions in three domains
What is restricted, what needs approval, what gets logged, what triggers escalation, and who owns it. The answers differ by domain; the questions do not.
| Domain | Restricted action | Requires approval | Logged evidence | Escalation trigger | Human owner |
|---|---|---|---|---|---|
| MarketsHow PSX Invest behaves | Placing or routing an order | Every entry and exit, without exception | The signal, its indicators, entry, target and stop | Confidence below the publication threshold | The trader holding the position |
| ClinicalIllustrative | Any diagnosis or prescription | Every recommendation reaching a patient record | Context used, options ranked, reasoning shown | Contraindication detected or context incomplete | The treating clinician |
| OperationsIllustrative | Irreversible changes and external communications | Actions above a defined value or blast radius | Trigger, tools touched, action taken, result | Retry limit reached or an unmapped exception | The named workflow owner on shift |
Markets
Operating- Restricted action
- Placing or routing an order
- Requires approval
- Every entry and exit, without exception
- Logged evidence
- The signal, its indicators, entry, target and stop
- Escalation trigger
- Confidence below the publication threshold
- Human owner
- The trader holding the position
Clinical
Illustrative- Restricted action
- Any diagnosis or prescription
- Requires approval
- Every recommendation reaching a patient record
- Logged evidence
- Context used, options ranked, reasoning shown
- Escalation trigger
- Contraindication detected or context incomplete
- Human owner
- The treating clinician
Operations
Illustrative- Restricted action
- Irreversible changes and external communications
- Requires approval
- Actions above a defined value or blast radius
- Logged evidence
- Trigger, tools touched, action taken, result
- Escalation trigger
- Retry limit reached or an unmapped exception
- Human owner
- The named workflow owner on shift
Observability
What an operator should be able to see
If a question below cannot be answered for a decision your system made last month, that is the gap worth closing first.
- 01What the system received
- The inputs and context as they were at decision time, not as they look now.
- 02What it concluded
- The options generated and the ranking it produced.
- 03Which rules were applied
- The policies evaluated and their results, including the ones that passed silently.
- 04What it proposed
- The action put forward, and the reasoning offered for it.
- 05Who approved it
- The person, their authority and the timestamp.
- 06What actually happened
- The action taken and its effect on the systems it touched.
- 07Whether the outcome matched expectation
- The comparison that turns a log into learning, and the only way to know if quality is holding.
Who does what
The split
We build the controls and the evidence trail. You define the rules and who holds authority, because those are business decisions rather than engineering ones.
Barion AI contributes
- Input validation and output constraints
- Permission and action boundaries
- Confidence thresholds and approval routing
- Audit logging and evidence trails
- Performance monitoring and drift detection
- Failure handling and human override
You contribute
- The rules the system must respect
- Who holds authority for each action
- Existing audit and reporting obligations
- Access to production behaviour
These controls are not bolted on per engagement. They are the cross-cutting layer of Barion Core, which every product inherits.
Review your AI control model
Bring a system that is already running. We will work through what it can and cannot currently evidence, and what closing that gap would involve.