When not to use an AI agent
In short
An AI agent is the wrong tool when ownership of the workflow is undefined, the underlying data is unreliable, actions are irreversible without approval, permissions cannot be bounded, nobody would ever read the audit log, or the process is not understood well enough to write down. Each of these turns automation into an amplifier for an existing problem.
- Published
- Last reviewed
- Reading time
- 8 min
- Written by
- Barion AI
The failure mode nobody plans for
The usual worry about agents is that one will do something dramatic and wrong. That happens, and it is recoverable, because it is visible.
The more common and more expensive failure is quieter. An agent is pointed at a process that nobody fully understands, it runs correctly according to its own logic, and it produces thousands of small changes that are individually plausible and collectively untraceable. Nothing alarms. The problem surfaces weeks later, and by then the work of unpicking it costs more than the process ever cost to run by hand.
Automation is a multiplier. Pointed at a well-understood process it multiplies throughput. Pointed at a confused one it multiplies the confusion.
Six conditions that should stop you
Each of these is a reason to fix something first, not to abandon the idea.
1. Ownership of the workflow is undefined
If you cannot name the person who is accountable when the workflow produces a wrong result, an agent will not fill that gap. It will obscure it.
The question to ask is not "who runs this process?" but "who gets called when it goes wrong?" If the honest answer is "it depends" or "we would work it out", that ambiguity is currently absorbed by humans applying judgement. An agent has no judgement to apply, and no discretion to escalate to a person who has never been named.
2. The underlying data is unreliable
Agents act on what they read. If the data is stale, incomplete, or contradicted by a second system that also claims to be authoritative, the agent will act confidently on whichever version it happened to see.
A person doing this work develops an instinct for when a record looks wrong and goes to check. That instinct is doing far more load-bearing work than anyone documents, and it does not transfer.
3. Actions are irreversible and cannot wait for approval
Autonomy should be governed by reversibility:
How much a system may do alone is bounded by how easily its actions can be undone.
An action that can be reversed in one step tolerates a wide autonomy envelope. An action that sends money, contacts a customer, deletes a record, or changes something outside your control does not. If such an action genuinely cannot tolerate the latency of an approval gate, the workflow is not a candidate yet.
4. Permission boundaries cannot be drawn
You should be able to state, before anything runs, what the agent may read, what it may write, what it may send, and what it must never touch.
If access to the systems involved is all-or-nothing, the agent inherits far more authority than the task requires. "It only needs to update one field" is not a boundary if the credential it holds can update every field.
5. Nobody would ever read the audit log
An audit trail that no one reviews is storage cost, not control. If there is no review process, no periodic sampling, and nobody whose job includes looking, then the log exists to make people feel comfortable rather than to catch anything.
Ask directly: who reads this, how often, and what would they do about what they found? If those questions have no answers, build the review process before the agent.
6. The process is not understood well enough to write down
This is the condition that disqualifies the most candidates.
If the people who do the work cannot describe the decision rules without reaching for "it depends on the situation", the process contains undocumented judgement. That judgement is real and valuable. It is also invisible, and an agent will replace it with something confident and different.
Writing the process down is worth doing whether or not you ever automate it.
Readiness, as a table
The inverse of the list above. The more of these that are true, the better this goes.
| Condition | Ready | Not ready |
|---|---|---|
| Trigger | A specific, observable event | "When someone notices" |
| Data | Reachable programmatically, reliable | Scattered, stale or contradicted |
| Tools | Known, access grantable per operation | All-or-nothing credentials |
| Decisions | Follow a pattern you can write down | "It depends on the situation" |
| Permissions | You can state what it must never do | Boundaries never discussed |
| Exceptions | Known failure modes, named responses | Unknown unknowns |
| Escalation | A named person per failure class | "Someone on the team" |
| Outcome | You can tell afterwards if it was right | No measurable definition of correct |
Four or more in the right-hand column means the groundwork is the project, and the agent comes after it.
What to do instead, in order
- Write the process down. Triggers, decision rules, exceptions, and who currently decides. This alone frequently reveals that two people do the same job differently.
- Name the owner and the escalation path. Before any tooling.
- Fix the data reliability problem, or scope the agent to the subset of data that is trustworthy.
- Draw the permission boundary, and confirm your systems can actually enforce it at that granularity.
- Instrument the manual process so you have a baseline. Without one you cannot tell whether the agent improved anything.
- Then automate the narrowest useful slice, with an approval gate on anything consequential.
The distinction that actually matters
An assistant suggests and a person acts. A copilot helps while a person drives. Workflow automation follows fixed rules and cannot deviate. A controlled agent evaluates context and acts within policy.
Only the last of these carries real risk, because only the last takes actions the operator did not individually authorise. Everything on this page is about earning the right to that autonomy rather than assuming it.
The agentic operations page covers the assessment in more depth, and Barion Agents describes the control model the product is being built around. It is not available yet, which is stated plainly there rather than softened.