How to detect model drift before your users do

Nothing broke, and it stopped working

Drift is the failure mode that produces no error. The system stays up, latency is normal, no exception is thrown, and the output slowly stops being as good as it was. By the time someone notices, the question "when did this start?" has no answer, because nobody was measuring.

It has three distinct causes, and they need different responses.

The third is specific to systems built on external models and is the one most teams have no detection for at all, because nothing in their own codebase changed.

Why "monitor your model" is unhelpful advice

The standard recommendation is to watch accuracy. In production this is usually impossible, because ground truth arrives late or never.

If a system ranks options and a human picks one, you learn whether the chosen option worked out. You learn nothing about the ones not taken. If a trading signal is published and the trader ignores it, there is no outcome to score. If a clinical suggestion is overridden, the counterfactual is unavailable by definition.

So drift detection in practice means watching proxies that move before outcomes do, and treating them as an early warning rather than a verdict.

The four proxies worth instrumenting

1. Input distribution

Track the shape of what arrives: field completeness, value ranges, category frequencies, text length, the proportion of records missing optional fields.

This is the cheapest signal and the earliest. It catches an upstream schema change, a new customer segment, or a source silently degrading — usually days before anything reaches output quality.

2. Output distribution

Track the shape of what the system produces: the spread of confidence scores, how often each recommendation type is chosen, the length and structure of generated content, the proportion of "no recommendation" responses.

A model that starts producing more high-confidence outputs than it used to has either got better or become miscalibrated, and those are worth distinguishing quickly.

3. Constraint hit rate

How often does output get held by a policy?

This is an underused and very informative signal. A rising hold rate means the system is producing output closer to its boundaries than it used to. Nothing has failed — the guardrails are doing their job — but the underlying behaviour has moved, and the guardrails are absorbing it silently.

4. Human override rate

How often does the person disagree with the system?

Where a human sits in the loop, their behaviour is the highest-quality signal available, and it is free. A rising override rate is the clearest possible statement that output quality has fallen, and it usually shows up well before anyone files a complaint.

Falling override rates deserve equal attention. They can mean the system improved, or that people have stopped reading carefully. Pair the rate with time-to-decision to tell those apart.

Calibration, where it is available

Where outcomes eventually arrive, calibration is the strongest measure available. A confidence score is calibrated if things it rates at 70% happen about 70% of the time — across many cases, not any single one.

Calibration degrades before accuracy becomes obviously bad, and it degrades visibly: the reliability curve pulls away from the diagonal. That makes it a genuine leading indicator rather than a lagging one.

It requires two things most systems lack: outcomes recorded against the original prediction, and enough volume for the buckets to mean anything. If you have both, this is the measurement worth building first.

Setting thresholds without guessing

The common failure is alerting on absolute values chosen by intuition, which produces either constant noise or permanent silence.

A more reliable approach:

1. Establish a baseline window while the system is behaving acceptably. Two to four weeks of normal operation.
2. Compute the natural variance of each proxy within that window. Most metrics move day to day; you need to know by how much before you can say what is abnormal.
3. Set thresholds relative to that variance, not to a round number.
4. Require persistence. One bad hour is noise. A metric outside its band for several consecutive periods is a signal. This single rule removes most false alarms.
5. Alert on rate of change as well as level. A metric moving steadily in one direction matters before it crosses any threshold.
6. Route the alert to someone who can act. An alert with no owner is a log line with extra steps.

When drift is detected, in order

1. Establish when it started. With versioned logs this is a query rather than an investigation.
2. Check for a model version change first. It is the cheapest thing to rule out and a common cause.
3. Check upstream data sources. A changed schema or a new value in a categorical field explains a great deal of apparent model degradation.
4. Compare recent inputs against the baseline window. Data drift and concept drift look similar from output alone; the inputs separate them.
5. Tighten the affected constraint temporarily. Lowering an autonomy threshold or routing more output to review is a reversible mitigation that buys time.
6. Only then consider changing the system. Retraining or re-prompting in response to an unexplained shift frequently fixes a symptom and hides a cause.

What monitoring cannot do

Drift detection tells you that behaviour changed. It does not tell you whether the new behaviour is worse.

A shift can be a genuine improvement, or a correct response to a world that actually did change. Deciding which requires judgement from someone who understands the domain, which is why the monitoring should route to a person rather than trigger an automatic rollback.

This is also why outcome feedback matters more than any dashboard. Proxies tell you when to look. Only outcomes tell you whether you were right.

Monitoring sits in the cross-cutting layer of Barion Core, alongside logging and evaluation, because a measurement taken only after an incident is a reconstruction. The wider set of controls it belongs to is on the governance and observability page.