AI drift
The phenomenon where an AI system's behavior changes over time without explicit code changes — caused by model version updates, training data shifts, or vendor-side changes.
AI drift is one of the trickiest production problems in 2026. You ship a working agent on Monday. On Wednesday, your model vendor silently updates the underlying model. Now your agent behaves slightly differently — outputs are 2% less accurate, response style has shifted, occasional new failure modes appear.
The fix is monitoring. Run a small "smoke eval" on every release — a fixed set of inputs with expected outputs — and alert on drift. Vendors usually announce major model changes but minor updates (bug fixes, tuning) can shift behavior without notice.
For agent builders, drift is why evals are non-negotiable. Without eval coverage, drift is invisible until users complain. With evals, you detect drift in hours and either roll back to a pinned version or update prompts to compensate.
Frequently asked
How often does AI drift happen?+
For closed-vendor models: every few months at minimum. Major versions are announced; minor updates often are not. For open-weight models you self-host, drift only happens when you choose to update.
How do I detect AI drift?+
Run a fixed eval suite (50–200 test cases) on a schedule. Compare results week-over-week. Significant drops in eval scores = drift. Tools like Braintrust automate this monitoring.