How to tell when an AI model quietly gets worse — and how we measure it across every major provider.
gpt-5.5 or claude-opus-5 at any time, and your code keeps calling the same string. If output quality drops, nothing in the API tells you. AI Stupid Level benchmarks every major model continuously and applies change-point detection to the results, so a sustained decline becomes a measurement instead of a hunch.AI drift is a measurable change in a model's output quality over time while its public identifier stays the same. It is distinct from prompt drift (your inputs changing) and from data drift in classical ML (the input distribution moving). Here the model itself is the thing that moved.
Drift is not always downward — models sometimes improve after an update. The problem is that neither direction is announced, so teams building on a model have no way to know their evaluation results are still valid. That uncertainty is the actual cost: you cannot tell whether a regression in your product came from your last deploy or from your provider.
A provider routes a stable alias to a new checkpoint. Behaviour changes; the model name does not.
Quantisation, distillation and serving-stack changes cut inference cost, and can cut output quality with it — particularly on long-context and multi-step reasoning work.
Tightened filters raise refusal rates on legitimate requests. The model is not less capable, but it is measurably less useful for the task you had.
Under heavy load, providers may shift defaults such as sampling parameters or reasoning budget, which shows up as higher variance rather than a clean step down.
A single weak response is not evidence. Model output is stochastic, so scores vary run to run even when nothing has changed upstream. A naive threshold alarm on raw scores either fires constantly on that noise or is set so loose it misses real decline.
We use CUSUM — a cumulative sum control technique from statistical process control. Instead of testing each run in isolation, CUSUM accumulates each run's deviation from an established baseline. Random noise has no consistent sign, so it cancels out and the running total stays near zero. A genuine sustained decline pushes deviations consistently in one direction, so the total climbs until it crosses a decision threshold and a drift event fires.
The threshold is what trades false alarms against detection lag. We tune it against historical benchmark variance so that ordinary fluctuation stays quiet while a real shift is caught within hours. Every score is also published with a confidence interval, so you can see how much of a gap between two models is meaningful and how much is noise. The full statistical approach is documented on our benchmarking methodology page.
Drift is tracked per benchmark suite, because a model can decline in one dimension while holding steady in another — a common pattern after a cost-optimisation update.
Correctness, adherence to spec, code quality, efficiency, stability, refusal rate and error recovery, scored by executing the generated code rather than grading it by similarity.
Multi-step problem solving, plan coherence, long-context retention and hallucination rate — usually the first place a quantised model shows decline.
Tool selection, argument accuracy, error handling and recovery. Critical for agents, where a small drop in argument accuracy compounds across a long chain of calls.
Providers publish benchmark numbers at launch. Nobody publishes them continuously afterwards, and no provider announces that a model got worse. Independent, ongoing measurement is the only way to know whether the model you evaluated is the model you are running — which matters for anyone with a model choice baked into production, a vendor contract to justify, or an AI feature whose quality they are accountable for.
Every model we track has its own page with a historical score chart, per-suite breakdown and drift status. Start from the live AI model leaderboard, or read the benchmarking FAQ for shorter answers to common questions.
AI drift detection is the practice of continuously measuring a model's output quality so you can tell when it changes. Because providers serve updated models behind stable API names, the version you call today may not behave like the one you tested against months ago. Drift detection replaces the anecdotal "it feels worse lately" with a measurement you can point at.
Several things: a provider silently swapping in a new checkpoint, changes to safety filters that increase refusals, quantisation or serving-stack changes made to cut inference cost, altered default sampling parameters, and capacity pressure during peak demand. From the outside these are indistinguishable — all you observe is that the same prompt now returns weaker output.
You need a fixed test set, repeated runs, and a statistical baseline. A single bad response proves nothing — model output is stochastic, so quality varies run to run even with no change at the provider. The signal you want is a sustained shift in the average across many runs, which is exactly what change-point detection is designed to isolate.
CUSUM (cumulative sum) is a change-point detection algorithm that accumulates deviations from a baseline mean. Small random fluctuations cancel out over time, but a genuine sustained shift accumulates until it crosses a decision threshold. This makes it far better suited to catching gradual decline than a simple threshold alarm, which either fires constantly on noise or misses slow degradation entirely.
Benchmarks run continuously, with the code suite refreshing multiple times a day and deeper reasoning and tool-use suites on their own cadence. Every completed run feeds the drift calculation, so a sustained change surfaces within hours rather than whenever someone happens to notice.
Current models across OpenAI, Anthropic, Google, DeepSeek, Moonshot AI and Zhipu AI — including the GPT-5 series, Claude Opus and Sonnet, Gemini, DeepSeek V4, Kimi and GLM. Every model in the leaderboard has its own page with a historical performance chart.
All benchmark data is free to browse — no account needed. Compare current models side by side, or open any model to see its full performance history and drift status.
VIEW LIVE MODEL RANKINGS →