Is your AI getting worse?
Is your AI getting worse? What the same tests show
Over the last 24 hours none of the 24 models on the board is below its recent baseline on any board, and no drift detector has fired.
Three models free · email when a watched model's coding score falls five points in a week
As of 26 Oct 2026, 11:37 UTC · 24-hour boards · per-suite Page-Hinkley detectors
- 24
- models on the 24-hour boards, every one on the same tests
- 66 of 68
- per-suite drift detectors armed with no detection
- 13
- dated changes in the record, September 2025 to September 2026, each with its before-and-after values
01
AI model nerfed? What the word means and what we can measure
Three different things get called a nerf. Only one of them is, and all three are told apart the same way.
- Nerfed
- A deliberate change that cuts a model’s cost or capability. Intent is invisible from outside; behaviour is not. On fixed tasks a nerf looks like any other sustained fall, so it is confirmed the same way: by a Page-Hinkley (a CUSUM-family test) detector per suite over days of daily medians, never by one bad answer.
- Drift
- Any change in behaviour behind a stable model name, in either direction, announced or not: a new build, a serving change, a tuned filter. This is the word for what the detectors watch. The record below shows what drift has looked like on the same tasks: longer answers, slower or faster responses, a renamed served model, withdrawn endpoints.
- A harness change
- Your tool changed, not the model: a new version of a coding agent, a different system prompt, more context stuffed into the window. The API-side score, taken with nothing of yours in between, is the control that separates the two.
Over the last 24 hours none of the 24 models on the board is below its recent baseline on any board, and no drift detector has fired. The verdict is read from the 24-hour boards and the per-suite detectors, as of 26 Oct 2026, 11:37 UTC.
02
Why is my AI model getting dumber? Four causes we can measure
Each cause leaves a different trace on fixed tasks, which is why the suites measure more than correctness.
- Silent version swaps
- A provider routes a stable name to a new checkpoint or build. Where a provider returns a build identifier, we record it with every call: that is how two new builds were dated to the minute in the record below. Where it does not, the daily series on the same tasks is the evidence.
- Serving changes
- Quantisation, a new serving stack, a different default budget. These show first as answer length and response time rather than correctness, so both are tracked on the same tasks; the record below has the steps — answer lengths that doubled, response times that stepped up or down under an unchanged name.
- Safety and refusal tuning
- A tightened filter makes a model decline work it used to do. A prompt the provider declines is recorded as a refusal with the provider’s stated category and that session’s score is withheld, never imputed, so a tuned filter shows as a refusal rate rather than as invented incapability.
- Long conversations
- Not a change in the model at all, but the commonest reason an AI “gets dumber” mid-task: the context grows and the model uses it worse. How accuracy falls as the context grows, from 8K to 1M tokens, is measured on context rot.
Page-Hinkley change-point detection (a CUSUM-family test) run per suite on each model’s daily median, constants published: δ 0.01; λ 0.30 for coding and reasoning, λ 0.50 held for three consecutive daily runs for tool use; detectors run on daily medians and are armed after ten days on the current test version, so a real shift is confirmed over days, not hours. An hourly canary of two fixed probes catches sharp drops with Welch’s t-test (12-point fall, p < 0.01). The detector and its constants are on AI drift detection; its measured false-alarm rate is in How often our drift detector is wrong, measured.
03
What changed in the record
Dated, past-tense changes from the test record, September 2025 to September 2026, newest first: the same tasks before and after, with the values.
- · DeepSeek V4 Flash DeepSeek
- Served model name: deepseek-v4-flash → deepseek-flash. From 10 Sep 2026 requests for deepseek-v4-flash were answered by a model identifying itself as deepseek-flash: the name we were paying to measure was redirected to a different served model.
- · DeepSeek V4 Pro DeepSeek
- Mean answer length and response time on the same tasks (build fp_v4pro_20260812, first seen 18:09 UTC): 727 tokens, 11.8 s → 1,477 tokens, 20.3 s. A new build behind the same model name, identified by the response fingerprint. Correctness held or improved slightly; the same work cost about twice as much and took 70% longer.
- · DeepSeek V4 Flash DeepSeek
- Mean answer length and response time on the same tasks (new build, first seen 02:10 UTC): 510 tokens, 5.6 s → 1,054 tokens, 9.3 s. A new build behind the same model name, identified by the response fingerprint. Correctness held or improved slightly; at the same list price per token, the same work cost about twice as much.
- · DeepSeek V4 Pro DeepSeek
- Response time on the same tasks: 20.4 s → 10.5 s (−49%). An isolated serving-speed step with no change on our side; the pass rate on the same tasks did not step with it.
- · GLM-4.6 Z.ai
- Answer length and pass rate on the same tasks, before and after 10 Mar 2026: 915 tokens, 93.1% pass → 1,149 tokens, 87.2% pass. Answers came back 26% longer on the same tasks. The extra failures were almost all answers cut off at the output cap: a wordier model that, under a fixed budget, failed where it used to pass.
- · Claude 3.7 Sonnet Anthropic
- Endpoint withdrawn: answering → HTTP 404 from 19 Feb 2026. Its last successful run and its first 404 fell on the same day.
- · Kimi Thinking (preview) Moonshot AI
- Endpoint withdrawn: answering (last successful run 7 Jan 2026) → HTTP 404 from 9 Feb 2026. For an application that hard-codes a model name, a withdrawal looks exactly like an outage.
- · GPT-5.2-codex OpenAI
- Listed but never served: listed to our account → HTTP 404 on all 83 attempts, 21 Dec 2025 – 6 Jan 2026. The model was listed to us but never answered a single call on our account.
- · Gemini 2.5 Pro (preview 03-25) Google
- Endpoint withdrawn: answering (last successful run 9 Nov 2025) → HTTP 404 from 2 Dec 2025. For an application that hard-codes a model name, a withdrawal looks exactly like an outage.
- · Kimi K2 (0711 preview) Moonshot AI
- Response time on the same tasks: 7.4 s → 4.1 s (−45%). An isolated serving-speed step with no change on our side; the pass rate on the same tasks did not step with it.
- · Gemini 2.5 Flash Google
- Response time on the same tasks: 1.4 s → 1.0 s (−28%). An isolated serving-speed step with no change on our side; the pass rate on the same tasks did not step with it.
- · GPT-5 nano OpenAI
- Response time on the same tasks: 8.5 s → 12.8 s (+51%). An isolated serving-speed step with no change on our side; the pass rate on the same tasks did not step with it.
- · Gemini 2.5 Pro (preview 03-25) Google
- Response time on the same tasks: 5.3 s → 3.8 s (−28%). An isolated serving-speed step with no change on our side; the pass rate on the same tasks did not step with it.
Method: daily medians of the same nine coding tasks, binary segmentation with Welch's t-test; provider builds and served model names read from the identifier each response carries. Withdrawals are read from the per-call record: a last successful call, then HTTP 404 on every later attempt. Every row, with its dates and how it was derived, is in What 13 months of monitoring LLM APIs caught.
04
By family
The same verdict, family by family, each with its own page: the models on the four boards, the API's availability, the tool-or-model question and the dated record.
- Is ChatGPT getting worse?
- Over the last 24 hours none of the 8 GPT models we track is below its recent baseline on any board, and no drift detector has fired.
- Is Claude getting worse?
- Over the last 24 hours none of the 9 Claude models we track is below its recent baseline on any board, and no drift detector has fired.
- Is Gemini getting worse?
- Over the last 24 hours none of the 3 Gemini models we track is below its recent baseline on any board, and no drift detector has fired.
- Is DeepSeek getting worse?
- Over the last 24 hours none of the 2 DeepSeek models we track is below its recent baseline on any board, and no drift detector has fired.
- Moonshot AI (Kimi) models
- Over the last 24 hours the Kimi model we track is not below its recent baseline on any board, and no drift detector has fired.
- Z.ai (GLM) models
- Over the last 24 hours the GLM model we track is not below its recent baseline on any board, and no drift detector has fired.
Every model on the board has a page of its own, with its history and drift status:
05
How to check yourself
The same rules that keep our verdict honest work at any scale.
- Fix the tasks. The same prompts and the same output budget every time; a changed prompt measures the prompt.
- Repeat. Model output varies run to run even when nothing has changed; each of our coding tasks is run seven times and scored by the median.
- Decide the rule before you look. A baseline, daily medians and a change-point test with a confirmation window — the constants are on the methodology page.
- Test the API, not the app. The app adds its own prompt, tools and context; the API is the model as served.
- Count refusals as refusals. A declined prompt is a tuned filter, not a lost ability.
Or run our tests on your own keys: Test Your Keys uses the same scoring code as the published runs, so your numbers and the boards are comparable.
Free account · your own API keys · the same scoring code as our published runs
06
Questions people ask
Are AI models really getting worse over time?
Not as a rule, on fixed tasks. Over the last 24 hours none of the 24 models on the board is below its recent baseline on any board, and no drift detector has fired. Over September 2025 to September 2026 the record caught changes in how models answer — longer answers, slower or faster responses, a renamed served model, withdrawn endpoints — more often than changes in whether they answer correctly. Each change is listed above with its date and its before-and-after values, and every model on the board is re-tested on the same schedule — coding every four hours, reasoning and tool use daily, with an hourly canary probe — so the answer is re-derived rather than remembered.
What does "AI model nerfed" mean?
A nerf is a deliberate change that cuts a model's cost or capability. Drift is any change behind a stable model name, deliberate or not, in either direction. A harness change is your tool changing while the model stays the same. From outside, intent is invisible and behaviour is not, so all three are told apart the same way: fixed tasks, repeated runs, a baseline, and a change-point test that confirms a sustained shift over days. The API-side score, taken with nothing of yours in between, is the control that separates a harness change from the other two.
Why is my AI model getting dumber?
Four causes the tests can see: a silent version swap behind the same name (where a provider returns a build identifier we record it with every call); a serving change, which shows first as answer length and response time rather than correctness; safety tuning, which shows as refusals, recorded as refusals rather than scored as failures; and a long conversation, which is not a change in the model at all but the commonest reason an AI feels dumber mid-task. The three model-side causes are what the per-suite detectors watch; the long-conversation effect is measured separately on context rot, from 8K to 1M tokens.
Is ChatGPT getting worse?
Over the last 24 hours none of the 8 GPT models we track is below its recent baseline on any board, and no drift detector has fired. The GPT models are tested through the OpenAI API, model by model, on the same tasks as every other vendor's models: coding every four hours, reasoning and tool use daily, with an hourly canary probe. The model-by-model answer, the API's availability and the dated record for the family are on Is ChatGPT getting worse?.
Is Claude getting worse?
Over the last 24 hours none of the 9 Claude models we track is below its recent baseline on any board, and no drift detector has fired. The Claude models are tested through the Anthropic API, model by model, on the same tasks as every other vendor's models: coding every four hours, reasoning and tool use daily, with an hourly canary probe. The model-by-model answer, the API's availability, the Claude Code question and the dated record for the family are on Is Claude getting worse?.
Can I get an alert when a model I use changes?
Yes. Free account: every ranking over 24 hours, 7 days and 30 days, a 3-model watchlist with email alerts, 1,000 routed requests a month and a Data API key. No card. The watchlist sends an email when a watched model’s coding score falls five points below a week earlier, and a weekly summary of what moved; paid plans tune the threshold and watch more models. The methodology is public — every weight, statistical test and drift constant. The test tasks are private, so no lab can train on them.
Know the next time it changes
Free account: every ranking over 24 hours, 7 days and 30 days, a 3-model watchlist with email alerts, 1,000 routed requests a month and a Data API key. No card.
Three models free · an email when a watched model’s coding score falls five points below a week earlier, and a weekly summary of what moved
The Smart Router is drift-aware by default, on your own provider keys · 1,000 requests a month free