The Answer Came Back Too Fast, and It Was Somebody Else's
A model advertised 1,048,576 tokens of context over a memory pool that held 681,280. Past that line it answered anyway, out of other people's pages.
A model's sign said 1,048,576 tokens. The memory reserved behind it held 681,280. Prompts that crossed the line were supposed to get an error, and instead got an answer: coherent, early, and partly made out of other customers' pages. The bug lives in the path that handles rejection, which is the path nobody watches.
Prefer it read to you? Zara does voices now (11 min).
Zara is a character and this voice is synthesized. Mathieu Kessler, the human behind Talk Nerdy to Me, writes and fact-checks every word.
You send a very long question. The answer comes back quicker than you expected, it reads perfectly well, and somewhere in the middle of it is a paragraph about a subject you never raised.
Nothing about that feels like a security incident. It feels like the model wandered off. Models wander off. You shrug, you run it again, you get on with your day. The part you are least likely to question is the part that looked like good news. It was fast.
Between 27 August and 9 September 2026, 155 responses from one model on Cloudflare's Workers AI came back exactly like that. Fast, fluent, and partly made out of other people's sessions. Cloudflare published the full account on 26 October, including the parts that flatter nobody.
A sign on the door, and the chairs in the room
A context window is the sign on the door. It is how much text a model will look at in one go, and it is the number that goes in the announcement. This one, GLM-5.3 from Z.ai, was deployed on Workers AI with a sign reading 1,048,576 tokens. That is two to the twentieth power, because engineers like a number with clean edges.
The chairs are a different thing entirely. As a model reads your prompt, every token gets turned into two sets of numbers, a key and a value, and the attention machinery asks for them again at every step of writing the answer. So they get kept. That store is the key value cache, it sits in GPU memory, and the memory is carved into fixed-size slots called pages.
The cache is not something you can skip in practice, and it is not small. You can recompute all of it at every step instead, if you have no particular attachment to the answer arriving. The pool gets whatever GPU memory is left once the model weights are loaded, and it is the allocation that grows fastest as traffic and prompt lengths grow, which is why it has a ceiling.
On this deployment, on the prefill side, the ceiling was 681,280 tokens.
The sign said 1,048,576. The room held about two thirds of that. Cloudflare does not dance around it: "We were aware of this mismatch before launch."
Why there are two rooms now
Answering a prompt is two jobs with opposite appetites. Reading the prompt is one big parallel burst of arithmetic, called prefill. Writing the answer is a long series of small steps, one token at a time, where the hardware spends most of its life waiting on memory, called decode. Run both on the same GPU and each one makes the other worse.
Some of the large deployments separate them instead. A pool of prefill machines reads prompts and produces the cache. A separate pool of decode machines generates the answer out of it. The cache travels between the two over remote direct memory access, one machine reaching into another's memory without troubling either processor, with a transfer engine called Mooncake doing the moving. The architecture is called disaggregated serving and it is a good idea.
It also creates a sentence that did not exist before. The machine writing your answer never read your prompt. It is trusting that somebody else did.
The stand-in
The path that broke is the one you would never think to test, because it is the path where the system says no.
A prompt arrives that is bigger than the cache pool. SGLang, the inference engine, does the right thing and refuses it. The rejection path records the error to hand back to the client, then does something tidy: it swaps the enormous prompt for a placeholder one token long, so the dead request can drain out through the normal batch machinery without anybody reserving memory for a prompt that is never going to run.
Send a stand-in to the meeting. The stand-in says nothing, the meeting closes, everyone goes home, and the person who called it gets told it did not happen. On a single machine that is precisely what occurs. The placeholder runs, the batch finishes, the client gets its error. The behaviour is correct.
Split across two pools, the stand-in walks into a different room. The prefill side processes the one-token placeholder, produces a cache for it, and ships that across. The decode side receives a completion signal and reads it the way it has always read it, as meaning the pages for this request now hold this request's prompt.
They did not. Cloudflare walks it through with a worked example of eight pages, and the shape is the part that matters. The first page holds the single token the placeholder wrote. Every page behind it still holds what earlier requests left there, which is to say other people's prompts and other people's half-finished generations. A real request at this length runs to far more pages than eight, and the shape does not change with the count: one page written by this request, the rest belonging to whoever had them last. The decode machine attended over all of them, as instructed, and wrote a fluent answer across the top of strangers.
The client got HTTP 200 where an error belonged. The reported prompt token count matched the long prompt that had been discarded. The reply made sense. And it arrived faster than prefill on a prompt that long should take, which was the one tell that did not look like a problem.
How it surfaced
On 8 September 2026 at 23:11 UTC, security researcher Gadi Evron of Knostic reported it through Cloudflare's HackerOne programme, with a script that reproduced it on demand. A reproducible script turns an argument into a fact, and it compresses the following day into hours.
Cloudflare declared a security incident on 9 September at 13:22. Product Security reproduced the leak across two independent accounts within 25 minutes, which is the only confirmation that counts for this class of bug: one account's text surfacing in another account's session. At 14:00 the model went private and began returning HTTP 400. At 19:40 it was public again. All on the same day.
The bug affected 155 responses in total, every one of them an answer to a prompt longer than 681,280 tokens. Cloudflare's reading of the timing and the volume is that both look like ordinary long-context use rather than somebody working the bug on purpose, and Cloudflare is the party holding the logs, so that is the version to carry. The write-up does not say how many customers were involved or whose text ended up where, and neither will I.
The test could not see it
They did test long contexts before launch. In Cloudflare's own words:
we only validated HTTP response codes and engine performance at long contexts, without validating model output and coherence
Read that as a procedure rather than an apology. The test asked two questions. Did the request complete, and how quickly did it come back. It got a 200 and a good latency figure, and both of those answers were, strictly speaking, true. The failure had arranged itself entirely inside the one dimension nobody was measuring, which was what the thing said.
The dashboard was not lying. It was answering a narrower question than anyone realised they had asked.
The remediation carries the other lesson. The cache pool was raised to roughly 2.04 million tokens, about double the context window rather than equal to it. That number states a design rule out loud: the room has to be bigger than the sign on the door. Equal is not a safety margin, equal is a tripwire. SGLang patched the engine in the two releases that followed, and v0.5.22 or later is the floor Cloudflare recommends. That is where it stood when Cloudflare published on 26 October.
The same class of bug had already been filed against SGLang by another user, as issue #30233 on 6 July 2026, and Cloudflare found that while investigating its own incident. GLM-5.3 was the only model affected, because every other disaggregated deployment on Workers AI ran with a cache pool larger than its context window, so no request on those could reach the rejection path at all. This exposure needed a room smaller than its sign in order to happen.
The part that travels
The number on the sign and the number of chairs are two different numbers, and almost everything interesting happens in between.
A refusal path runs rarely by design, and it still has to behave correctly while everything around it is going wrong. This one did, on one machine.
Completing is not the same as being correct. If your checks only ask whether a call came back, and how fast, they cannot see a wrong answer. They will applaud the quick ones.
And shared memory gets reused. Pages are handed around all day, and what stood between one customer's text and another's here was the assumption that prefill had written over a page before decode read it. That assumption held for this path on one machine. It stopped holding when the writing and the reading moved into different buildings.
The placeholder was meant to be a formality. It is the smallest object in the system, one token wide, invented so that a refusal could leave the building tidily. It is also the piece that walked an answer out of the door with other people's words inside it.
Cloudflare knew the room was smaller than the sign. The test it ran could not see what was going wrong. It found out from a stranger with a script. Then it published all three of those facts about itself, which is the only reason you get to read any of this.
Sources
More Where This Came From
Plain-language translations of the machinery and the money behind the tech headlines. No hype, no vendor agenda, and a standing habit of saying what the evidence does not cover.
Related Posts
The GPU Money Merry-Go-Round: How the Same Dollar Gets Counted Three Times
Nvidia invests in the garages. The garages spend the money on Nvidia chips. Nvidia promises to pay for whatever capacity nobody else rents, through 2032. The same dollar rides the AI carousel three times, and everyone claps.
Neocloud, Explained: The Cloud That Sells Exactly One Thing
A hyperscaler is a supermarket. A neocloud is the guy selling one incredible thing out of a garage. There are now more than a hundred garages, and the category passed $25 billion in revenue in 2025.
The Padlock Is a Promise, and Somebody Else Had the Pen
The little padlock in your address bar is not a lock. It is a receipt saying that somebody your browser trusts has vouched for the server on the other end. In October 2026, eight of those vouching keys turned out to have been recoverable from public data since April 2025, because of a bug in a key generation ceremony. The browsers blocked them inside 72 hours, which is impressive and also raises the question nobody asks out loud: who decided they get to? And what about all the software that never checks whether anything was revoked at all?