My load test had 1,000 players. None of them were late.

David Mashiah — Mon, 02 Nov 2026 06:42:58 +0000

Before the first event I ran 1 vs 100 against a thousand simulated players, and nothing broke. Then a lecture hall of real people broke it with a fraction of that. The test measured load correctly. It just never contained the one thing real people always do: arrive late.

I tested 1 vs 100 properly before anyone played it. A script opened a thousand simulated players over real WebSockets, started a question, and had all of them answer in the same instant: the thundering herd, the worst moment a live quiz has.

It held. No dropped connections, the question reached every player in tens of milliseconds, and the reveal came back in under half a second.

At the first real event, it broke anyway. Not under load. Under people.

What the test could not contain

Here is what the script did, in order: connect every player, wait until every one of them has joined, and only then press Start.

That is the one order a real room never follows. In a lecture hall, people are still scanning the QR code when the host decides the room looks full. The game had a rule for anyone who joined after Start: watch, do not play. My thousand bots never triggered it, because not one of them was late. Everyone in the real room after the first 45 or so was, and their phones showed every question and never let them answer. The rule and the fix are their own note.

The test was faithful about load and wrong about time. It measured how many people the server could hold, and assumed they would all arrive at once, on cue, and stay on the page. Real people arrive late, switch tabs and reload. None of that is load, and all of it is what broke.

The first ten minutes

The first people with the problem came to me, and I did not know what it was. I went around to them, reassured them, and opened my laptop next to them to look. Technically, nothing was wrong anywhere I could see. The server was healthy and the game was running.

I told them their answers were being counted. I believed it. They were not, and I did not know that yet.

When more and more people had the same problem, I went outside to deal with it properly, making phone call after phone call to understand what was happening.

Not starting again

After about ten minutes the event coordinator came out to find me and asked what was going on. Her suggestion was to start the entire event again.

I asked her not to. Restarting would have thrown away the game for everyone who was playing fine, to fix it for the people who were not. Instead I asked her to tell the crowd that anyone hit by the bug would go through to the next stage rather than be eliminated.

That kept the event moving and kept the people who had been wronged in the game. Meanwhile I changed the rule and deployed the fix while the event was still running, and the stranded players came back in at the next question.

What a load test is for

A load test answers one question well: can the server take this many people at once. It is worth running, and mine was right.

It cannot answer the question that decided the night: what happens when people do not behave like a script. That needs a different test, one that is written from watching a real room: people joining after the start, leaving the page, coming back. Measuring the wrong thing precisely felt like safety. It was only confidence.

First published on davidmashiah.com, with the rest of my notes on real-time systems, machine learning reliability and supply chains.

One Durable Object per room, and a rule that stranded a hall

David Mashiah — Sat, 31 Oct 2026 21:09:37 +0000

At a faculty event the vice dean played 1 vs 100 against a room of students. About 45 of them got into the game. Everyone after that got a phone that showed the questions and never let them answer. Nothing had crashed. A rule that was correct on paper had met a real room.

1 vs 100 is a live multiplayer quiz for a lecture hall: one person on stage against the crowd on their phones, everyone answering the same question against the same clock.

The first time it ran at a real faculty event, the vice dean was The One. About 45 students got into the game. The rest of the room was holding phones that showed every question and never offered a single button to press.

The server was fine. Nothing ran out of memory, nothing timed out. The game was doing exactly what I had told it to do.

How a room works

Each game is one of Cloudflare's Durable Objects, addressed by the six-digit PIN on the projector. Every phone in the room opens a WebSocket to that one object, and that object is the only place the game exists.

A Durable Object handles one event at a time. A hall full of people answering in the same second cannot corrupt the scores, because there is never a second thread to race against. There are no locks in the code because there is nothing for them to protect.

There is also no shared database. The room keeps its own state in the object's built-in storage: the players, their scores, the question in flight. That is what lets a room survive being evicted from memory and pick up where it was. When the game ends, an alarm deletes the room's data. Each game is its own small database, and it lives exactly as long as the game.

Answers arrive in bursts, so broadcasts are batched, at most four a second, instead of one per answer. Idle sockets use WebSocket Hibernation, so a quiet room costs nothing. It runs on Cloudflare's free plan.

The rule that was right on paper

The game has knockout rounds. Get a question wrong and you are out. So I wrote a rule for late joiners that seemed obvious: if you join after the game has started, you watch. You cannot walk into a knockout halfway through and claim you survived questions you never saw.

In a test, that rule never fires. Everyone is already in the lobby when you press Start, because you are the one deciding when to press it.

A real lecture hall does not work like that. People are still walking in, still finding the QR code, still typing a name, when the host decides the room looks full enough. The host pressed Start with about 45 people in. Every phone that finished joining a second later was a spectator, for the whole game.

Fixing it while it was running

We kept the event going. Whoever had been stranded was carried into the next round by hand, so nobody lost their place for something that was not their fault.

Meanwhile I changed the rule, and deployed it during the event. In a round where nobody can be eliminated, a latecomer now joins as a full player immediately. In a knockout round they watch the current question and come in alive at the next one, so nobody is knocked out by a question they never saw. The only thing that keeps someone out for good is an actual elimination.

The part I am most glad of: anyone already stuck as a spectator did not have to do anything. When the next question started, the server brought them back in. By the following round, most of the room was playing.

The two players who were knocked out

Two players were knocked out who should not have been, and by the end of the event I did not know why.

It turned out not to be the server at all. Both had switched away from the tab in the middle of the game. When they came back, the phone reloaded the page, and in a knockout round that cost them their place.

That is the other half of the same lesson. A phone is not a test browser. It suspends tabs it is not showing, it reloads them when it wants the memory back, and nobody in a lecture hall keeps one tab open for an hour.

What I took from it

The architecture held. One object per room, no shared state, a single thread: none of that was the problem, and none of it needed to change.

What failed was an assumption about people. Practice is not the same as the real thing, and the difference is never in the parts you tested. It is the queue at the door, and the tab someone switched away from.