This is the third post about running Klooboo, a free game app for teens (13 and up), with a team of AI agents. The first was the org chart. The second was how the team keeps moving with or without me. This one is about the piece that made both possible.

On 2 November, 31 open pull requests sat in the “being built” column waiting for one thing: me pressing merge. Tests green, screenshots checked by the reviewer agent, and nothing happening. I asked Scout, the AI head of product, what my click was adding, and then answered my own question:

“I am not reviewing the code, this is the team’s job and not mine anymore … I don’t see any risk since the tests are passing and it’s behind a feature flag. Maybe just keep the ones that you need to flag something like a risk or downtime.”

The important words there are behind a feature flag. Without that, letting agents merge their own work would be reckless. With it, a merge is close to a no-op for players.

---

Every change ships OFF

The rule is simple to state: anything a player would notice ships behind a flag, and the flag is off. Off means the app behaves exactly as it did before the PR. Not “mostly the same”, not “the new button is hidden but the layout moved”. The same.

That rule turns two decisions that used to be one into two separate ones:

• Merge = “this code is correct and can live on main.” The team decides.
• Go live = “players should see this now.” I decide.

Most of the risk in shipping lives in the second one. Once they’re separate, the first can be automated.

The registry: every flag explains itself

Flags live in one registry on the server, KNOWN_FLAGS. A PR that adds a flag adds an entry, and each entry carries a full note of what ON does and what OFF does:

export const KNOWN_FLAGS: ReadonlyArray<{ name: string; area: string; note: string }> = [
  {
    name: "jigsaw_edges",
    area: "Jigsaw Snap",
    note: "A 🧱 button slides every loose border piece to the rim of the table. " +
          "Free, any number of times; score unchanged. Event jigsaw_edges_used. " +
          "OFF: no button, as today",
  },
  // …one entry per flag
];

The notes are long on purpose. Some run to a paragraph: which screens change, which events fire, what it costs, which other flags it needs, and a final sentence starting OFF: that says what happens with it off. That last sentence is the one the reviewer checks against the code before a merge, and the one I read before I press Go live.

The admin page lists every flag with its note and whether it’s on, so at any moment there’s one place to answer “what is live, and what does this switch actually do?” Over time the registry has also become the most honest changelog we have. What the agents add to the app is described there in plain words.

One narrow door for merging

Agents can’t run a plain gh pr merge. That command stays blocked for every team run. There’s exactly one door: a script that merges one PR, and only if every guard passes. Otherwise it refuses and prints why.

The guards, in order:

1. The PR is open and not a draft.
2. GitHub says it’s mergeable and clean: no conflict, not behind.
3. Every check is green. Nothing pending, nothing failed.
4. It’s behind an OFF flag, or it only touches admin pages, tests or docs.
5. No desk card asks me about it. If the reviewer marked the PR as needing me (a risk, a migration, downtime), the card carries that reason and the script won’t touch it.

The shape is boring on purpose:

[[ $state == OPEN && $draft == false ]]        || refuse "not open"
[[ $mergeable == MERGEABLE && $merge_state == CLEAN ]] || refuse "conflict or behind"
[[ $failing_checks == 0 ]]                     || refuse "checks not green"
pr_names_off_flag_or_admin_or_tests            || refuse "needs Itay: ✅ Merge"
! desk_card_asks_itay "$pr"                    || refuse "a card asks Itay first"
gh pr merge "$pr" --squash

Anything that fails a guard goes to me as a card with a ✅ Merge button and the reason in one line. The things that always come to me: a player-visible change with no flag, a data migration or delete, money, store or prices, the deploy pipeline, and auth.

Every merge and every refusal goes into a log, so “why didn’t this merge?” has an answer without asking an agent to remember.

Go live is my click, on the desk, never in chat

Turning a flag on is the other half, and it stays human. It happens in one place: the Go live button on a card on the desk, the small board where decisions reach me. Pressing it switches that card’s flags on in production. Agents never switch a flag on, and neither does a chat message.

Two things happen before a card even shows a Go live button:

• Lens, the AI design director, has passed it. It looks at real frames of the feature running and returns pass or fix. A fix goes back to building.
• The card has frames and a “try it on your phone” link that opens the new screens with the flag on, so I can play it before anyone else sees it.

I keep this out of chat on purpose. When Scout once offered to approve a batch of Go live cards in one question, I turned it down. Most of those cards were waiting because I had comments, not because I hadn’t got to them. “Go live” is the last taste check, and I want the frames in front of me when I make it.

Lesson one: “behind a flag” isn’t the same as “OFF”

The first close call happened on the day the merge door went in.

A PR added portals: a glowing ring in one game that briefly sends you into a world another player made, and a push notification to that maker. The PR description said it was behind its flags. The tests were green. The merge script would have merged it: guard four only checks that the PR’s title or description says OFF.

The reviewer checked what the flags actually were. The portal relied on two existing flags: the Hoops game’s own flag, and the flag for “tell the sharer when their share lands” push notifications. Both were already ON in production. Merging would have started sending a brand-new kind of push to makers the moment it deployed. No Go live click, no design pass, nothing.

The reviewer stopped, put a card on the desk saying exactly that, and asked me. I read it and said merge. I was fine with that push going out. But it was my decision to make, and the script alone would have made it for me.

What I took from it: “this PR is behind a flag” is a claim about the code. “This PR changes nothing for players” is a claim about production, and it depends on every flag the change reads, not just the new one. So before a merge, the question to ask is the live state of every flag the change reads, not only the flag it adds.

Lesson two: “every check green” can mean “no tests ran yet”

The second one was worse, because the script did exactly what it said.

It merged a PR whose checks were all green. There was one check. It was a quick policy check that runs in seconds. The main CI workflow, the one with the test jobs, hadn’t started yet. “Every check is green” was true. It was also true of zero test runs.

The fix: the script now refuses unless the main workflow’s first job, its gate job, is on the PR and green. If that job isn’t there, CI hasn’t run, and “all green” means nothing.

That fix had its own edge case. Docs-only PRs (memory notes, Markdown) never trigger the main workflow, so they never get the gate job, so the script now refused all of them forever. They needed their own exception: if every file in the PR is documentation, the missing gate job is expected, and the PR’s own checks are enough.

Both mistakes have the same shape. A guard that checks “nothing is failing” passes when nothing is running. The useful question is “did the thing I expect to have run actually run, and pass?”

The CI fix that made many merges safe

Letting agents merge also changed how often main moved. Several merges in a few minutes became normal, and that exposed an old problem in CI.

Our CI decides which tests to run by diffing the merge against the previous commit on main. When merges come close together, GitHub cancels a run that’s still queued in favour of the newer one. The newer run diffed only against its own parent, so the changes from the cancelled run were never tested by anything.

The fix was to make change detection diff against the last successful main run, not the previous commit. Now each run tests everything that changed since main was last known good, however many merges landed in between. With that in place, the merge script no longer has to wait for CI to go idle before merging the next PR.

What I’d take from it

• Separate merge from go live. A flag that is off by default turns “should players see this?” into a decision you can make later, by a different person, with frames in front of you.
• Make OFF mean the old app, exactly. If off isn’t the old app exactly, the flag doesn’t protect you.
• Write down what every flag does, ON and OFF, where people can read it. The registry note is the contract the reviewer checks against.
• Give agents one narrow door, not a key. One script, a short list of guards, a log, and plain merge blocked.
• Check production, not the PR description. A change is only as off as every flag it reads.
• “Nothing failed” isn’t “it passed”. Require the job you expect to see, and plan for the PRs where it legitimately never runs.

I don’t review code any more. I review frames, and I press Go live. Feature flags are the reason those two things are enough.