The bot has one fully automatic path. An error-tracker alert (Sentry) comes in, the bot diagnoses it, writes a fix, has the fix reviewed, deploys it to the development environment and runs QA against that deployment. Nobody is in the loop until the fix is proposed for production, where a person clicks approve.
The question I was asked was reasonable: "so it only does this when the fix is small, right?" I went to look for the size gate and there was not one. There were seven other gates. Size was simply not a thing the path had ever been asked about, so I pulled every alert-triggered run from the database and looked at what the fixes had been.
The gates that did exist
Every one of these is about whether the bot should act or how much it may spend. None is about how much code the action produces. A fix that touches 200 files passes the same gates as a fix that touches one.
44 runs, nine days
All alert-triggered runs in the database, 2026-09-14 to 2026-09-22. 24 of the 44 landed on a single day.
| outcome | runs |
|---|---|
| fix written, reviewed, deployed to development, QA'd | 19 |
| diagnosed as not our code (upstream timeout, browser extension, backend business rule, ...) | 15 |
| same fingerprint already judged within 14 days | 6 |
| legacy repository: diagnosed only, by policy | 1 |
| failed: review objection unresolved after two corrections | 1 |
| failed: diagnosis timed out at 300 seconds | 1 |
| cancelled by a person | 1 |
(source: the bot's run table; outcomes are the recorded status plus the diagnosis note)
So 43% of alerts became code. The 15 "not ours" verdicts are the diagnosis gate doing its job: an OOM that turned out to be in a coupon upstream, an exception whose entire stack was a Chrome extension monkey-patching XMLHttpRequest, a 400 that was a backend business rule refusing an unset date. None of those should produce a patch and none did.
What the 19 fixes looked like
For each run the final patch's git diff --stat is stored. File counts:
| files in the final patch | fixes |
|---|---|
| 1 | 12 |
| 2 | 2 |
| 3 | 3 |
| 10 | 1 |
| 232 | 1 |
Wall clock from alert to the end of the run for the 19: 5.4 to 19.7 minutes, median 8.5. For the five runs after cost accounting was added on 2026-09-19, $1.77 to $4.12 each; the earlier fourteen have no cost rows and I will not guess.
Seventeen of nineteen touched three files or fewer. That is roughly the shape I would have guessed, and it is the reason nobody had asked for a size gate: the path had not produced anything alarming. The two outliers are the interesting part, and they are interesting for opposite reasons.
The 10-file fix that was a 1-file fix
The 10-file patch has 7,991 insertions and 9,492 deletions. Nine of the ten files are regenerated OpenAPI JSON under packages/api/swagger/, accounting for 7,960 of the insertions. The change a reviewer would care about is one file, 31 lines, in a dashboard page. A gate on file count or line count would have held this run for a person who would then have scrolled through 17,000 lines of generated JSON to find 31 lines of intent.
The 232-file fix that was a 131-file fix that was a 2-line fix
The largest run has three patches recorded. The first is 131 files, 198 insertions, 166 deletions: an exception-class signature changed, and 131 service classes updated to match, two lines each. Mechanical, but real, and the kind of thing I would want a person to look at before it went anywhere.
The third patch, from a re-fix after review, is recorded as 232 files and 3,949 insertions. It includes a 2,796-line HTML event template and a 441-line project-instructions file that the fix had no reason to touch. The diff was taken against a base that had moved underneath the branch, so other people's commits are in the stat. That number is a measurement error, not a bigger fix.
Both outliers say the same thing about a file-count gate: the count you get from diff --stat is only meaningful if you subtract generated files and diff against a clean base. Neither is hard. Both have to be done before the number is worth holding a run on.
What I am adding
A size gate, with two rules for the number it reads:
The distribution above is the argument for N being small. Anything over three files is 2 runs in 19, and both of those are runs I would have wanted to see. What N should be is a decision I have not made yet; it is the kind of number that should be set by the person who will be paged, not by the person who wrote the gate.
What the numbers do not say
They do not say the 17 small fixes were correct. They passed review and QA and were deployed to development; production approval is a separate step with its own record, and this post is about size, not correctness.
They also do not say a size gate would have caught anything harmful in these nine days. It would have held two runs, and on inspection one of the two was a bad diff base rather than a big change. The gate is there for the run that has not happened yet.
Frequently Asked Questions
Why not gate on lines instead of files?
The 10-file fix is the answer: 17,483 lines changed, 31 of them by hand. Lines are dominated by generated content even more than files are. Either metric works only after generated files are removed, and once they are, files are the easier number to explain in a hold message.
Why does the bot deploy to development at all without a person?
Because the development environment is where the fix gets exercised, and a fix that has not been exercised is a hypothesis. The path is designed so that the human decision happens once, at the production step, with a build that has already run somewhere.
Isn't "not ours" 34% of alerts a lot of wasted diagnosis?
Each of those cost a diagnosis call and nothing else, and each produced a written reason that a person can read in ten seconds instead of opening the alert. The alternative is a person triaging 44 alerts by hand. The cost cap for a diagnosis call is $1.50; the two "not ours" verdicts recorded since cost accounting began cost $0.94 and $0.56.
What happened to the run with the review objection?
The review pass objected, the bot made two correction passes, the objection stood, and the run failed by rule. That is gate 5 working: a fix the reviewer would not sign off on did not reach deploy. It is also the only one of the 44 where a disagreement between two model passes decided the outcome.