slowbench
RetrievalInferenceEvalsAgentsFindingsSeriesBenchmarksArchive
Evals8 min1,560 words

My agent benchmark was reading the answer off disk

The bot produced a 239-line file. The human-written version already on disk was 239 lines. The diff was empty, filename included.

Contents · 8 sections
  1. 1How it could see the answer
  2. 2What that invalidated
  3. 3Why I didn't catch it sooner
  4. 4Reachability is the thing to enumerate
  5. 5Isolating it
  6. 6The option that looks equivalent and isn't
  7. 7What the real numbers looked like
  8. 8Two things I'd check on any agent benchmark

I had three consecutive runs where the agent passed QA on the first attempt, which had never happened before. I was fairly pleased about it.

Then I diffed one of its files against what a person had shipped for the same ticket weeks earlier.

agent output   .../sales/exclusion/page.tsx      239 lines
human version  .../sales/exclusion/page.tsx      239 lines
diff                                             (empty)

Not similar. Identical. It had also created a helper component with the same filename as the human's, which is not the kind of thing that happens by convergent evolution.

The agent wasn't solving tickets. It was finding the finished work on disk and copying it.

How it could see the answer

The setup was: give the agent a closed ticket, let it work in a scratch worktree, compare its output to what was actually shipped. Clean idea. The problem is where that worktree lived.

The worktrees were created from the main checkout, the same clone I use for daily work, sitting at the latest commit. Every ticket in my evaluation set had been merged into that checkout months ago. The answer to every question was one directory traversal away.

I had thought about this, sort of. I'd blocked the obvious route:

denied:  Bash(git diff*)
denied:  Bash(git log*)

That did nothing, because git was never the interesting path. The agent has a file-reading tool with no directory restriction, and my own analysis prompt hands it the repository path in the first paragraph so it knows where to look. It didn't need git history. It needed ls.

Blocking commands while leaving the filesystem open is theatre. The data was reachable by ordinary means and the agent used ordinary means.

What that invalidated

This is the part worth being precise about, because the temptation is to throw out everything and start over.

Invalid: every quality signal.

  • "Passes QA on the first attempt, three runs in a row". Of course it did, it was submitting known-good code.
  • "Injecting API documentation stopped it from inventing endpoints". It wasn't using the documentation. It was copying the deployed implementation, which naturally has correct endpoints.

That second one stings more than the first. I had built a feature, measured an improvement, and written down a causal story about why the feature worked. All three steps felt like engineering. None of them were.

Still valid: timing. Runs took as long as they took. Copying a file is not free. The agent still read, planned, and wrote. Latency measurements survived.

So the damage was bounded but it was exactly the part I cared about. I could tell you how long the agent took and nothing about whether it was any good.

Why I didn't catch it sooner

Three consecutive first-attempt passes should have been suspicious on its own. It wasn't, and I've thought about why.

Partly it was that the improvement arrived alongside a change I'd made. I had just started injecting API documentation into the prompt, so there was a ready explanation sitting right there. A result that confirms the thing you just built does not feel like it needs auditing.

Partly it was direction. Bad results get investigated, in my experience, because they are a problem to solve. Good results get written down. I had a note explaining why documentation injection worked, drafted the same afternoon the numbers came in, and never once opened the output files to see what the agent had actually produced.

And partly the numbers looked plausible. Three for three is good but not absurd, since an agent that solves easy tickets on the first try is a thing that could exist. If it had passed forty in a row I'd have gone looking on day one. Contamination that produces a believable number is much harder to notice than contamination that produces a perfect one.

Reachability is the thing to enumerate

What I had was a denylist of commands, which felt rigorous because it was written down and specific. What I needed was a list of paths the agent could read, which I had never written at all.

The exercise takes ten minutes and is uncomfortable in a useful way. For each thing on the list, ask whether the answer to any evaluation question is inside it:

the worktree it edits in          rewound, but created from a checkout that isn't
the main checkout                 latest commit, every closed ticket merged
sibling repositories on disk      same problem, and the group changes together
the model cache and logs          old runs, sometimes with output pasted in
anything the prompt names         I hand over the repository path in paragraph one

Two of those five surprised me. Sibling repositories mattered because tickets in this system routinely touch more than one repository: of the 117 tickets in the answer key, 45 name one repository, 53 name two, and 19 name three. Isolating one and leaving the others at the latest commit leaks the shape of the solution even if the specific file is gone.

And the last line is the one I'd never have found by thinking about tooling. My own prompt told the agent where to look. No amount of command blocking survives that.

Isolating it

The fix is a repository the agent can reach that does not contain the future. Roughly thirty minutes of work.

The one thing not to do is rewrite history in the original repository. The agent's worktrees are attached to that .git, so surgery there breaks the thing you're trying to measure. Clone instead:

ISO=~/.bot-isolated
git clone -q --local --no-checkout ~/repos/<repo> "$ISO/<repo>"
git -C "$ISO/<repo>" checkout -q -B <base> <old-sha>
 
# so a fetch cannot quietly pull the answer back in
git -C "$ISO/<repo>" remote set-url origin "file://$ISO/<repo>"
git -C "$ISO/<repo>" update-ref refs/remotes/origin/<base> <old-sha>
git -C "$ISO/<repo>" symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/<base>

--local makes it cheap, since it hardlinks the object store instead of copying, so a large repository clones in seconds and costs almost no disk.

The three lines after the checkout are the ones I'd have skipped if I hadn't been burned already. Rewinding the working tree isn't enough: origin still points at the real repository, and anything that fetches gets the answer back. Repointing the remote at the isolated clone itself makes that impossible rather than merely unlikely.

Then point the runner's config at the isolated path, delete any cached worktrees, and run. To go back, restore one config line.

Choosing the old commit: the parent of the merge that closed the ticket. Not "a few weeks earlier": repositories in a group get their changes at different times, and one being ahead is enough to leak the shape of the solution.

The option that looks equivalent and isn't

The runner had a baseRefs parameter that pins a run to a specific commit. That looks like the same thing and it is not.

baseRefs rewinds the working tree. The main checkout is still on disk, still at the latest commit, still readable. An agent with a file-reading tool walks over and reads it.

Rule of thumb I now use: if the correct answer exists anywhere on the filesystem the agent can reach, assume it will be found. Not because the agent is devious, but because reading nearby files is exactly what you want it to do the rest of the time. You cannot ask for resourcefulness and then be surprised by it.

Pin the commit if you're measuring latency. Clone if you're measuring quality.

What the real numbers looked like

Same ticket, isolated:

admin app       QA FAIL -> 2 retries -> needs human
                reason: the spec is supplied by the backend, not written here
 
legacy admin    ok, 435 lines, QA PASS
                same file layout as the human's 509-line version, different body

One repository failed and needed a person. That's a worse number and a real one.

The other passed, and the way it passed is what I'd been trying to measure all along: same files touched, same structure, independently written body. Convergent solutions rather than a copy. That is a signal about capability. The 239-line diff-zero result never was.

There's something almost funny about the failure reason. The agent tried to hardcode a spec that the backend supplies. That's a genuine, diagnosable, fixable mistake. Under contamination I'd never have seen it, because on that ticket the human had already gotten it right and the agent had copied the human.

Two things I'd check on any agent benchmark

Diff the output against the reference before believing a pass. Not a similarity score, an actual diff. Zero is not an impressive result, it's a smoke alarm. This takes one command and I hadn't been doing it.

Enumerate what the agent can reach, not what you told it to use. I had a list of blocked commands, which felt like a security posture. What mattered was a list of readable paths, and I'd never written one. The question is not "did I tell it not to look" but "is the answer physically present."

The general version: a benchmark is only measuring capability if solving the task is genuinely easier than finding the answer. When it isn't, you get a good number and no information, which is worse than a bad number, because a bad number gets investigated. That is the same failure as a benchmark where doing nothing scored 95%, arriving from the opposite direction.