I fixed the code three times and never checked the symptom

A code search tool I depend on has a --per-file flag. It caps how many chunks any single file can contribute to a result set, so five results mean five different files instead of five lines from one.

It was parsed. It was documented. In JSON output it did nothing.

That is the whole bug, and it is not the interesting part. The interesting part is the second fix, which was correct, which I checked in the diff, and after which I told somebody the flag worked. It did not.

How it stayed invisible for three months

git log -S on the option name puts the flag and the JSON branch in the tree together at least as far back as June 1. It may be older, since that history predates the fork. So call it three months as a lower bound.

Three months during which nobody hit it, because the flag worked fine in text output. The folding step ran on the text path and was skipped on the JSON path.

Nothing used the JSON path until an agent did. The agent that consumes this tool had been scraping the text output with a regex, and switched to JSON. That switch is what surfaced the bug. Not a test, not a report, just a consumer moving to the branch where the feature was missing.

Two things have to be true at once for a flag to disappear this quietly: it has to be accepted by the parser, and it has to be applied somewhere other than where you look. A rejected flag errors. An undocumented flag gets no users. A flag that is parsed, documented, and applied on only one of two paths gets used with confidence and does nothing.

Three fixes

The first fix worked. Duplicates stopped reaching the output, which was the symptom anybody had actually reported.

The second symptom is the one in the third row: ask for five, get three. That looks like folding running after truncation, which it was, so the fix was to move the fold ahead of the cut. Correct diagnosis, correct patch, and the person who wrote it said so.

The other session re-ran it and got three files. Then it ran two more:

That table is the diagnosis. Folding was running by then. If the count still scales with what you ask for, the pool being folded is tied to what you ask for.

The layer nobody was looking at

The third layer was one line in the retrieval stage:

let take_multiplier = if should_filter_by_path { 3 } else { 1 };
let take_count = options.max_results * take_multiplier;

With no path filter the multiplier is 1, so the number requested is the number fetched. Five results requested, five chunks fetched. Folding removes duplicates from a candidate pool, and there was no pool, just exactly as many chunks as the caller asked for. If three of those five came from the same file, folding could throw away two and had nothing to put in their place.

Reordering the pipeline could not fix that, because the problem was never the order. Two rounds of moving the fold earlier were both real improvements to a pipeline that had nothing to fold.

There were three causes, not two, and I saw the second symptom twice.

The first round found a real cause: the JSON branch returned before reaching the fold. Fixed, and the duplicates went away.

Then the count came back short, and the explanation that arrived was folding-after-truncation. That explanation was true. Moving the fold earlier was a real improvement to the pipeline. It just was not the whole reason the count was short, and because it was true, nothing about checking it flagged anything.

A cause that is partly right survives longer than one that is wrong. A wrong diagnosis fails its own test. A partial one passes, ships, and closes the ticket.

What the fix actually changed

The query was a form-validation lookup, the kind where the answer could be in any of several services.

asked for   5 results, at most 1 per file
got         5 slots, 3 of them the same file
wanted      5 distinct files

Routing a ticket to a repository is a breadth problem. The ranker gets to look at whatever the retrieval stage hands it, and if three of five slots are different lines of one file, the ranker sees three fewer candidate files. It was not returning wrong results. It was returning fewer options than the caller had asked for, while reporting five.

The part I want to keep

The person fixing it confirmed the code changed. They did not confirm the symptom went away. Those are different checks, and only the second one is about the user.

Confirming the diff is cheap and feels like verification, because you can see the thing you intended to do. Re-running the command is also cheap. The gap between them is where a correct fix sat for a round, announced as finished, with the output still short.

What closed it was somebody else running the same command again and getting the old number.

The same habit, killing a hypothesis

On the same day, in the same tool, re-measuring did the opposite favor.

Queries in this corpus often carry a bracketed prefix, a tag at the front of the ticket title. It looked like noise. Stripping it made one query noticeably better, and there was a clean story for why: the prefix is boilerplate, it dilutes the embedding, remove it.

Measured across the set:

The prefix was the strongest signal in the string. Removing it removed the part that said which system the ticket belonged to.

One case, a mechanism that sounded right, and a result that reversed under the full set. I have written this same shape here before from the other direction, a tie at four hundred queries that was a loss at fifteen hundred, and it keeps arriving in new clothes.

Both of these are the same discipline pointed at different things. The flag bug survived because nobody re-measured after a fix. The prefix hypothesis would have survived because nobody measured past the case that suggested it. Re-measuring rescued one and killed the other, and there is no way to know in advance which one you are holding.