Three runs, one boundary
What reranking can and cannot do
A reranker reorders a list someone else built. That single fact decides which problems it can touch and which it cannot, and it is the difference between a week well spent and a week spent swapping models that could never have helped. These three posts find the boundary, then find out why the reranker inside it was failing anyway.
3 posts
Where this stands
- Held to be true
- Reranking reorders a candidate set and cannot add to it, so its ceiling is whatever retrieval already returned. Every measured gain here came from changing what was retrieved, and every reranker change moved ordering without moving recall.
- What would break it
- A reranker configuration that moves recall at all, or a case where reordering the same fifty candidates changes whether the answer is present.
- Still open
- Whether reranking pays for itself once retrieval is fixed. Every comparison so far ran on a candidate set that was itself broken, so the reranker was being asked to rescue something.
- 1Retrieval11 min
Reranking cannot find what retrieval missed
RAG reranking cannot recover missed recall: I swapped four rerankers and watched Pass@1 climb while recall sat at exactly 47.6% every time.
- 2Retrieval9 min
Your reranker speaks English. Your tickets do not.
A multilingual reranker problem: turning on the default reranker made search worse, and the model name had a two-letter suffix I had never looked at.
- 3Retrieval11 min
Fusing two rankers made routing worse, and the reason is structural
Reciprocal rank fusion made routing worse: one signal covered every repository, the other 17 of 28, and RRF punished exactly the repositories the second could not see.