One failure, read three times
Routing a ticket to a repository
Twenty-eight repositories and a ticket that belongs to one of them. Fusing two rankers lost, then lost again, then won — with nothing about the fusion changed. Read in order, these are the same event from three distances: the mechanism, the verdict I drew from it, and what the verdict was actually a specification for.
1 posts
Where this stands
- Held to be true
- Picking the right repository is a retrieval problem with almost no published benchmarks, and the obvious signals do not work on it. Absolute similarity scores failed three separate times as a basis for any threshold. Size bias is real and survives the usual normalisations.
- What would break it
- A similarity threshold that separates good routing results from bad ones on a held-out set.
- Still open
- Whether concentration replaces similarity as the usable signal. One repository showed clean separation at p 0.016, the cut point was chosen after seeing it, and no second repository has both hits and misses to test on.