Topic
embeddings
9 posts
- Retrieval4 min
The scores spread out 15 times wider and the ranking did not move
Embedding anisotropy: vectors sit in a narrow cone, so unrelated documents score 0.90 against anything. Centering fixed that exactly as advertised, and rank 1 went down by two.
- Retrieval10 min
Same bytes, a different folder, a different score
Embedding model evaluation that I made worse by being careful: indexing identical files under a different path returns different vectors.
- Retrieval14 min
The biggest repository wins, and it has nothing to do with relevance
Retrieval bias toward big repositories: every repo scored between 0.924 and 0.943 for every query, and the 0.019 that separated them was measuring size.
- Inference10 min
What running embeddings locally actually costs
Local embeddings at scale: 672,000 chunks across 28 repositories, indexed on a laptop. The bill was not money and not really time either.
- Inference9 min
Bigger batches made my GPU slower
Embedding batch size on a GPU: batch 4 beat batch 256 by 1.9x on the same device with the same weights. A third of the larger batch was computing padding.
- Retrieval8 min
Picking an embedding model when the queries and the code are in different languages
Best embedding model for code when queries and code are in different languages: two of five models scored below 4% on Pass@1, where random guessing gets 2%.
- Retrieval13 min
Three percent of my index was eating the top five results
Semantic code search crowded out by docs: 89 documentation files sank one repository, and removing them more than doubled MRR. The same fix on five other repositories moved nothing.
- Inference6 min
I measured Metal wrong three times before it beat the CPU
Apple Silicon GPU benchmarks for embeddings: Metal looked slower than the CPU in every early run, and all three runs were measuring something I had set up badly.
- Retrieval9 min
Your reranker speaks English. Your tickets do not.
A multilingual reranker problem: turning on the default reranker made search worse, and the model name had a two-letter suffix I had never looked at.