slowbench
RetrievalInferenceEvalsAgentsFindingsSeriesBenchmarksArchive

Measure first. Then decide.

Most of what I believed about my own systems did not survive measurement.

Notes on retrieval, evaluation, and the systems around them — measured before written. Every number here comes from a run against the system that actually serves traffic — not a copy made for the benchmark. Where a result rests on one run, it says so.

Posts
50
Words
77,902
Sections
4

Where things stand

All 9 →

Retrieval

16

Code search, embeddings, and ranking.

Inference

5

Local inference, Metal/GPU, and batch tuning.

Evals

16

Measurement methodology and benchmark design.

Agents

13

Coding agents and context management.

Publishing

Nothing scheduled. The next measurement decides.

42 in the last 42 days · 0 scheduled