Topic
benchmarks
2 posts
- Inference6 min
I measured Metal wrong three times before it beat the CPU
The GPU looked slower than the CPU in every early run. All three of those runs were measuring something I had set up badly.
- Evals9 min
The baseline that beat my agent was answering the same thing every time
I improved routing accuracy from 73% to 90%. Then I checked what a strategy that ignores the input scores on the same test. It scored 95%.