slowbench
RetrievalInferenceEvalsAgentsSeriesArchive

Contact

Getting in touch

The most useful thing you can send is a correction. If a number here is wrong, or a claim does not hold on your corpus, that is worth more to this site than agreement.

Replies are not guaranteed, but corrections are read and acted on. When a measurement here turns out to be wrong it gets corrected in place with a note saying what changed — not quietly deleted.

What is useful to include

  • Which post and which number.
  • What you measured instead, and on what — corpus size, language, hardware. Most disagreements here turn out to be different inputs rather than different conclusions.
  • Whether you could reproduce the setup as described. If the post was not specific enough to reproduce, that is a defect in the post.

Reusing what is here

Quoting with attribution and a link back is welcome. Every measurement is also available as a table and as a stated claim with its caveat — if you are citing a number, please carry the caveat with it. A measurement without its conditions is the thing this site exists to complain about.