Writing
Short pieces about what the numbers around AI agents actually measure. Each one comes with code and data you can rerun.
-
On a 66-task benchmark, ten points is a tie
I tested two popular fixes for AI agent leaderboards on 3,300 public Terminal-Bench runs. Neither reorders the ranking beyond noise, because with 66 tasks most gaps between neighbouring places are inside it.