Is it getting better?
Scroll through HAI's history: every milestone, decision and result, oldest first. The score on the right follows you as you go.
All results
The full picture, one line per configuration. The x-axis is how many tasks the agent has worked through.
Cost per solved task
Total spend divided by tasks solved, including memory, search, verification and retries. Lower is better; the frontier line is the reference to get near.
Held-out pass rate
Share of unseen tasks solved, with memory frozen at that point. Higher is better.
Regression rate
Tasks that used to pass and now fail. Lower is better; this is the main check on self-improvement.
Steps per solved task
Average agent steps on tasks it solved. Lower means it's getting more efficient.
Memory items stored
Size of the memory store. Should level off as weak items are retired, not grow forever.