arc-agi-benchmarking

Runner for ARC-AGI: multi-provider (OpenAI, Anthropic, Gemini, etc.), rate limits, retries, and scoring.

evalsprovider-agnosticpython

Stars: 365
Adoption surface: mostly simple
Autonomy: headless
Recovery: retry
License: ✅ open-source
Category: Evaluation and benchmarking harnesses

Repository ↗ Example: o3 prompt example ↗

Related in Evaluation and benchmarking harnesses

Agent Lightning
SWE-bench
AgentBench
inspect_ai
WebArena
ClawBench

Maintain arc-agi-benchmarking? Add the badge

Paste this into your README so readers can find the list and see how arc-agi-benchmarking compares:

[![Best of Agent Harnesses](https://img.shields.io/badge/%F0%9F%8F%86_Best_of-Agent_Harnesses-5ac4bf)](https://github.com/RyanAlberts/best-of-Agent-Harnesses)