SWE-bench

LMs resolve real GitHub issues; Docker harness, instance IDs; standard for code-agent evals.

evalssandboxpython

Stars: 6.1k
Adoption surface: slightly complex
Autonomy: headless
Recovery: resumable
License: ✅ open-source
Category: Evaluation and benchmarking harnesses

Repository ↗ Example: SWE-bench Verified leaderboard ↗

Related in Evaluation and benchmarking harnesses

Agent Lightning
AgentBench
inspect_ai
WebArena
ClawBench
WebVoyager

Maintain SWE-bench? Add the badge

Paste this into your README so readers can find the list and see how SWE-bench compares:

[![Best of Agent Harnesses](https://img.shields.io/badge/%F0%9F%8F%86_Best_of-Agent_Harnesses-5ac4bf)](https://github.com/RyanAlberts/best-of-Agent-Harnesses)