SUPER

Agents that set up and run ML/NLP from GitHub repos; 45 expert problems, 152 masked tasks, 602 AutoGen tasks; Docker-based.

sandboxpython

Stars: 58
Adoption surface: slightly complex
Autonomy: headless
Recovery: none
License: ✅ open-source
Category: Evaluation and benchmarking harnesses

Repository ↗ Example: SUPER EMNLP paper ↗

Related in Evaluation and benchmarking harnesses

Agent Lightning
SWE-bench
AgentBench
inspect_ai
WebArena
ClawBench

Maintain SUPER? Add the badge

Paste this into your README so readers can find the list and see how SUPER compares:

[![Best of Agent Harnesses](https://img.shields.io/badge/%F0%9F%8F%86_Best_of-Agent_Harnesses-5ac4bf)](https://github.com/RyanAlberts/best-of-Agent-Harnesses)