# Best of Agent Harnesses

> Hand-curated, ranked list of 167 AI agent harnesses — the runtimes that close the loop between a stateless model and the outside world. 12 categories, a 4-tier adoption-surface rating (simplicity ↔ capability), capability tags, a license signal, and one concrete example link per project. Stars captured 2026-09-20. The harness matters more than the model: the same weights score about 30% on ARC-AGI-3 bare and 95.5% inside Prime Agent, and harness rankings barely transfer across models, so pick harness and model as a pair (sources: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/why-the-harness-matters/ ; this week's top picks per category: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/best-ai-agent-harnesses-2026/ ).

Maintained at https://github.com/RyanAlberts/best-of-Agent-Harnesses (CC-BY-SA-4.0).
Structured data: https://raw.githubusercontent.com/RyanAlberts/best-of-Agent-Harnesses/main/harnesses.json
Tiers, least to most adoption surface: super simple → mostly simple → slightly complex → complex.

## Pick by use case

- I want a turnkey coding agent today: opencode (https://github.com/anomalyco/opencode), Cline (https://github.com/cline/cline), Codex (https://github.com/openai/codex), Gemini CLI (https://github.com/google-gemini/gemini-cli), OpenHands (https://github.com/OpenHands/OpenHands), crush (https://github.com/charmbracelet/crush), Prime Agent (https://github.com/PrimeIntellect-ai/prime-agent)
- I want an always-on personal agent that lives in my chat apps: OpenClaw (https://github.com/openclaw/openclaw), Hermes (https://github.com/NousResearch/hermes-agent), Khoj (https://github.com/khoj-ai/khoj), Agent Zero (https://github.com/agent0ai/agent-zero), OpenHarness (HKUDS) (https://github.com/HKUDS/OpenHarness), QM (https://github.com/yc-software/qm), OpenJarvis (https://github.com/open-jarvis/OpenJarvis)
- I want to extend Claude Code, Codex, or OpenCode with skills and slash commands: Anthropic Skills (https://github.com/anthropics/skills), wshobson/agents (https://github.com/wshobson/agents), superpowers (https://github.com/obra/superpowers), GStack (https://github.com/garrytan/gstack), pmstack (https://github.com/RyanAlberts/pmstack)
- I want to build my own coding harness from scratch: Claude Agent SDK (https://github.com/anthropics/claude-agent-sdk-python), Google ADK (https://github.com/google/adk-python), AutoHarness (https://github.com/aiming-lab/AutoHarness), SWE-agent (https://github.com/SWE-agent/SWE-agent), RepoMaster (https://github.com/QuantaAlpha/RepoMaster), claw-code-agent (https://github.com/HarnessLab/claw-code-agent)
- I want a drop-in memory layer for agents: Mem0 (https://github.com/mem0ai/mem0), Graphiti (Zep) (https://github.com/getzep/graphiti), claude-mem (https://github.com/thedotmack/claude-mem), agentlog (https://github.com/RyanAlberts/agentlog), letta (https://github.com/letta-ai/letta)
- I want to plug hundreds to thousands of tools without context bloat: MCP-Zero (https://github.com/xfey/MCP-Zero), ToolGen (https://github.com/Reason-Wang/ToolGen), ToolRAG (https://github.com/antl3x/ToolRAG), langgraph-bigtool (https://github.com/langchain-ai/langgraph-bigtool)
- I want multi-agent orchestration: openai-agents-python (https://github.com/openai/openai-agents-python), crewAI (https://github.com/crewAIInc/crewAI), autogen (https://github.com/microsoft/autogen), Microsoft Agent Framework (https://github.com/microsoft/agent-framework), PraisonAI (https://github.com/MervinPraison/PraisonAI), agent-squad (https://github.com/2FastLabs/agent-squad)
- I want a general LLM app framework: langgraph (https://github.com/langchain-ai/langgraph), langchain (https://github.com/langchain-ai/langchain), llama-index (https://github.com/run-llama/llama_index), pydantic-ai (https://github.com/pydantic/pydantic-ai), agno (https://github.com/agno-agi/agno)
- I want low-code / visual workflows: langflow (https://github.com/langflow-ai/langflow), Flowise (https://github.com/FlowiseAI/Flowise), Dify (https://github.com/langgenius/dify), n8n (https://github.com/n8n-io/n8n)
- I want browser-using agents: browser-use (https://github.com/browser-use/browser-use), Stagehand (https://github.com/browserbase/stagehand), WebVoyager (https://github.com/MinorJerry/WebVoyager), puppeteer-real-browser-mcp (https://github.com/withLinda/puppeteer-real-browser-mcp-server)
- I want sandboxed code execution for agent-generated code: E2B (https://github.com/e2b-dev/E2B), Agent Sandbox (https://github.com/kubernetes-sigs/agent-sandbox), Daytona (https://github.com/daytonaio/daytona), smolagents (https://github.com/huggingface/smolagents), OpenHands (https://github.com/OpenHands/OpenHands)
- I want to evaluate or benchmark agents: SWE-bench (https://github.com/SWE-bench/SWE-bench), Terminal-Bench (https://github.com/harbor-framework/terminal-bench), AgencyBench (https://github.com/GAIR-NLP/AgencyBench), inspect_ai (https://github.com/UKGovernmentBEIS/inspect_ai), WebArena (https://github.com/web-arena-x/webarena), VitaBench (https://github.com/meituan-longcat/vitabench)
- I want a deep research / autonomous research agent: deepagents (https://github.com/langchain-ai/deepagents), gpt-researcher (https://github.com/assafelovic/gpt-researcher), openagents (https://github.com/OpenAgentsInc/openagents)
- I want a provider-agnostic LLM pipe (not a framework): LiteLLM (https://github.com/BerriAI/litellm), vercel/ai (https://github.com/vercel/ai)

## Decision guides

- Agent evals: SWE-bench vs inspect_ai vs Terminal-Bench: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/agent-eval-harnesses/ — You changed your agent: new model, new prompt, new tools. Did it get better or worse? An eval is how you answer that with a number instead of a feeling. The word covers two different products, and knowing which one you need is most of the decision. A **benchmark** is a fixed public exam with a leade
- The best AI agent harnesses in 2026, ranked by category: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/best-ai-agent-harnesses-2026/ — These are the top three agent harnesses in each of the 12 categories of best-of-Agent-Harnesses, a hand-curated list of 167 harnesses re-ranked from live GitHub data every week; the stars below were captured 2026-09-20, and the page regenerates with every refresh.
- Browser agents: browser-use vs Stagehand vs Playwright MCP vs chrome-devtools-mcp: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/browser-agents/ — "Browser agent" covers three different kinds of product, and most bad picks here come from comparing across the lanes instead of within one. **Agent libraries** own the whole job: you hand them a goal in plain language ("find the cheapest flight, fill the form") and they decide every click. browser-
- Browser infrastructure for agents: Browserbase vs Steel vs Hyperbrowser: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/browser-infrastructure/ — Agent libraries like browser-use and Stagehand decide what to click; something still has to run the browsers they click in. At small scale that's Chrome on your own machine. At production scale it becomes its own operations problem: hundreds of concurrent sessions, sites that block automation, CAPTC
- Claude Code skill packs: superpowers vs GStack vs get-shit-done vs Anthropic Skills: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/claude-code-skill-packs/ — A skill is a folder of instructions (a SKILL.md file, plus any scripts it needs) that a coding agent loads only when the task matches, instead of carrying every instruction all the time. A skill pack is a curated bundle of them: someone else's working habits for your agent, installed as files. Nothi
- Eval and observability platforms: Langfuse vs LangSmith vs Braintrust vs Phoenix: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/eval-platforms/ — Benchmarks tell you how a model ranks; your production agent still fails in ways no public exam covers. An eval and observability platform is where teams watch what their agent actually did (tracing: recording every step, tool call, and token of a run) and score it continuously (evals: checks that r
- How to pick a harness: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/how-to-pick-a-harness/ — This is the decision guide for [best-of-Agent-Harnesses](../README.md), a curated, ranked list of the runtimes that turn an AI model into a working agent: the software that decides what the model's thinking is allowed to touch. Six questions, in order. Each one eliminates most of the list; by the en
- How to test-drive a harness: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/how-to-test-drive-a-harness/ — Spec sheets cannot answer "which harness should I use," because an agent's performance is a property of the *pairing* between harness and model, not of either alone: the same model passes [46% of tasks in one harness and 80% in another](https://www.mindstudio.ai/blog/agent-harness-scaffolding-matter
- Managed vs self-hosted always-on agents: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/managed-vs-self-hosted-always-on-agents/ — An always-on agent keeps working between your messages, and in 2026 you can rent one (Grok Bot, Claude Managed Agents), run one for your whole team (QM), run one for yourself (OpenClaw, Hermes), or keep one on your own laptop (OpenJarvis). This guide says who owns the computer in each case, who pays
- Agent memory layers: Mem0 vs Zep vs Letta vs claude-mem: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/memory-layers/ — Agents forget. A model keeps nothing between sessions, so anything your agent should still know tomorrow (who the user is, what was decided, what failed last time) has to live in a memory system outside the model. "Add memory to my agent" then hides genuinely different products, and picking by star 
- Multi-agent orchestration: OpenAI Agents SDK vs CrewAI vs AutoGen vs Agent Framework vs LangGraph: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/multi-agent-orchestration/ — Orchestration is the layer that coordinates several AI agents working on one job: who acts next, what they share, and what happens when a step fails. The five frameworks here answer that with genuinely different architectures, and the pick is expensive to get wrong because the coordination model sha
- OpenClaw vs Hermes: the always-on personal-agent debate: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/openclaw-vs-hermes/ — An always-on personal agent is a program that runs all day on your own machine, talks to you through the chat apps you already use (WhatsApp, Telegram, Slack, Discord), and keeps acting between your messages: watching inboxes, running scheduled jobs, remembering what you told it last month. OpenClaw
- Context files for agents: AGENTS.md vs CLAUDE.md vs skills vs MCP tool search: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/progressive-disclosure/ — A model has a context window: a fixed amount of text it can consider at once. Everything competes for that space: your instructions, the definitions of every tool the agent could call, and the output of every tool it already called. Context bloat is what happens when the "might need it" pile crowds 
- Agent sandboxing: what it is and how to pick: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/sandboxed-code-execution/ — An AI agent does not just suggest code. It runs code, opens web pages, and edits files on a real computer. Agent sandboxing means making that computer a disposable one, so that when the agent makes a mistake, or a malicious web page talks it into one, the damage lands in a box you can throw away ins
- Terminal coding agents: opencode vs Codex vs Gemini CLI vs crush vs goose: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/terminal-coding-agents/ — The most-asked pick in this list: *"I want a turnkey coding agent in my terminal today."* A terminal coding agent is a program you run in your shell that takes a plain-language request, then works in a loop: the model proposes an action (edit this file, run this command, search the repo), the tool e
- Why the harness matters more than the model: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/why-the-harness-matters/ — The same model weights score about 30% on ARC-AGI-3 as a bare model and 95.5% inside a good harness, and that gap is the number agent builders keep quoting. This page collects who says the harness matters more than the model, what each of them measured, and what the claim does not mean.

## FAQ

### What is the best agent harness if I want a turnkey coding agent today?
Top picks: opencode, Cline, Codex. See the “Coding agent products (IDEs, CLIs, full suites)” category for the full ranked list.

### What is the best agent harness if I want an always-on personal agent that lives in my chat apps?
Top picks: OpenClaw, Hermes, Khoj. See the “Personal agent runtimes” category for the full ranked list.

### What is the best agent harness if I want to extend Claude Code, Codex, or OpenCode with skills and slash commands?
Top picks: Anthropic Skills, wshobson/agents, superpowers. See the “Coding harness configs and SDKs” category for the full ranked list.

### What is the best agent harness if I want to build my own coding harness from scratch?
Top picks: Claude Agent SDK, Google ADK, AutoHarness. See the “Coding harness configs and SDKs” category for the full ranked list.

### What is the best agent harness if I want a drop-in memory layer for agents?
Top picks: Mem0, Graphiti (Zep), claude-mem. See the “Plugins, MCPs, CLI tools” category for the full ranked list.

### What is the best agent harness if I want to plug hundreds to thousands of tools without context bloat?
Top picks: MCP-Zero, ToolGen, ToolRAG. See the “Progressive disclosure harnesses” category for the full ranked list.

### What is the best agent harness if I want multi-agent orchestration?
Top picks: openai-agents-python, crewAI, autogen. See the “Multi-agent and orchestration” category for the full ranked list.

### What is the best agent harness if I want a general LLM app framework?
Top picks: langgraph, langchain, llama-index. See the “Frameworks” category for the full ranked list.

### What is the best agent harness if I want low-code / visual workflows?
Top picks: langflow, Flowise, Dify. See the “Frameworks” category for the full ranked list.

### What is the best agent harness if I want browser-using agents?
Top picks: browser-use, Stagehand, WebVoyager. See the “Plugins, MCPs, CLI tools” category for the full ranked list.

### What is the best agent harness if I want sandboxed code execution for agent-generated code?
Top picks: E2B, Agent Sandbox, Daytona. See the “Libraries and SDKs” category for the full ranked list.

### What is the best agent harness if I want to evaluate or benchmark agents?
Top picks: SWE-bench, Terminal-Bench, AgencyBench. See the “Evaluation and benchmarking harnesses” category for the full ranked list.

### What is the best agent harness if I want a deep research / autonomous research agent?
Top picks: deepagents, gpt-researcher, openagents. See the “Research and task-specific harnesses” category for the full ranked list.

### What is the best agent harness if I want a provider-agnostic LLM pipe (not a framework)?
Top picks: LiteLLM, vercel/ai. See the “Libraries and SDKs” category for the full ranked list.

### Which agent harnesses can run unattended (headless)?
Harnesses designed for unattended runs, batches, and fleets: opencode, OpenHands, goose, Symphony, Prime Agent, SWE-agent, Claude Agent SDK, RepoMaster.

### Which agent harnesses survive a crash mid-task (durable)?
Harnesses whose execution state persists across restarts: langgraph-bigtool, QM, n8n, langgraph, mastra, letta, deepagents, pydantic-ai.

### How many of these agent harnesses are open source?
124 of 167 carry a standard open-source license; the rest are source-available or unclear, and flagged per row.

### What is an agent harness?
The runtime that turns a model into an agent: it decides what the model's reasoning is allowed to touch, and supplies the orchestration, tool wiring, memory, error recovery, and guardrails around per-turn inference.

### Does the harness matter more than the model?
Often, yes, and measurably: the same weights scored about 30% on ARC-AGI-3 as a bare model and 95.5% inside the Prime Agent harness, and on SWE-bench Pro swapping only the harness moved GLM-5.2 from 23% to 52%. Harness rankings barely transfer across models (rank correlation about -0.05), so pick the harness and the model as a pair, and re-pick when the model changes.
More: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/why-the-harness-matters/

### Is Grok Bot an agent harness?
Yes, a managed one: xAI owns the loop, the tool wiring, the memory, and the approval rules, and every Bot on an account shares one cloud computer. It is not in the ranked list because the list ranks open repositories; the managed-vs-self-hosted guide compares it with Claude Managed Agents, QM, OpenClaw, Hermes, and OpenJarvis.
More: https://ryanalberts.github.io/best-of-Agent-Harnesses/compare/managed-vs-self-hosted-always-on-agents/

### How is this list ranked?
By relevance to harness concerns (environment, orchestration, lifecycle, guardrails) and by GitHub stars (captured 2026-09-20); each project also carries an adoption-surface tier and autonomy/recovery scores.

### How can an AI agent use this list directly?
Three machine-readable surfaces: harnesses.json (structured), llms.txt (one file), and an MCP server (uvx agent-harnesses-mcp) exposing recommend, compare, pick_harness, and search_harnesses.

## Progressive disclosure harnesses (8 projects)

Formats, runtimes, and patterns that reveal context, tools, or instructions in layers—index first, details on demand—to control tokens and improve agent focus (the "map, not encyclopedia" principle).

- [Headroom](https://github.com/headroomlabs-ai/headroom) — ⭐73.2k, mostly simple, autonomy: n/a, recovery: n/a, open-source: Compresses tool outputs, logs, files, and RAG chunks with content-aware compressors before they reach the model—claimed 20% fewer tokens for coding agents and 60–95% fewer for JSON, same answers. Ships as a library, HTTP proxy, or MCP server, so it drops in front of whatever harness you already run. [mcp, rag]
- [awesome-cursorrules](https://github.com/PatrickJS/awesome-cursorrules) — ⭐40.8k, super simple, autonomy: n/a, recovery: n/a, open-source: Curated .cursorrules and skills that leverage Cursor's index-then-load model; the canonical collection for rules-as-progressive-disclosure in the IDE. [ide]
- [agents.md](https://github.com/agentsmd/agents.md) — ⭐24.5k, super simple, autonomy: n/a, recovery: n/a, open-source: Open format for repo-scoped agent briefings; nested AGENTS.md files scope instructions per directory, so agents get a map of what exists and load only what's relevant. Read by 20+ tools including Codex, Cursor, and Copilot. [ide, typescript]
- [context-mode](https://github.com/mksglu/context-mode) — ⭐23.8k, mostly simple, autonomy: n/a, recovery: n/a, restricted (Elastic-2.0): Context-window optimization layer that sandboxes tool output before it reaches the model (claimed 98% reduction) and persists session memory across 17 agent platforms via MCP and hooks—progressive disclosure applied to tool results, not just instructions. [mcp, memory, sandbox]
- [langgraph-bigtool](https://github.com/langchain-ai/langgraph-bigtool) — ⭐558, slightly complex, autonomy: bounded, recovery: durable, open-source: Build LangGraph agents with large tool sets; retrieval and on-demand tool loading so agents scale beyond context without stuffing every schema upfront. [tool-discovery, python]
- [MCP-Zero](https://github.com/xfey/MCP-Zero) — ⭐512, complex, autonomy: bounded, recovery: none, open-source: Active tool discovery for autonomous agents: model requests tools by requirement; hierarchical semantic routing over 308 servers / 2,797 tools with ~98% token reduction (APIBank). [tool-discovery]
- [ToolGen](https://github.com/Reason-Wang/ToolGen) — ⭐184, complex, autonomy: n/a, recovery: n/a, unknown: ICLR 2025: unified tool retrieval and calling via generation; 47k+ tools without context stuffing—retrieval and invocation in one generative step. [tool-discovery, python]
- [ToolRAG](https://github.com/antl3x/ToolRAG) — ⭐34, mostly simple, autonomy: n/a, recovery: n/a, open-source: Semantic tool retrieval for LLMs; serves only the tools the user query demands (MCP-compatible), unlimited tool sets with zero context penalty. [mcp, tool-discovery]

## Coding agent products (IDEs, CLIs, full suites) (24 projects)

Turnkey coding agents you install and run: IDE extensions, terminal CLIs, Dockerized workspaces. Each entry notes which part is the harness (the agent loop, tool wiring, approval model) versus the UI shell (VS Code extension, TUI, browser client).

- [opencode](https://github.com/anomalyco/opencode) — ⭐209k, slightly complex, autonomy: headless, recovery: resumable, open-source: Open-source terminal coding agent (formerly `sst/opencode`; transferred to anomalyco). The **harness** is a multi-provider tool-call loop (Claude, OpenAI, Gemini, local) with strong plugin and MCP support; the TUI is the shell. 100% OSS, very actively shipped. [mcp, provider-agnostic, cli, tui, typescript]
- [Codex](https://github.com/openai/codex) — ⭐125k, slightly complex, autonomy: bounded, recovery: resumable, open-source: OpenAI's terminal coding agent. The **harness** is the sandboxed tool-call loop with multi-provider support; the CLI is the shell. Reference implementation for "official CLI that ships code." [sandbox, provider-agnostic, cli]
- [pi](https://github.com/earendil-works/pi) — ⭐108k, slightly complex, autonomy: bounded, recovery: resumable, unknown: The upstream AI agent toolkit behind this list's oh-my-pi fork: a unified multi-provider LLM API, agent loop, and TUI shell providing the **harness** that oh-my-pi's Rust rewrite builds on. [provider-agnostic, tui, rust]
- [Gemini CLI](https://github.com/google-gemini/gemini-cli) — ⭐107k, slightly complex, autonomy: bounded, recovery: resumable, open-source: Google's first-party terminal agent for Gemini. The **harness** is the plugin/MCP tool-call loop; the terminal is the shell—Google's parallel to Claude Code / Codex, not just an API. [mcp, cli, typescript]
- [OpenHands](https://github.com/OpenHands/OpenHands) — ⭐88.6k, complex, autonomy: headless, recovery: resumable, restricted ((multi-license)): Dockerized software-engineering agent. The **harness** is the bash/editor/browser toolset with micro-agents and event-stream session bridging; Docker is the sandbox. Main OSS choice for teams self-hosting autonomous repo work. [memory, browser, sandbox, python]
- [Cline](https://github.com/cline/cline) — ⭐68.9k, slightly complex, autonomy: step-gated, recovery: resumable, open-source: VS Code extension whose **harness** is a plan-then-act loop with per-step human approval and cost transparency; the VS Code integration is the UI shell. Open-source counterweight to Cursor. [ide, typescript]
- [Open Interpreter](https://github.com/openinterpreter/openinterpreter) — ⭐68.4k, mostly simple, autonomy: bounded, recovery: resumable, open-source: Lightweight terminal coding agent oriented to open models (DeepSeek, Kimi, Qwen). The **harness** is a code-execution loop — the model writes code, the harness executes it with confirmation gates; the CLI is the shell. The original "let the LLM run code on my machine" project, reborn for open weights. [cli, python]
- [goose](https://github.com/aaif-goose/goose) — ⭐54.5k, slightly complex, autonomy: headless, recovery: resumable, open-source: Block-originated Rust agent, now stewarded by the Linux Foundation's Agentic AI Foundation (`aaif-goose/goose`). The **harness** is the MCP/ACP extension model with recipes and provider choice; there's no fixed UI slot—you bolt it into whatever shell you use. [mcp, rust]
- [DeepSeek-Reasonix](https://github.com/esengine/DeepSeek-Reasonix) — ⭐35.6k, slightly complex, autonomy: n/a, recovery: n/a, unknown: DeepSeek-native terminal coding agent. The **harness** is engineered around prefix-cache stability for long-running sessions; the TUI is the shell. [memory, cli, tui, typescript]
- [oh-my-pi](https://github.com/can1357/oh-my-pi) — ⭐32.1k, slightly complex, autonomy: bounded, recovery: resumable, open-source: Terminal coding agent (fork of Pi) that wires the IDE into the **harness**: hash-anchored edits, a 32-tool loop tuned per-model, LSP rename/references/diagnostics on every write, a real DAP debugger (lldb/dlv/debugpy), long-lived Python + Bun execution kernels that call back into the agent's tools, browser control, and 40+ providers (Claude/OpenAI/Gemini/local). ~55k-line Rust core. [browser, provider-agnostic, cli, ide, rust, python]
- [crush](https://github.com/charmbracelet/crush) — ⭐28.2k, slightly complex, autonomy: bounded, recovery: resumable, restricted (FSL-1.1-MIT): Charm's terminal coding agent (Charm's fork of the original OpenCode). The **harness** is the tool-calling loop with session persistence; the Bubble Tea TUI is the shell. [memory, cli, tui]
- [vibe-kanban](https://github.com/BloopAI/vibe-kanban) — ⭐28.1k, slightly complex, autonomy: n/a, recovery: n/a, unknown: Kanban-style fleet manager for running Claude Code, Codex, or any coding agent across many tasks at once. The **harness** contribution is the task-queue/review layer on top of whichever agent executes; not an agent loop itself.
- [qwen-code](https://github.com/QwenLM/qwen-code) — ⭐28k, slightly complex, autonomy: bounded, recovery: resumable, unknown: Alibaba's official terminal coding agent, forked from Gemini CLI's agent loop and retuned for Qwen models. The **harness** is the same sandboxed tool-call loop as its upstream; the terminal is the shell. [sandbox, cli, typescript]
- [Kilo Code](https://github.com/Kilo-Org/kilocode) — ⭐27.4k, slightly complex, autonomy: step-gated, recovery: resumable, unknown: VS Code extension and CLI in the Cline/Roo-Code lineage — a natural pick now that Roo-Code is archived upstream. The **harness** is an approval-gated autonomous-mode loop with a provider/tool marketplace; the IDE is the shell. [mcp, cli, ide, typescript]
- [Symphony](https://github.com/openai/symphony) — ⭐27.3k, complex, autonomy: headless, recovery: resumable, unknown: OpenAI's harness for fanning a task out into many isolated, autonomous coding-agent implementation runs and surfacing the ones that pass, so a team manages outcomes instead of supervising each session. [sandbox]
- [Roo Code](https://github.com/RooCodeInc/Roo-Code) — ⭐24.3k, slightly complex, autonomy: step-gated, recovery: resumable, open-source: VS Code/Cursor extension in the Cline lineage. The **harness** is the approval-gated agent with custom modes and a strong MCP story; the IDE is the UI. Popular community fork when you want that workflow without the upstream extension. [mcp, workflow, ide, typescript]
- [Prime Agent](https://github.com/PrimeIntellect-ai/prime-agent) — ⭐21.1k, slightly complex, autonomy: headless, recovery: resumable, open-source: Prime Intellect's self-improving coding and research agent. The **harness** is a recursive language model loop (context as a variable, sub-agents as function calls inside a persistent IPython REPL) plus a Continual Harness that lets the agent create, read, update, and delete its own prompts, skills, memory, and sub-agents mid-run; daemon-backed sessions survive a closed terminal, and autonomous mode runs on turn, token, and time budgets. 95.5% on ARC-AGI-3 with Opus 5; the terminal UI is the shell. [memory, multi-agent, cli, typescript]
- [jcode](https://github.com/1jehuang/jcode) — ⭐19.9k, slightly complex, autonomy: n/a, recovery: n/a, unknown: Rust terminal coding agent pitched as the most RAM-efficient **harness** in its class; MCP support, multi-provider (Claude/OpenAI). [mcp, memory, provider-agnostic, cli, rust]
- [eigent](https://github.com/eigent-ai/eigent) — ⭐15.3k, complex, autonomy: n/a, recovery: n/a, unknown: Open-source desktop **harness** positioned as a local, free alternative to Claude Cowork and Codex: multi-agent workspace orchestration in a self-hosted app rather than a hosted product. [multi-agent, local]
- [cc-haha](https://github.com/NanmiCoder/cc-haha) — ⭐14.7k, complex, autonomy: n/a, recovery: n/a, unknown: Local-first desktop workspace **harness** for Claude Code and other agents: multi-agent sessions, Git worktrees, code diffs, a skill marketplace, and chat-app access (WeChat, Telegram, WhatsApp). [memory, multi-agent, typescript]
- [claw-code-agent](https://github.com/HarnessLab/claw-code-agent) — ⭐545, slightly complex, autonomy: checkpoint-gated, recovery: none, unknown: Python reimplementation of the Claude Code agent architecture with zero external dependencies; interactive chat, streaming, plugin runtime, nested agent delegation, cost tracking, MCP transport—portable harness without the Rust/TS toolchain. [mcp, rust, python, typescript]
- [Proliferate](https://github.com/proliferate-ai/proliferate) — ⭐505, complex, autonomy: bounded, recovery: resumable, open-source: Open-source AI IDE for Claude Code, Codex, OpenCode, and more. The **harness** contribution is the workspace/session orchestration layer: run multiple coding agents in parallel, locally or in the cloud, with isolated workspaces, reusable workflows, and shared team context. [multi-agent, sandbox, ide, typescript]
- [AgentBox](https://github.com/madarco/agentbox) — ⭐467, slightly complex, autonomy: n/a, recovery: n/a, open-source: Runs multiple coding agents in parallel, each in its own sandboxed VM, locally or in the cloud, from one command. The **harness** contribution is the VM-per-agent isolation and fleet fan-out layer; whichever agent runs inside owns the loop. [sandbox, typescript]
- [YYLO](https://github.com/yylo-dev/yylo) — ⭐60, slightly complex, autonomy: n/a, recovery: n/a, open-source: Command-line orchestrator for coding agents (drives Pi and Codex subagents): every task gets a dedicated branch/worktree and a typed lifecycle — read-only preflight, risk-scaled merge-queue review, release-readiness gates — with receipt-backed changes. The **harness** contribution is the git-native task/merge orchestration layer on top of whichever agent executes; not an agent loop itself. [multi-agent, typescript]

## Coding harness configs and SDKs (17 projects)

Skill packs, slash-command libraries, meta-prompting frameworks, and official SDKs that give you the harness (the agent loop, planning, memory, hooks) without bundling a specific IDE or CLI shell.

- [superpowers](https://github.com/obra/superpowers) — ⭐289k, complex, autonomy: n/a, recovery: n/a, open-source: Performance-oriented harness pack for Claude Code and 13 other harnesses (Codex, Cursor, OpenCode, Gemini CLI, more): skills, instincts, memory, security, research-first workflows. Treats harness engineering itself as the performance lever. [memory, cli, ide]
- [Anthropic Skills](https://github.com/anthropics/skills) — ⭐177k, mostly simple, autonomy: n/a, recovery: n/a, restricted (Anthropic terms): Anthropic's official Agent Skills repository: SKILL.md-based folders (instructions, scripts, resources) Claude dynamically loads on Claude Code, Claude.ai, and the API. The reference for progressive-disclosure skill packs in 2026.
- [GStack](https://github.com/garrytan/gstack) — ⭐134k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Garry Tan's Claude Code skill stack: 23 slash-command modes (CEO/eng/design review, QA, ship, browse, retro, …) that structure one assistant as a virtual engineering team. Daily driver while running YC. [typescript]
- [addyosmani/agent-skills](https://github.com/addyosmani/agent-skills) — ⭐97.5k, mostly simple, autonomy: n/a, recovery: n/a, open-source: Addy Osmani's production-grade skill pack: 24 engineering skills and 4 specialist agent personas that encode senior-dev workflows (spec through deploy) across 70+ coding agents including Claude Code, Cursor, and Copilot. The **harness** contribution is the skill/workflow layer, not a new agent loop. [workflow, ide]
- [awesome-claude-code](https://github.com/hesreallyhim/awesome-claude-code) — ⭐54.3k, super simple, autonomy: n/a, recovery: n/a, unknown: Large community-curated index of Claude Code skills, slash commands, status lines, and plugins—resources for extending the harness, not a harness itself, but the most-followed catalog of the genre.
- [wshobson/agents](https://github.com/wshobson/agents) — ⭐39.8k, super simple, autonomy: n/a, recovery: n/a, open-source: Cross-harness marketplace of drop-in subagents and skills for Claude Code, Codex CLI, Cursor, OpenCode, and Copilot; specialized, production-ready agent definitions you install rather than hand-write. [multi-agent, cli, ide]
- [planning-with-files](https://github.com/OthmanAdi/planning-with-files) — ⭐27k, mostly simple, autonomy: n/a, recovery: n/a, unknown: Skill for persistent, file-based planning across long-running coding-agent sessions: crash-proof markdown plans, session recovery after `/clear`/compaction, and a deterministic completion gate—Manus-style planning as a drop-in **harness** layer via the Agent Skills standard. [memory]
- [SWE-agent](https://github.com/SWE-agent/SWE-agent) — ⭐20.4k, slightly complex, autonomy: headless, recovery: resumable, open-source: LM-driven harness built for SWE-bench: edit state, command execution, and issue-focused loop—the reference agent stack next to the benchmark itself. [memory, evals, python]
- [get-shit-done](https://github.com/open-gsd/gsd-core) — ⭐9.7k, mostly simple, autonomy: bounded, recovery: resumable, open-source: Goal-backward planning and wave-based execution over fresh context windows; avoids context rot by design. Python/JS meta-prompting for Claude Code, OpenCode, Gemini CLI. [cli, python]
- [Claude Agent SDK](https://github.com/anthropics/claude-agent-sdk-python) — ⭐8.1k, complex, autonomy: headless, recovery: resumable, open-source: Official Anthropic SDK (Python + [TypeScript](https://github.com/anthropics/claude-agent-sdk-typescript), [demos](https://github.com/anthropics/claude-agent-sdk-demos), [quickstarts](https://github.com/anthropics/claude-quickstarts)): built-in tools, MCP, long-running coding agents with session bridging. [mcp, memory, python, typescript]
- [agents-cli](https://github.com/google/agents-cli) — ⭐6k, mostly simple, autonomy: n/a, recovery: n/a, unknown: Google's official CLI and skill pack that layers agent-creation, evaluation, and deployment skills on top of whatever coding assistant you already run, rather than shipping its own agent loop—the **harness** as a config/skills add-on, not a new runtime. [evals, cli]
- [skillhub](https://github.com/iflytek/skillhub) — ⭐5.1k, mostly simple, autonomy: n/a, recovery: n/a, unknown: iFlytek's self-hosted registry for publishing, versioning, and governing agent skill packages—the **harness** config layer treated as an enterprise artifact store rather than a CLI or IDE shell. [local, cli, ide]
- [Meta-Harness](https://github.com/stanford-iris-lab/meta-harness) — ⭐1.6k, slightly complex, autonomy: n/a, recovery: n/a, unknown: Reference implementation from the Meta-Harness paper: an academic testbed for harness-engineering research, not a product—useful as a citation-grade baseline rather than something you'd run in production.
- [RepoMaster](https://github.com/QuantaAlpha/RepoMaster) — ⭐553, slightly complex, autonomy: headless, recovery: none, unknown: Repo-scoped research harness: builds function-call and module-dependency graphs to explore only what's needed; large relative gains on MLE-bench and GitTaskBench with lower token use. [workflow, python]
- [AutoHarness](https://github.com/aiming-lab/AutoHarness) — ⭐378, super simple, autonomy: bounded, recovery: none, open-source: Lightweight governance harness: wraps any LLM client in ~2 lines for automated harness engineering—6–14 step pipeline, YAML constitution, risk-pattern matching, session persistence with cost tracking, multi-agent profiles. [memory, multi-agent, provider-agnostic, python]
- [LoopTroop](https://github.com/looptroop-ai/LoopTroop) — ⭐150, mostly simple, autonomy: bounded, recovery: retry, open-source: Config layer that chains LLM councils for planning, Ralph loops for iterative refinement, and OpenCode worktrees for shipping. The **harness** contribution is the council → loop → worktree pipeline; OpenCode underneath executes. [typescript]
- [pmstack](https://github.com/RyanAlberts/pmstack) — ⭐8, super simple, autonomy: n/a, recovery: n/a, open-source: Claude Code config for AI product managers: CLAUDE.md plus skills for competitive analysis, PRD-from-signal, metric frameworks, stakeholder briefs, and agent eval design. "GStack for PMs." [evals]

## Personal agent runtimes (13 projects)

Always-on, self-hosted agents you run as a daemon and talk to from chat apps: gateway runtimes, second brains, and self-improving assistants. The agent as a product you operate, not a library you build with.

- [OpenClaw](https://github.com/openclaw/openclaw) — ⭐390k, complex, autonomy: headless, recovery: resumable, open-source: Self-hosted, always-on personal agent (formerly Clawdbot/Moltbot): a gateway + event-loop runtime that treats messages, heartbeats, crons, and webhooks as one input queue, persists state to local files, and lives in your chat apps (WhatsApp, Telegram, Slack, Discord). 13,700+ community skills; the fastest-growing repo in GitHub history. [typescript, multi-agent]
- [Hermes](https://github.com/NousResearch/hermes-agent) — ⭐247k, slightly complex, autonomy: headless, recovery: resumable, open-source: Nous Research's self-improving agent: a learning loop turns experience into reusable skills, builds a persistent user model across sessions, and checkpoints state to disk with rollback; lean enough for a $5 VPS, driven from chat, and model-agnostic (Nous Portal, OpenRouter, OpenAI, or any endpoint). [memory, python, provider-agnostic]
- [AnythingLLM](https://github.com/Mintplex-Labs/anything-llm) — ⭐66.3k, complex, autonomy: headless, recovery: resumable, open-source: Self-hosted "AI second brain" **harness**: chat with your documents, run built-in agent skills (web search, code execution, browsing), and manage multi-user workspaces with a bundled vector DB, so there is no separate RAG stack to wire up. [rag, typescript]
- [nanobot](https://github.com/HKUDS/nanobot) — ⭐48.4k, mostly simple, autonomy: n/a, recovery: n/a, unknown: Ultra-lightweight, self-hosted personal agent framework: the **harness** is a Python daemon wiring tools, memory, and MCP into chat/webhook front ends (Telegram, Discord, web); minimal footprint alternative to heavier personal-runtime stacks. [mcp, memory, local, python]
- [CowAgent](https://github.com/zhayujie/CowAgent) — ⭐47.1k, slightly complex, autonomy: n/a, recovery: n/a, unknown: Self-hosted **harness** (formerly chatgpt-on-wechat) that plans tasks, runs tools/skills, and self-evolves via memory; multi-model, multi-channel (WeChat, Telegram, etc.), one-line install. [memory, python]
- [Khoj](https://github.com/khoj-ai/khoj) — ⭐37.4k, complex, autonomy: headless, recovery: resumable, open-source: Self-hostable "AI second brain": answers over your docs and the web, custom agents, scheduled automations, and multi-client reach (web, Obsidian, Emacs, WhatsApp). A personal-agent harness with retrieval at the core. [python]
- [Eliza](https://github.com/elizaOS/eliza) — ⭐19.4k, complex, autonomy: headless, recovery: resumable, open-source: Open "agentic operating system" (elizaOS): persistent multi-agent runtime with character files, a plugin ecosystem, and social/platform integrations — the harness behind a large share of autonomous social agents. [memory, multi-agent, typescript]
- [Agent Zero](https://github.com/agent0ai/agent-zero) — ⭐19.2k, slightly complex, autonomy: bounded, recovery: resumable, unknown: Organic, prompt-defined personal agent framework: hierarchical sub-agents, persistent memory, browser and code tools, and self-modifying behavior; runs in Docker with a web UI. [memory, multi-agent, browser, sandbox, python]
- [OpenHarness (HKUDS)](https://github.com/HKUDS/OpenHarness) — ⭐15.8k, complex, autonomy: bounded, recovery: resumable, open-source: Open agent harness with a built-in personal agent ("Ohmo") that runs across Feishu, Slack, Telegram, and Discord; core tool-use, skills, memory, multi-agent coordination with auto-compaction for multi-day sessions. [memory, multi-agent]
- [QM](https://github.com/yc-software/qm) — ⭐15.2k, complex, autonomy: headless, recovery: durable, open-source: Y Combinator's multiplayer agent **harness** for work, open-sourced from months of internal use: every person and room gets scoped memory, files, credentials, permissions, crons, web apps, and a durable sandbox; sessions, memory, and the work queue live in Postgres, sandboxes are a resource the agent reaches for rather than the place it lives, and Pi, OpenCode, Codex, or Claude Code can drive the same core. Slack and the web app are the shells. [memory, sandbox, typescript]
- [OpenJarvis](https://github.com/open-jarvis/OpenJarvis) — ⭐10k, slightly complex, autonomy: headless, recovery: none, open-source: Stanford Hazy Research's local-first personal AI **harness**: on-device model inference (Ollama built in), agent execution, memory, and learning as shared primitives, with on-demand, scheduled, and continuous agents and evaluations that count energy, latency, and dollars alongside accuracy; a cloud model can tune the local configuration once so run time stays on your own hardware. [memory, local, python]
- [AIlice](https://github.com/myshell-ai/AIlice) — ⭐1.4k, slightly complex, autonomy: bounded, recovery: none, open-source: Fully autonomous general-purpose agent; one binary, Docker-ready, for when you want "set goal and walk away" without a framework. [sandbox, python]
- [Talon](https://github.com/dylanneve1/talon) — ⭐83, slightly complex, autonomy: headless, recovery: resumable, open-source: Multi-platform personal agent living in Telegram, Discord, Teams, and the terminal. The **harness** is a pluggable-backend loop (Claude, Kilo, OpenCode, Codex, OpenAI Agents) with full MCP tool access and persistent background agents (Goals, Heartbeat, Dream); the chat apps are shells. [mcp, memory, cli, typescript]

## Frameworks (26 projects)

General-purpose agent and LLM application frameworks (the app layer, not harnesses per se).

- [n8n](https://github.com/n8n-io/n8n) — ⭐205k, complex, autonomy: headless, recovery: durable, restricted (Fair-code): Fair-code workflow engine with 400+ nodes and native AI nodes; the self-hosted Zapier that actually does agents and LangChain. [workflow, local, typescript]
- [AutoGPT](https://github.com/Significant-Gravitas/AutoGPT) — ⭐187k, complex, autonomy: headless, recovery: resumable, restricted (Polyform-SU): The original autonomous loop: goal in, agent iterates with tools and memory; Forge is the dev framework, Benchmark the eval harness. [memory, evals, python]
- [Dify](https://github.com/langgenius/dify) — ⭐157k, complex, autonomy: headless, recovery: retry, restricted (Fair-code): One-stop LLM app platform: visual workflows, RAG pipeline, 50+ tools, model management; "ship from prototype to prod" in a single UI. [low-code, rag, python]
- [langflow](https://github.com/langflow-ai/langflow) — ⭐155k, complex, autonomy: headless, recovery: retry, open-source: Low-code UI to build and deploy LangChain/LangGraph flows; visual DAG editor and one-click run. [low-code, python]
- [langchain](https://github.com/langchain-ai/langchain) — ⭐147k, complex, autonomy: bounded, recovery: retry, open-source: Chains, tools, retrievers, and agents; the usual entry point for "add tools to an LLM" in Python/JS. [python]
- [browser-use](https://github.com/browser-use/browser-use) — ⭐116k, slightly complex, autonomy: bounded, recovery: retry, open-source: Python web-agent **harness**: natural-language goals become browser actions, driven directly over the Chrome DevTools Protocol (it dropped Playwright in August 2025). The biggest community in the browser-agent category. [browser, python]
- [Flowise](https://github.com/FlowiseAI/Flowise) — ⭐55.5k, complex, autonomy: headless, recovery: retry, restricted (Apache+CLA): Drag-and-drop LangChain UI; deploy flows without code. The low-code sibling to Langflow, with a different component and hosting story. [low-code, typescript]
- [llama-index](https://github.com/run-llama/llama_index) — ⭐52.2k, complex, autonomy: bounded, recovery: retry, open-source: Data-centric: indexing, RAG, and query engines; agent abstractions sit on top of your data pipelines. [rag, python]
- [agno](https://github.com/agno-agi/agno) — ⭐42.3k, complex, autonomy: bounded, recovery: resumable, open-source: Python agents with memory, knowledge bases, tools, and structured outputs; continues the PhiData-era product line under the Agno name—production apps, evals, and pipelines. [memory, evals, python]
- [langgraph](https://github.com/langchain-ai/langgraph) — ⭐42k, slightly complex, autonomy: headless, recovery: durable, open-source: State-machine graphs over LLM steps; checkpointing, human-in-the-loop, and durable execution so workflows survive restarts. [workflow, python]
- [semantic-kernel](https://github.com/microsoft/semantic-kernel) — ⭐28.6k, complex, autonomy: bounded, recovery: retry, open-source: Microsoft's plugin and planner layer for LLMs; C#, Python, Java; strong on enterprise auth and orchestration. [python]
- [mastra](https://github.com/mastra-ai/mastra) — ⭐28.2k, slightly complex, autonomy: bounded, recovery: durable, restricted (Elastic-2.0): TypeScript-first; agents, tools, and workflows with a single runtime and minimal boilerplate. [typed, typescript]
- [Haystack](https://github.com/deepset-ai/haystack) — ⭐26.6k, complex, autonomy: n/a, recovery: n/a, open-source: Open-source orchestration framework for context-engineered LLM apps: modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation—closer to LangChain's territory than a coding-agent harness. [memory, rag, python]
- [letta](https://github.com/letta-ai/letta) — ⭐24.8k, mostly simple, autonomy: headless, recovery: durable, open-source: Python agent runtime with tool use and control flow; lean API; stateful agents with long-horizon memory. [memory, python]
- [Stagehand](https://github.com/browserbase/stagehand) — ⭐24.6k, slightly complex, autonomy: bounded, recovery: none, open-source: Browserbase's SDK for browser agents: natural-language actions (act, extract, observe) and deterministic Playwright code mix in one script, so agent flexibility and repeatable automation live in the same **harness**. [browser, typescript]
- [Google ADK](https://github.com/google/adk-python) — ⭐21.6k, complex, autonomy: headless, recovery: resumable, open-source: Google's official Agent Development Kit: code-first Python toolkit for building, evaluating, and deploying agents. Optimized for Gemini but model-agnostic; deploys to Cloud Run / Vertex AI; ships a dev UI with eval and a code-execution sandbox. [evals, sandbox, python]
- [rasa](https://github.com/RasaHQ/rasa) — ⭐21.3k, complex, autonomy: headless, recovery: resumable, open-source: Conversational AI stack (NLU, dialogue, actions); long-standing OSS choice for chat and voice bots. [voice, python]
- [botpress](https://github.com/botpress/botpress) — ⭐14.9k, complex, autonomy: headless, recovery: resumable, open-source: Visual bot builder and runtime; multi-channel, open-source alternative to commercial bot platforms. [low-code, typescript]
- [R2R](https://github.com/SciPhi-AI/R2R) — ⭐8k, complex, autonomy: headless, recovery: retry, open-source: RAG-first: hybrid search, knowledge graphs, multimodal; the framework for "production RAG" when you care more about retrieval than chat UI. [vision, rag, workflow, python]
- [agent-squad](https://github.com/2FastLabs/agent-squad) — ⭐7.8k, slightly complex, autonomy: bounded, recovery: resumable, open-source: AWS-originated orchestrator (now under 2FastLabs): intent classification, streaming, SupervisorAgent; "agent-as-tools" so one agent delegates to a squad. [multi-agent]
- [AgentVerse](https://github.com/OpenBMB/AgentVerse) — ⭐5.1k, complex, autonomy: headless, recovery: none, open-source: Task-solving and simulation envs for multi-LLM agents; deploy many agents in custom environments without building infra from scratch. [multi-agent, python]
- [youtu-agent](https://github.com/TencentCloudADP/youtu-agent) — ⭐4.6k, mostly simple, autonomy: bounded, recovery: retry, unknown: Tencent Cloud's agent framework: a minimal tool-calling **harness** designed to perform well with open-source models, positioned as a lighter alternative to heavier orchestration frameworks.
- [Bee Agent Framework](https://github.com/i-am-bee/beeai-framework) — ⭐3.4k, complex, autonomy: bounded, recovery: resumable, open-source: Python + TypeScript, LF AI–backed; MCP/ACP, workflows, Requirement Agent; the one that pushes "production multi-agent" without LangChain. [mcp, multi-agent, python, typescript]
- [AgentStack](https://github.com/agentstack-ai/AgentStack) — ⭐2.2k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Scaffolds full agent projects; plugs in CrewAI, LangGraph, OpenAI Swarm, LlamaStack and wires AgentOps observability from day one.
- [AgentSilex](https://github.com/howl-anderson/agentsilex) — ⭐456, super simple, autonomy: bounded, recovery: none, open-source: ~300 lines of readable agent code on top of LiteLLM; the "I want to see the whole loop" option for learning or minimal production. [python]
- [SuperAgentX](https://github.com/superagentxai/superagentx) — ⭐203, mostly simple, autonomy: bounded, recovery: none, open-source: Lightweight multi-agent orchestrator with an AGI-angle; minimal surface, docs-first, for teams that want orchestration without the kitchen sink. [multi-agent, python]

## Multi-agent and orchestration (12 projects)

Harnesses and patterns for multi-agent coordination and handoffs.

- [MetaGPT](https://github.com/FoundationAgents/MetaGPT) — ⭐70.5k, complex, autonomy: headless, recovery: resumable, open-source: The "AI software company" multi-agent framework: role-played PM, architect, and engineer agents turn a one-line requirement into specs, designs, and code along an SOP assembly line. The landmark of the genre; development pace has slowed in 2026. [multi-agent, python]
- [autogen](https://github.com/microsoft/autogen) — ⭐61.1k, complex, autonomy: bounded, recovery: resumable, open-source: Conversable agents and group chats; code execution and human-in-the-loop; Microsoft origin, AG2 ecosystem. ⚠️ In maintenance mode since late 2025: no new features, and Microsoft directs new users to Agent Framework. [multi-agent, python]
- [crewAI](https://github.com/crewAIInc/crewAI) — ⭐58.8k, complex, autonomy: bounded, recovery: resumable, open-source: Role-based agents (roles, goals, backstories) in Crews; Flows add event-driven and hierarchical control for production. [python]
- [OpenManus](https://github.com/FoundationAgents/OpenManus) — ⭐58.4k, complex, autonomy: bounded, recovery: none, open-source: Open, invite-free general agent from the MetaGPT team: planning plus tool use over a multi-agent loop, aimed at reproducing Manus-style autonomous task completion on your own keys. [multi-agent, python]
- [ChatDev](https://github.com/OpenBMB/ChatDev) — ⭐34.3k, slightly complex, autonomy: headless, recovery: none, open-source: Multi-agent software-company simulation (CEO, CTO, programmer, tester) built on chat chains with communicative dehallucination; ChatDev 2.0 continues the line. MetaGPT's conversational sibling. [python]
- [openai-agents-python](https://github.com/openai/openai-agents-python) — ⭐29.6k, mostly simple, autonomy: bounded, recovery: resumable, open-source: Handoffs, guardrails, and multi-LLM routing; minimal surface so you own the loop. [python]
- [Microsoft Agent Framework](https://github.com/microsoft/agent-framework) — ⭐13.6k, slightly complex, autonomy: bounded, recovery: resumable, open-source: Microsoft's convergence of AutoGen and Semantic Kernel: build, orchestrate, and deploy agents and multi-agent workflows in Python and .NET, with graph-based workflows and checkpointing — the designated successor harness for both lines. [multi-agent, workflow, python]
- [hive](https://github.com/aden-hive/hive) — ⭐11.1k, complex, autonomy: bounded, recovery: resumable, unknown: Self-hosted multi-agent **harness** aimed at production workloads: human-in-the-loop checkpoints and a self-improving agent loop, distinct from single-session coding-agent shells. [multi-agent, python]
- [omnigent](https://github.com/omnigent-ai/omnigent) — ⭐10.1k, complex, autonomy: n/a, recovery: n/a, unknown: Open-source meta-**harness**: orchestrates Claude Code, Codex, Cursor, Pi, and custom agents behind one policy/sandboxing layer so teams swap harnesses without rewriting workflows. [sandbox, ide, python]
- [PraisonAI](https://github.com/MervinPraison/PraisonAI) — ⭐9.1k, mostly simple, autonomy: bounded, recovery: none, open-source: Autonomous multi-agent teams with a single entry point; emphasis on minimal config. [multi-agent, python]
- [AG2](https://github.com/ag2ai/ag2) — ⭐4.9k, complex, autonomy: n/a, recovery: n/a, unknown: AG2 (formerly AutoGen): the community-governed continuation of the original AutoGen project after Microsoft's fork diverged—conversable multi-agent groups, code execution, and human-in-the-loop under an open-source AgentOS banner. Graduated off the radar this cycle. [multi-agent, python]
- [AgentRL](https://github.com/THUDM/AgentRL) — ⭐354, complex, autonomy: headless, recovery: resumable, open-source: Multitask, multiturn RL for LLM agents; Ray-based scaling, rollout/actor workers—for teams that want to train agents, not just run them. [training, python]

## Plugins, MCPs, CLI tools (19 projects)

IDE plugins, concrete MCP servers, and CLI tools that give agents tools and context.

- [MCP Servers](https://github.com/modelcontextprotocol/servers) — ⭐90.5k, mostly simple, autonomy: n/a, recovery: n/a, open-source: The official reference collection of Model Context Protocol servers (filesystem, git, fetch, memory, time, and more)—the canonical, vetted toolset agents connect to, and the pattern every other MCP server is measured against. [mcp, memory, typescript]
- [Context7](https://github.com/upstash/context7) — ⭐62.2k, super simple, autonomy: n/a, recovery: n/a, open-source: MCP server that injects up-to-date, version-specific library docs into an agent's context on demand; kills the stale-training-data hallucinations that plague codegen. [mcp, training, typescript]
- [chrome-devtools-mcp](https://github.com/ChromeDevTools/chrome-devtools-mcp) — ⭐52.4k, mostly simple, autonomy: n/a, recovery: n/a, open-source: Google's official Chrome DevTools MCP server: exposes console, network, and performance-trace inspection as tool calls agents can drive directly, instead of a human clicking through the DevTools panel. [mcp, browser, typescript]
- [aider](https://github.com/Aider-AI/aider) — ⭐49.1k, slightly complex, autonomy: checkpoint-gated, recovery: resumable, open-source: Git-aware CLI pair programmer; edits in-repo, supports multiple models and MCP so agents see version control and tools. [mcp, cli, python]
- [Playwright MCP](https://github.com/microsoft/playwright-mcp) — ⭐37.4k, mostly simple, autonomy: n/a, recovery: n/a, open-source: Playwright's official MCP server: structured browser control (navigate, click, fill, extract) via the accessibility tree rather than screenshots, so web tasks stay fast and deterministic. [mcp, vision, browser, typescript]
- [continue](https://github.com/continuedev/continue) — ⭐36k, complex, autonomy: checkpoint-gated, recovery: resumable, open-source: Open-source IDE extension (VS Code, JetBrains); in-editor completion and chat with local or API models. [ide, typescript]
- [github-mcp-server](https://github.com/github/github-mcp-server) — ⭐33.1k, slightly complex, autonomy: n/a, recovery: n/a, open-source: GitHub's official MCP server (Go): repos, issues, PRs, code search, Actions. Replaces the older community `cyanheads/github-mcp-server` as the canonical way to give agents GitHub access. [mcp]
- [MCP Python SDK](https://github.com/modelcontextprotocol/python-sdk) — ⭐24.4k, mostly simple, autonomy: n/a, recovery: n/a, open-source: Official SDK to build and consume MCP servers/clients in Python; stdio and SSE transports. [mcp, python]
- [MCP TypeScript SDK](https://github.com/modelcontextprotocol/typescript-sdk) — ⭐13.4k, mostly simple, autonomy: n/a, recovery: n/a, open-source: Official MCP implementation for Node/TS; reference for the protocol. [mcp, typescript]
- [MCP Inspector](https://github.com/modelcontextprotocol/inspector) — ⭐10.9k, super simple, autonomy: n/a, recovery: n/a, open-source: GUI to test and debug MCP servers; inspect tools, resources, and prompts. [mcp, typescript]
- [MCP Registry](https://github.com/modelcontextprotocol/registry) — ⭐7.3k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Official, community-driven registry for MCP servers—the "app store" MCP clients use to discover servers. Maintained by Anthropic + ecosystem maintainers; v0.1 API frozen, production-grade. [mcp]
- [Agent Governance Toolkit](https://github.com/microsoft/agent-governance-toolkit) — ⭐6.3k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Microsoft's policy-enforcement layer for autonomous agents: zero-trust identity, execution sandboxing, and OWASP Agentic Top-10 coverage sit in front of the agent loop as a **harness** security layer—the governance counterpart to Infisical's agent-vault. [sandbox, python]
- [mcp-context-forge](https://github.com/IBM/mcp-context-forge) — ⭐4.5k, complex, autonomy: n/a, recovery: n/a, unknown: IBM's official AI gateway/registry/proxy that sits in front of any MCP, A2A, or REST/gRPC API: unified endpoint, centralized discovery, guardrails, and plugin support—the enterprise front door for **harness** tool calling. Graduated off the radar this cycle. [mcp, python]
- [cocoindex-code](https://github.com/cocoindex-io/cocoindex-code) — ⭐2.7k, mostly simple, autonomy: n/a, recovery: n/a, unknown: Embedded, tree-sitter/AST-based code-search CLI and MCP server that gives coding agents fast semantic lookups over a repo instead of grepping or re-reading whole files into context. [mcp, cli]
- [agent-vault](https://github.com/Infisical/agent-vault) — ⭐2.2k, mostly simple, autonomy: n/a, recovery: n/a, unknown: Infisical's HTTP credential proxy that fronts secrets for Claude Code, OpenClaw, and other agent harnesses so the agent's tool calls never see raw credentials—a **harness** security layer, not an agent loop itself.
- [Docker MCP Gateway](https://github.com/docker/mcp-gateway) — ⭐1.6k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Docker's official MCP CLI plugin / gateway; container-aware MCP tooling from Docker (replaces deprecated `docker/mcp-servers` path). [mcp, sandbox, cli]
- [puppeteer-real-browser-mcp](https://github.com/withLinda/puppeteer-real-browser-mcp-server) — ⭐26, mostly simple, autonomy: n/a, recovery: n/a, unknown: Puppeteer MCP with real-browser and anti-detection; for agents that need to drive sites that block headless. [mcp, browser, typescript]
- [Better-OpenCodeMCP](https://github.com/ajhcs/Better-OpenCodeMCP) — ⭐9, mostly simple, autonomy: n/a, recovery: n/a, open-source: MCP server for OpenCode/Crush: async task execution, model bridging (e.g. Claude→Gemini), process pooling. [mcp, typescript]
- [agentlog](https://github.com/RyanAlberts/agentlog) — ⭐1, super simple, autonomy: n/a, recovery: n/a, open-source: Persistent decision memory for any project: `remember`, `recall`, `reflect`. Single-file Python CLI that stores decisions as JSONL and uses Claude or Gemini to retrieve and synthesize patterns—Karpathy's LLM Wiki concept as a CLI. [memory, cli, python]

## Memory and state (5 projects)

Persistent memory layers that give agents recall across turns and sessions: knowledge graphs, vector stores, and session-capture tools that survive a restart. The state a harness needs but rarely ships with.

- [claude-mem](https://github.com/thedotmack/claude-mem) — ⭐94.3k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Session-memory plugin for Claude Code, Codex, OpenClaw, Gemini, Copilot, and more: captures everything an agent does during a session, AI-compresses it, and injects the relevant context into future sessions. Session-to-session memory as a drop-in. [memory]
- [Mem0](https://github.com/mem0ai/mem0) — ⭐65.7k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Universal memory layer for AI agents: stores user/org/session memory, retrieves on demand. Apache-2.0; the de-facto memory primitive paired with most harnesses in 2026. [memory, python]
- [Graphiti (Zep)](https://github.com/getzep/graphiti) — ⭐31k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Zep's open-source memory engine: real-time temporal knowledge graphs that track how facts about users and entities change over time, so agents can answer "what was true when." The layer behind Zep's hosted memory platform. [memory, rag, workflow, python]
- [cognee](https://github.com/topoteretes/cognee) — ⭐30.9k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Open-source memory layer for agents: an extract–cognify–load pipeline that turns your data into a queryable knowledge graph plus vector store, so agents recall facts and relationships across sessions instead of re-reading context. [memory, rag, workflow, python]
- [beads](https://github.com/gastownhall/beads) — ⭐27.3k, mostly simple, autonomy: n/a, recovery: n/a, unknown: Portable persistent-memory layer for coding agents: tracks decisions and task state outside the harness's own context window so it survives session resets and model swaps. [memory]

## Evaluation and benchmarking harnesses (19 projects)

Agentic eval systems, reasoning benchmarks, and open agent benchmarks.

- [Agent Lightning](https://github.com/microsoft/agent-lightning) — ⭐18.4k, complex, autonomy: headless, recovery: resumable, open-source: Microsoft's training-oriented harness: optimization loops for agent behavior—when you need to improve policies over rollouts, not only score a fixed prompt. [evals, training, python]
- [SWE-bench](https://github.com/SWE-bench/SWE-bench) — ⭐5.9k, slightly complex, autonomy: headless, recovery: resumable, open-source: LMs resolve real GitHub issues; Docker harness, instance IDs; standard for code-agent evals. [evals, sandbox, python]
- [AgentBench](https://github.com/THUDM/AgentBench) — ⭐3.7k, complex, autonomy: headless, recovery: none, open-source: ICLR'24 benchmark: agents across AlfWorld, DB, knowledge graphs, OS, webshop; Docker Compose, function-calling interface. [evals, sandbox, rag, workflow, python]
- [inspect_ai](https://github.com/UKGovernmentBEIS/inspect_ai) — ⭐2.8k, complex, autonomy: headless, recovery: resumable, open-source: Inspect AI core: composable eval tasks, sandboxes, scorers, and multi-model runs; the framework behind inspect_evals, not just the task bundle. [evals, sandbox, python]
- [WebArena](https://github.com/web-arena-x/webarena) — ⭐1.6k, complex, autonomy: headless, recovery: none, open-source: Realistic web env (e.g. e‑commerce, CMS, dev tools); 812 tasks; measures end-to-end web agent success. [python]
- [WebVoyager](https://github.com/MinorJerry/WebVoyager) — ⭐1.1k, slightly complex, autonomy: headless, recovery: none, open-source: End-to-end web agent with LMMs: screenshots + actions on real sites; benchmark on 15 sites, GPT-4V for automatic eval. [evals, vision]
- [agent-qa](https://github.com/vostride/agent-qa) — ⭐887, slightly complex, autonomy: headless, recovery: retry, restricted (FSL-1.1-ALv2): Self-improving QA **harness** for web and mobile apps: natural-language tests, memory-backed self-healing, dashboard/CLI, MCP and skills support, plus sandboxed hooks for production regression checks. [mcp, memory, sandbox, cli, typescript]
- [ClawBench](https://github.com/TIGER-AI-Lab/ClawBench) — ⭐801, complex, autonomy: headless, recovery: none, open-source: Open web-agent evaluation **harness**: runs selectable agents in isolated Docker containers across 153 live-site tasks (plus 130 in V2), intercepts irreversible requests, and records video, screenshots, HTTP traffic, actions, and agent messages for replayable scoring. [evals, vision, sandbox, python]
- [swe-smith](https://github.com/SWE-bench/SWE-smith) — ⭐775, slightly complex, autonomy: headless, recovery: none, open-source: Data generation for SWE agents; 50k+ instances across 128 repos; used for SWE-agent-LM training. [training, python]
- [SWE-Gym](https://github.com/SWE-Gym/SWE-Gym) — ⭐742, slightly complex, autonomy: headless, recovery: none, open-source: Training and evaluation for SWE agents and verifiers (ICML 2025). [evals, training, python]
- [ARC-AGI-2](https://github.com/arcprize/ARC-AGI-2) — ⭐738, super simple, autonomy: n/a, recovery: n/a, open-source: ARC Prize task set: grid-based abstraction/reasoning; public and private splits for generalization.
- [Terminal-Bench](https://github.com/harbor-framework/terminal-bench) — ⭐738, slightly complex, autonomy: headless, recovery: none, open-source: The terminal-task benchmark coding agents now cite next to SWE-bench: hard, containerized terminal tasks scored end to end. Terminal-Bench 2.0 runs on the harbor evaluation framework; the 1.0 tasks live on in the org's terminal-bench-1 repo. [evals, cli, python]
- [inspect_evals](https://github.com/UKGovernmentBEIS/inspect_evals) — ⭐675, slightly complex, autonomy: headless, recovery: resumable, open-source: UK AISI/Arcadia/Vector: GAIA and other evals in Inspect AI; level 1–3, sandboxed, tool-calling solvers. [evals, sandbox]
- [arc-agi-benchmarking](https://github.com/arcprize/arc-agi-benchmarking) — ⭐362, mostly simple, autonomy: headless, recovery: retry, open-source: Runner for ARC-AGI: multi-provider (OpenAI, Anthropic, Gemini, etc.), rate limits, retries, and scoring. [evals, provider-agnostic, python]
- [VitaBench](https://github.com/meituan-longcat/vitabench) — ⭐177, complex, autonomy: headless, recovery: none, open-source: ICLR'26: 66 tools, real-world apps (delivery, travel, retail); 100 cross-scenario + 300 single-scenario tasks; adopted by Qwen/Seed.
- [AgencyBench](https://github.com/GAIR-NLP/AgencyBench) — ⭐100, complex, autonomy: headless, recovery: none, open-source: Long-horizon agent benchmark: 32 scenarios, 138 tasks, ~1M tokens and ~90 tool calls; Docker sandbox and rubric-based + LLM judges. [evals, sandbox, python]
- [letta-evals](https://github.com/letta-ai/letta-evals) — ⭐82, mostly simple, autonomy: headless, recovery: none, open-source: Eval harness for stateful Letta agents; configurable suites and grading (LLM or rule-based) so you can measure what you ship. [memory, python]
- [SUPER](https://github.com/allenai/super-benchmark) — ⭐58, slightly complex, autonomy: headless, recovery: none, open-source: Agents that set up and run ML/NLP from GitHub repos; 45 expert problems, 152 masked tasks, 602 AutoGen tasks; Docker-based. [sandbox, python]
- [TRAIL](https://github.com/patronus-ai/trail-benchmark) — ⭐24, mostly simple, autonomy: n/a, recovery: n/a, open-source: Trace reasoning and agentic issue localization; 148 long-context traces, 841 errors, 20+ error types; Hugging Face dataset.

## Observability and eval-ops (4 projects)

Tracing, monitoring, and production evaluation for live agent runs: capture every step, tool call, and token, then score and debug in the loop. Distinct from the fixed-task benchmarks above—this is what you run against your own traffic.

- [Langfuse](https://github.com/langfuse/langfuse) — ⭐34.9k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Open-source LLM engineering platform: full-trace observability, online and offline evals, prompt management, and cost metrics for agent runs in production—the monitoring layer most harnesses lack out of the box. [evals, typescript]
- [MLflow](https://github.com/mlflow/mlflow) — ⭐28.1k, complex, autonomy: n/a, recovery: n/a, open-source: Mature ML platform now covering GenAI: MLflow Tracing captures every agent step, tool call, and token, with built-in LLM evals and prompt versioning—observability for teams already standardized on MLflow. [evals, python]
- [Opik](https://github.com/comet-ml/opik) — ⭐22.2k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Comet's open-source agent observability and evaluation platform: tracing, scoring, and experiment comparison with the whole core feature set free to self-host under Apache-2.0. [evals, python]
- [Arize Phoenix](https://github.com/Arize-ai/phoenix) — ⭐11.5k, slightly complex, autonomy: n/a, recovery: n/a, restricted (Elastic-2.0): Arize's source-available, local-first tracing and eval layer: run it on your laptop or your own infra, and graduate to the managed Arize AX platform only when you need it. [evals, python]

## Research and task-specific harnesses (5 projects)

Deep research, document QA, and domain-specific agent loops.

- [DeerFlow](https://github.com/bytedance/deer-flow) — ⭐82.8k, complex, autonomy: headless, recovery: resumable, open-source: ByteDance's long-horizon research **harness**: a LangGraph-based agent loop that researches, codes, and creates using sandboxes, memory, tools, and subagents behind a message gateway—an alternative to the reference gpt-researcher stack for multi-hour tasks. [memory, multi-agent, sandbox, python]
- [gpt-researcher](https://github.com/assafelovic/gpt-researcher) — ⭐29.5k, complex, autonomy: bounded, recovery: retry, open-source: Autonomous deep-research agent: web + local sources, citation-grounded reports, multi-agent and deep-research modes. The reference open-source research harness. [multi-agent, python]
- [AutoResearchClaw](https://github.com/aiming-lab/AutoResearchClaw) — ⭐14.5k, complex, autonomy: n/a, recovery: n/a, unknown: Fully autonomous, self-evolving research **harness**: multi-agent debate and citation verification carry a project from idea to a written paper without a human in the loop. [multi-agent]
- [MiroThinker](https://github.com/MiroMindAI/MiroThinker) — ⭐8.4k, slightly complex, autonomy: headless, recovery: retry, unknown: Deep-research **harness** tuned for long browsing-and-reasoning chains; benchmarked on BrowseComp, GAIA, and HLE by pairing a dedicated agent loop with its own MiroThinker models rather than bolting search onto a generic chat agent. [evals]
- [openagents](https://github.com/OpenAgentsInc/openagents) — ⭐450, complex, autonomy: headless, recovery: resumable, open-source: Platform for autonomous agents and autopilot-style workflows; decentralized/Nostr-oriented (Pylon runtime, actively shipped in 2026).

## Libraries and SDKs (15 projects)

Lightweight runtimes, tool loops, and provider-agnostic harness primitives.

- [Daytona](https://github.com/daytonaio/daytona) — ⭐71.7k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Elastic dev environments for AI-generated code: workspaces, Git, previews—infra harness between "the model wrote a patch" and "it ran in a real machine." ⚠️ Public repo unmaintained since June 2026: core development moved to a private codebase (final open release v0.190.0, AGPL-3.0). [sandbox]
- [LiteLLM](https://github.com/BerriAI/litellm) — ⭐59.2k, mostly simple, autonomy: n/a, recovery: retry, open-source: One interface to 100+ LLMs; routing, caching, budgets. Not an agent framework—the pipe every agent framework uses. [provider-agnostic, python]
- [Composio](https://github.com/ComposioHQ/composio) — ⭐30.3k, complex, autonomy: n/a, recovery: n/a, open-source: 1,000+ toolkits with auth, tool search, and a sandboxed workbench—drop-in tool layer so agents stop reinventing OAuth + integrations. Python and TypeScript. [sandbox, tool-discovery, python, typescript]
- [deepagents](https://github.com/langchain-ai/deepagents) — ⭐29.6k, slightly complex, autonomy: bounded, recovery: durable, open-source: LangChain's Python+TypeScript agent harness on top of LangGraph: planning tool, virtual filesystem, shell sandbox, sub-agent spawning—the "Claude Code-style" harness as a reusable library. [multi-agent, sandbox, python, typescript]
- [smolagents](https://github.com/huggingface/smolagents) — ⭐29.4k, mostly simple, autonomy: bounded, recovery: none, open-source: Code-as-action agents: model outputs Python executed in sandbox (E2B, Modal, etc.); ~1k LOC core. [sandbox, python]
- [vercel/ai](https://github.com/vercel/ai) — ⭐26.9k, slightly complex, autonomy: bounded, recovery: retry, open-source: React and Node SDK for streaming, tool calls, and agent-style UIs; provider-agnostic. [provider-agnostic, typescript]
- [pydantic-ai](https://github.com/pydantic/pydantic-ai) — ⭐20.1k, slightly complex, autonomy: bounded, recovery: durable, open-source: Type-safe Python agents with Pydantic I/O; multi-provider, MCP, Logfire observability, and human-in-the-loop. [mcp, typed, provider-agnostic, python]
- [E2B](https://github.com/e2b-dev/E2B) — ⭐13.9k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Firecracker sandboxes for executing agent-generated code; the hosted isolation layer many tool-calling demos use instead of running arbitrary LLM output on your laptop. [sandbox, python]
- [Steel](https://github.com/steel-dev/steel-browser) — ⭐7.7k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Open-source browser API for agents: cloud or self-hosted Chrome sessions with stealth, residential proxies, CAPTCHA solving, and persistent profiles. The only open-source core in the hosted browser-infrastructure lane (Browserbase and Hyperbrowser are closed). [memory, browser, local]
- [strands-agents](https://github.com/strands-agents/harness-sdk) — ⭐7.4k, mostly simple, autonomy: bounded, recovery: resumable, open-source: Model-driven Python SDK; decorators for tools, native MCP, multi-agent; "minimal code" without sacrificing provider choice. [mcp, multi-agent, typed, python]
- [Cloudflare Agents](https://github.com/cloudflare/agents) — ⭐5.6k, slightly complex, autonomy: headless, recovery: durable, open-source: Persistent, stateful agents on Durable Objects: state, websockets, scheduling, and AI chat baked in. The serverless answer to "where does the agent live?" [memory, typescript]
- [Agent Sandbox](https://github.com/kubernetes-sigs/agent-sandbox) — ⭐4k, slightly complex, autonomy: n/a, recovery: n/a, open-source: Kubernetes-native sandbox primitive for agent runtimes: a Sandbox resource plus warm pools and claims for fast-start, isolated, stateful workloads. The self-hosted answer to hosted sandbox APIs, from the Kubernetes SIGs org. [memory, sandbox, local]
- [openai-agents-js](https://github.com/openai/openai-agents-js) — ⭐3.8k, slightly complex, autonomy: bounded, recovery: resumable, open-source: Official OpenAI Agents SDK for Node/TS: handoffs, guardrails, voice; the JS counterpart to openai-agents-python. [multi-agent, voice, typescript]
- [open-harness](https://github.com/MaxGfeller/open-harness) — ⭐612, slightly complex, autonomy: bounded, recovery: none, open-source: TypeScript Agent class on Vercel AI SDK; streaming events, filesystem/bash tools, MCP, and subagent delegation. [mcp, multi-agent, typescript]
- [Community-curated agent lists](https://github.com/brandonhimpfen/awesome-ai-agents) — ⭐15, super simple, autonomy: n/a, recovery: n/a, unknown: Broader directories: e.g. [brandonhimpfen/awesome-ai-agents](https://github.com/brandonhimpfen/awesome-ai-agents), [axioma-ai-labs/awesome-ai-agent-frameworks](https://github.com/axioma-ai-labs/awesome-ai-agent-frameworks), [mb-mal/awesome-ai-agents-frameworks](https://github.com/mb-mal/awesome-ai-agents-frameworks)—differ by scope and update cadence.


---

# Decision guides, full text

# Agent evals: SWE-bench vs inspect_ai vs Terminal-Bench

You changed your agent: new model, new prompt, new tools. Did it get better or worse? An eval is how you answer that with a number instead of a feeling. The word covers two different products, and knowing which one you need is most of the decision. A **benchmark** is a fixed public exam with a leaderboard: it tells you how a model or agent ranks against the field on tasks someone else wrote. An **eval framework** is a test runner for exams you write yourself: it tells you whether your agent works on your tasks. SWE-bench and Terminal-Bench are benchmarks; inspect_ai is a framework.

Why it matters: shipping on a public number alone has burned people. OpenAI [stopped reporting SWE-bench Verified results](https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/) after an audit found flawed test cases in a majority of the problems it sampled, and the loudest threads in this space are about [benchmark exploits](https://news.ycombinator.com/item?id=47733217), not benchmark scores.

| | [SWE-bench](https://github.com/SWE-bench/SWE-bench) | [inspect_ai](https://github.com/UKGovernmentBEIS/inspect_ai) | [Terminal-Bench](https://github.com/harbor-framework/terminal-bench) |
|---|---|---|---|
| ⭐ Stars | 5.9k | 2.8k | 738 |
| Shape | Benchmark: real GitHub issues, Docker harness | Framework: write tasks, attach scorers, run sandboxed across models | Benchmark: hard terminal tasks in containers |
| The question it answers | Can this agent fix real reported bugs in real repos? | Whatever question you write a task for | Can this agent do real work in a command-line shell? |
| Steward | The SWE-bench org (Princeton and Stanford researchers) | UK AI Security Institute, the UK government body that tests frontier models | The harbor-framework org |
| Maintenance (checked 2026-08-12) | Active | Active | Active |
| License | MIT | MIT | Apache-2.0 |

_Stars as captured for the main list. Star counts mislead here: these are small repos and field standards at the same time, and Terminal-Bench's count understates it (the same org's 1.0 task repo and its harbor runner hold several thousand more). Rating definitions for this site live in the [guide to rankings](../README.md#guide-to-rankings)._

## Pick by situation

- **You build a coding agent and want a number the field respects** → **SWE-bench**. Real GitHub issues, resolved or not, in a Docker harness. Know the family: [SWE-bench Verified](https://www.swebench.com/verified.html) is the 500-problem human-checked subset every launch cites, and [SWE-bench Pro](https://scale.com/blog/swe-bench-pro) is Scale's harder successor, built partly because the original's tasks have been public long enough to leak into training data.
- **You ship an agent product and need to know it works before users do** → **inspect_ai**. You define tasks, attach scorers (a scorer marks each attempt: either a scripted check or a second model grading the output, called model-graded), and run them in sandboxes across models. It is the framework behind the UK AISI's own evaluations, and [ready-made suites exist](https://github.com/UKGovernmentBEIS/inspect_evals) so you don't start from zero.
- **Your agent lives in a terminal** → **Terminal-Bench**. Containerized command-line tasks scored end to end, with a [public leaderboard](https://www.tbench.ai/leaderboard). This is the benchmark people now weigh [directly against SWE-bench](https://www.digitalapplied.com/blog/swe-bench-terminal-bench-benchmark-guide-2026) when the work is broader than fixing GitHub issues.

## A score is the model plus the harness

The same model produces different scores depending on the agent software (the harness) that drives it: one tracking site shows [Claude Opus 4.5 scoring 73.2 to 77.6 on SWE-bench Verified](https://tensorfeed.ai/harnesses/openhands) depending on whether SWE-agent, mini-SWE-agent, or OpenHands runs it. So read every leaderboard number as a stack number, not a model number, and when you compare your own runs, hold the harness constant. [SWE-agent](https://github.com/SWE-agent/SWE-agent) exists exactly for this: the reference harness published next to the benchmark.

## When the number lies

Three failure modes to check before trusting any benchmark score. **Aging**: public tasks end up in training data, so old exams flatter new models. **Broken tests**: OpenAI's audit found the majority of sampled SWE-bench Verified problems could be "solved" for the wrong reasons. **Gaming**: a [588-point Hacker News study](https://news.ycombinator.com/item?id=47733217) showed the major agent benchmarks can be exploited outright. This is also where AgentBench belongs now: the [ICLR 2024](https://arxiv.org/abs/2308.03688) benchmark (ICLR is a major machine-learning conference) that first scored LLMs as agents across eight environments, from OS shells to web shops. Its [leaderboard has run cold since 2025](https://benchmarkingagents.com/agentbench/), so treat it as the field's history, not a live target. For gaps these three don't cover, [AgencyBench](https://github.com/GAIR-NLP/AgencyBench) tests long tasks (about a million tokens and ninety tool calls per scenario) and [WebArena](https://github.com/web-arena-x/webarena) tests end-to-end web work.

## The adjacent purchase

Everything on this page is open source. The place money changes hands in evals is one layer up: hosted platforms that trace what your agent did in production and score it continuously. That decision, the most-compared purchase in the whole space, gets its own guide: [Eval and observability platforms](eval-platforms.md) (Langfuse vs LangSmith vs Braintrust vs Phoenix).

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# The best AI agent harnesses in 2026, ranked by category

These are the top three agent harnesses in each of the 12 categories of best-of-Agent-Harnesses, a hand-curated list of 167 harnesses re-ranked from live GitHub data every week; the stars below were captured 2026-09-20, and the page regenerates with every refresh.

<!-- Generated by scripts/generate.py from the list data. Do not edit by hand; edit the Project rows in the generator. -->

## How this page is ranked

Projects enter the list by editorial review against one rubric: does it own the agent loop, the tool wiring, the approval model, or the memory, and is it maintained. Within a category the order is GitHub stars, captured weekly, and each project carries its adoption-surface tier and its autonomy and recovery scores, so a high star count never hides a harness that cannot run unattended or survive a crash. Archived projects move to the graveyard and never appear here. The [six-question guide](how-to-pick-a-harness.md) turns these tables into a decision, and [why the harness matters more than the model](why-the-harness-matters.md) explains why the choice deserves the care.

## Progressive disclosure harnesses

_Formats, runtimes, and patterns that reveal context, tools, or instructions in layers—index first, details on demand—to control tokens and improve agent focus (the "map, not encyclopedia" principle)._ [Full category →](../README.md#progressive-disclosure-harnesses)

1. **[Headroom](https://github.com/headroomlabs-ai/headroom)** (73.2k stars, mostly simple, not a loop-owning harness (a format, config, or library)): Compresses tool outputs, logs, files, and RAG chunks with content-aware compressors before they reach the model—claimed 20% fewer tokens for coding agents and 60–95% fewer for JSON, same answers. Ships as a library, HTTP proxy, or MCP server, so it drops in front of whatever harness you already run.
2. **[awesome-cursorrules](https://github.com/PatrickJS/awesome-cursorrules)** (40.8k stars, super simple, not a loop-owning harness (a format, config, or library)): Curated .cursorrules and skills that leverage Cursor's index-then-load model; the canonical collection for rules-as-progressive-disclosure in the IDE.
3. **[agents.md](https://github.com/agentsmd/agents.md)** (24.5k stars, super simple, not a loop-owning harness (a format, config, or library)): Open format for repo-scoped agent briefings; nested AGENTS.md files scope instructions per directory, so agents get a map of what exists and load only what's relevant. Read by 20+ tools including Codex, Cursor, and Copilot.

## Coding agent products (IDEs, CLIs, full suites)

_Turnkey coding agents you install and run: IDE extensions, terminal CLIs, Dockerized workspaces. Each entry notes which part is the harness (the agent loop, tool wiring, approval model) versus the UI shell (VS Code extension, TUI, browser client)._ [Full category →](../README.md#coding-agent-products-ides-clis-full-suites)

1. **[opencode](https://github.com/anomalyco/opencode)** (209k stars, slightly complex, autonomy headless, recovery resumable): Open-source terminal coding agent (formerly `sst/opencode`; transferred to anomalyco). The **harness** is a multi-provider tool-call loop (Claude, OpenAI, Gemini, local) with strong plugin and MCP support; the TUI is the shell. 100% OSS, very actively shipped.
2. **[Codex](https://github.com/openai/codex)** (125k stars, slightly complex, autonomy bounded, recovery resumable): OpenAI's terminal coding agent. The **harness** is the sandboxed tool-call loop with multi-provider support; the CLI is the shell. Reference implementation for "official CLI that ships code."
3. **[pi](https://github.com/earendil-works/pi)** (108k stars, slightly complex, autonomy bounded, recovery resumable): The upstream AI agent toolkit behind this list's oh-my-pi fork: a unified multi-provider LLM API, agent loop, and TUI shell providing the **harness** that oh-my-pi's Rust rewrite builds on.

## Coding harness configs and SDKs

_Skill packs, slash-command libraries, meta-prompting frameworks, and official SDKs that give you the harness (the agent loop, planning, memory, hooks) without bundling a specific IDE or CLI shell._ [Full category →](../README.md#coding-harness-configs-and-sdks)

1. **[superpowers](https://github.com/obra/superpowers)** (289k stars, complex, not a loop-owning harness (a format, config, or library)): Performance-oriented harness pack for Claude Code and 13 other harnesses (Codex, Cursor, OpenCode, Gemini CLI, more): skills, instincts, memory, security, research-first workflows. Treats harness engineering itself as the performance lever.
2. **[Anthropic Skills](https://github.com/anthropics/skills)** (177k stars, mostly simple, not a loop-owning harness (a format, config, or library)): Anthropic's official Agent Skills repository: SKILL.md-based folders (instructions, scripts, resources) Claude dynamically loads on Claude Code, Claude.ai, and the API. The reference for progressive-disclosure skill packs in 2026.
3. **[GStack](https://github.com/garrytan/gstack)** (134k stars, slightly complex, not a loop-owning harness (a format, config, or library)): Garry Tan's Claude Code skill stack: 23 slash-command modes (CEO/eng/design review, QA, ship, browse, retro, …) that structure one assistant as a virtual engineering team. Daily driver while running YC.

## Personal agent runtimes

_Always-on, self-hosted agents you run as a daemon and talk to from chat apps: gateway runtimes, second brains, and self-improving assistants. The agent as a product you operate, not a library you build with._ [Full category →](../README.md#personal-agent-runtimes)

1. **[OpenClaw](https://github.com/openclaw/openclaw)** (390k stars, complex, autonomy headless, recovery resumable): Self-hosted, always-on personal agent (formerly Clawdbot/Moltbot): a gateway + event-loop runtime that treats messages, heartbeats, crons, and webhooks as one input queue, persists state to local files, and lives in your chat apps (WhatsApp, Telegram, Slack, Discord). 13,700+ community skills; the fastest-growing repo in GitHub history.
2. **[Hermes](https://github.com/NousResearch/hermes-agent)** (247k stars, slightly complex, autonomy headless, recovery resumable): Nous Research's self-improving agent: a learning loop turns experience into reusable skills, builds a persistent user model across sessions, and checkpoints state to disk with rollback; lean enough for a $5 VPS, driven from chat, and model-agnostic (Nous Portal, OpenRouter, OpenAI, or any endpoint).
3. **[AnythingLLM](https://github.com/Mintplex-Labs/anything-llm)** (66.3k stars, complex, autonomy headless, recovery resumable): Self-hosted "AI second brain" **harness**: chat with your documents, run built-in agent skills (web search, code execution, browsing), and manage multi-user workspaces with a bundled vector DB, so there is no separate RAG stack to wire up.

## Frameworks

_General-purpose agent and LLM application frameworks (the app layer, not harnesses per se)._ [Full category →](../README.md#frameworks)

1. **[n8n](https://github.com/n8n-io/n8n)** (205k stars, complex, autonomy headless, recovery durable): Fair-code workflow engine with 400+ nodes and native AI nodes; the self-hosted Zapier that actually does agents and LangChain.
2. **[AutoGPT](https://github.com/Significant-Gravitas/AutoGPT)** (187k stars, complex, autonomy headless, recovery resumable): The original autonomous loop: goal in, agent iterates with tools and memory; Forge is the dev framework, Benchmark the eval harness.
3. **[Dify](https://github.com/langgenius/dify)** (157k stars, complex, autonomy headless, recovery retry): One-stop LLM app platform: visual workflows, RAG pipeline, 50+ tools, model management; "ship from prototype to prod" in a single UI.

## Multi-agent and orchestration

_Harnesses and patterns for multi-agent coordination and handoffs._ [Full category →](../README.md#multi-agent-and-orchestration)

1. **[MetaGPT](https://github.com/FoundationAgents/MetaGPT)** (70.5k stars, complex, autonomy headless, recovery resumable): The "AI software company" multi-agent framework: role-played PM, architect, and engineer agents turn a one-line requirement into specs, designs, and code along an SOP assembly line. The landmark of the genre; development pace has slowed in 2026.
2. **[autogen](https://github.com/microsoft/autogen)** (61.1k stars, complex, autonomy bounded, recovery resumable): Conversable agents and group chats; code execution and human-in-the-loop; Microsoft origin, AG2 ecosystem. ⚠️ In maintenance mode since late 2025: no new features, and Microsoft directs new users to Agent Framework.
3. **[crewAI](https://github.com/crewAIInc/crewAI)** (58.8k stars, complex, autonomy bounded, recovery resumable): Role-based agents (roles, goals, backstories) in Crews; Flows add event-driven and hierarchical control for production.

## Plugins, MCPs, CLI tools

_IDE plugins, concrete MCP servers, and CLI tools that give agents tools and context._ [Full category →](../README.md#plugins-mcps-cli-tools)

1. **[MCP Servers](https://github.com/modelcontextprotocol/servers)** (90.5k stars, mostly simple, not a loop-owning harness (a format, config, or library)): The official reference collection of Model Context Protocol servers (filesystem, git, fetch, memory, time, and more)—the canonical, vetted toolset agents connect to, and the pattern every other MCP server is measured against.
2. **[Context7](https://github.com/upstash/context7)** (62.2k stars, super simple, not a loop-owning harness (a format, config, or library)): MCP server that injects up-to-date, version-specific library docs into an agent's context on demand; kills the stale-training-data hallucinations that plague codegen.
3. **[chrome-devtools-mcp](https://github.com/ChromeDevTools/chrome-devtools-mcp)** (52.4k stars, mostly simple, not a loop-owning harness (a format, config, or library)): Google's official Chrome DevTools MCP server: exposes console, network, and performance-trace inspection as tool calls agents can drive directly, instead of a human clicking through the DevTools panel.

## Memory and state

_Persistent memory layers that give agents recall across turns and sessions: knowledge graphs, vector stores, and session-capture tools that survive a restart. The state a harness needs but rarely ships with._ [Full category →](../README.md#memory-and-state)

1. **[claude-mem](https://github.com/thedotmack/claude-mem)** (94.3k stars, slightly complex, not a loop-owning harness (a format, config, or library)): Session-memory plugin for Claude Code, Codex, OpenClaw, Gemini, Copilot, and more: captures everything an agent does during a session, AI-compresses it, and injects the relevant context into future sessions. Session-to-session memory as a drop-in.
2. **[Mem0](https://github.com/mem0ai/mem0)** (65.7k stars, slightly complex, not a loop-owning harness (a format, config, or library)): Universal memory layer for AI agents: stores user/org/session memory, retrieves on demand. Apache-2.0; the de-facto memory primitive paired with most harnesses in 2026.
3. **[Graphiti (Zep)](https://github.com/getzep/graphiti)** (31k stars, slightly complex, not a loop-owning harness (a format, config, or library)): Zep's open-source memory engine: real-time temporal knowledge graphs that track how facts about users and entities change over time, so agents can answer "what was true when." The layer behind Zep's hosted memory platform.

## Evaluation and benchmarking harnesses

_Agentic eval systems, reasoning benchmarks, and open agent benchmarks._ [Full category →](../README.md#evaluation-and-benchmarking-harnesses)

1. **[Agent Lightning](https://github.com/microsoft/agent-lightning)** (18.4k stars, complex, autonomy headless, recovery resumable): Microsoft's training-oriented harness: optimization loops for agent behavior—when you need to improve policies over rollouts, not only score a fixed prompt.
2. **[SWE-bench](https://github.com/SWE-bench/SWE-bench)** (5.9k stars, slightly complex, autonomy headless, recovery resumable): LMs resolve real GitHub issues; Docker harness, instance IDs; standard for code-agent evals.
3. **[AgentBench](https://github.com/THUDM/AgentBench)** (3.7k stars, complex, autonomy headless, recovery none): ICLR'24 benchmark: agents across AlfWorld, DB, knowledge graphs, OS, webshop; Docker Compose, function-calling interface.

## Observability and eval-ops

_Tracing, monitoring, and production evaluation for live agent runs: capture every step, tool call, and token, then score and debug in the loop. Distinct from the fixed-task benchmarks above—this is what you run against your own traffic._ [Full category →](../README.md#observability-and-eval-ops)

1. **[Langfuse](https://github.com/langfuse/langfuse)** (34.9k stars, slightly complex, not a loop-owning harness (a format, config, or library)): Open-source LLM engineering platform: full-trace observability, online and offline evals, prompt management, and cost metrics for agent runs in production—the monitoring layer most harnesses lack out of the box.
2. **[MLflow](https://github.com/mlflow/mlflow)** (28.1k stars, complex, not a loop-owning harness (a format, config, or library)): Mature ML platform now covering GenAI: MLflow Tracing captures every agent step, tool call, and token, with built-in LLM evals and prompt versioning—observability for teams already standardized on MLflow.
3. **[Opik](https://github.com/comet-ml/opik)** (22.2k stars, slightly complex, not a loop-owning harness (a format, config, or library)): Comet's open-source agent observability and evaluation platform: tracing, scoring, and experiment comparison with the whole core feature set free to self-host under Apache-2.0.

## Research and task-specific harnesses

_Deep research, document QA, and domain-specific agent loops._ [Full category →](../README.md#research-and-task-specific-harnesses)

1. **[DeerFlow](https://github.com/bytedance/deer-flow)** (82.8k stars, complex, autonomy headless, recovery resumable): ByteDance's long-horizon research **harness**: a LangGraph-based agent loop that researches, codes, and creates using sandboxes, memory, tools, and subagents behind a message gateway—an alternative to the reference gpt-researcher stack for multi-hour tasks.
2. **[gpt-researcher](https://github.com/assafelovic/gpt-researcher)** (29.5k stars, complex, autonomy bounded, recovery retry): Autonomous deep-research agent: web + local sources, citation-grounded reports, multi-agent and deep-research modes. The reference open-source research harness.
3. **[AutoResearchClaw](https://github.com/aiming-lab/AutoResearchClaw)** (14.5k stars, complex, not a loop-owning harness (a format, config, or library)): Fully autonomous, self-evolving research **harness**: multi-agent debate and citation verification carry a project from idea to a written paper without a human in the loop.

## Libraries and SDKs

_Lightweight runtimes, tool loops, and provider-agnostic harness primitives._ [Full category →](../README.md#libraries-and-sdks)

1. **[Daytona](https://github.com/daytonaio/daytona)** (71.7k stars, slightly complex, not a loop-owning harness (a format, config, or library)): Elastic dev environments for AI-generated code: workspaces, Git, previews—infra harness between "the model wrote a patch" and "it ran in a real machine." ⚠️ Public repo unmaintained since June 2026: core development moved to a private codebase (final open release v0.190.0, AGPL-3.0).
2. **[LiteLLM](https://github.com/BerriAI/litellm)** (59.2k stars, mostly simple, not a loop-owning harness (a format, config, or library)): One interface to 100+ LLMs; routing, caching, budgets. Not an agent framework—the pipe every agent framework uses.
3. **[Composio](https://github.com/ComposioHQ/composio)** (30.3k stars, complex, not a loop-owning harness (a format, config, or library)): 1,000+ toolkits with auth, tool search, and a sandboxed workbench—drop-in tool layer so agents stop reinventing OAuth + integrations. Python and TypeScript.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). Regenerated from the list data on every weekly refresh._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# Browser agents: browser-use vs Stagehand vs Playwright MCP vs chrome-devtools-mcp

"Browser agent" covers three different kinds of product, and most bad picks here come from comparing across the lanes instead of within one. **Agent libraries** own the whole job: you hand them a goal in plain language ("find the cheapest flight, fill the form") and they decide every click. browser-use, Stagehand, and Skyvern live here. **Tool servers** give browser abilities to an agent you already run, over MCP (Model Context Protocol, the standard way to plug tools into AI agents); the agent you already pay for stays in charge. Playwright MCP and chrome-devtools-mcp live here, and they split the work between them: one acts on pages, the other inspects what a page did. **Browser infrastructure** is the third lane: hosted browsers (Browserbase, Steel, Hyperbrowser) that the first two lanes can run on when they need scale or stealth.

Why it matters: browser automation fails in production for unglamorous reasons: a login wall, a bot check, a page that changed shape. Picking the wrong lane means rewriting the whole integration the first time that happens.

| | [browser-use](https://github.com/browser-use/browser-use) | [Stagehand](https://github.com/browserbase/stagehand) | [Playwright MCP](https://github.com/microsoft/playwright-mcp) | [chrome-devtools-mcp](https://github.com/ChromeDevTools/chrome-devtools-mcp) |
|---|---|---|---|---|
| ⭐ Stars | 116k | 24.6k | 37.4k | 52.4k |
| Lane | Agent library | Agent library / SDK | Tool server (acts) | Tool server (inspects) |
| You write | A goal in plain language | Plain-language actions mixed with Playwright code | Nothing: your agent calls it | Nothing: your agent calls it |
| How it reads pages | Chrome DevTools Protocol, the browser's own remote-control wire (it [dropped Playwright in August 2025](https://browser-use.com/changelog/19-8-2025)) | Playwright, with plain-language act/extract/observe on top | The accessibility tree: the structured outline browsers build for screen readers, so actions are text lookups, not pixel guessing | Chrome DevTools surfaces: console, network, performance traces |
| Guardrails | A domain allowlist with a [documented bypass](https://github.com/advisories/GHSA-x39x-9qw5-ghrf) (patched, severity critical); no sandbox of its own | Inherits Playwright's controls; Browserbase hosts it for isolation | Your harness's permission prompts | Your harness's permission prompts |
| Steward | Browser Use (company) | Browserbase (company) | Microsoft (Playwright team) | Google (Chrome DevTools team) |
| License | MIT | MIT | Apache-2.0 | Apache-2.0 |

_Stars as captured for the main list; rating definitions live in the [guide to rankings](../README.md#guide-to-rankings). On this list's axes the two libraries own a bounded agent loop; the two tool servers own no loop at all, because the agent calling them stays in charge._

## Pick by situation

- **The browser task is the product** ("compare these prices", "file this form on 40 sites") → **browser-use**. The biggest community in the category and the shortest path from a sentence to browser actions. Treat it as untrusted-by-default: it ships no sandbox of its own, so run it contained (see [Agent sandboxing](sandboxed-code-execution.md)) and keep logged-in sessions away from pages you don't control.
- **You want the agent to draft the automation, then pin it down** → **Stagehand**. Its bet is mixing plain-language steps with regular Playwright code in one script, so what starts flexible can end deterministic and repeatable. The browser-use vs Stagehand choice is [the most-written-about matchup in the lane](https://scrapfly.io/blog/posts/stagehand-vs-browser-use).
- **Your coding agent needs browser hands** (end-to-end tests, scraping, form-driving from inside Claude Code, opencode, or Cursor) → **Playwright MCP**. First-party from the Playwright team; acting on the accessibility tree keeps actions structured and fast. The default answer inside a harness you already run.
- **Your agent needs to see why the page broke** (console errors, failed requests, slow traces) → **chrome-devtools-mcp**. First-party from the Chrome team; it turns the DevTools panel into tool calls. It complements Playwright MCP rather than replacing it, and running both (one to act, one to inspect) is a natural pairing.
- **The site fights automation** → [puppeteer-real-browser-mcp-server](https://github.com/withLinda/puppeteer-real-browser-mcp-server) adds real-browser and anti-detection modes (its own README says no automation is invisible), and the infrastructure lane ([Browserbase vs Steel vs Hyperbrowser](browser-infrastructure.md)) sells hosted stealth browsers at scale. For vision-first form filling, [Skyvern](https://github.com/Skyvern-AI/skyvern) is the common third name next to browser-use and Stagehand.

## Agent or tool?

If you already run a coding harness, you rarely need a second agent with its own loop, its own model bill, and its own failure modes; you need hands, which is what the MCP servers are, gated by permission prompts you already configured. Reach for an agent library when the browser work stands alone: a scheduled job, a product feature, a task no coding harness is part of. One caution applies to every lane: a browser agent reads untrusted pages by definition, and a page can carry instructions aimed at the agent, so treat page content as data, never as commands, and keep credentials out of reach. To grade whichever you pick, [WebArena](https://github.com/web-arena-x/webarena) and [WebVoyager](https://github.com/MinorJerry/WebVoyager) are public test suites that score web agents on realistic sites; see [Agent evals](agent-eval-harnesses.md).

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# Browser infrastructure for agents: Browserbase vs Steel vs Hyperbrowser

Agent libraries like browser-use and Stagehand decide what to click; something still has to run the browsers they click in. At small scale that's Chrome on your own machine. At production scale it becomes its own operations problem: hundreds of concurrent sessions, sites that block automation, CAPTCHAs, logins that must persist between runs, and recordings you'll need when a run goes wrong. Browser infrastructure vendors sell exactly that: hosted browsers with the ugly parts handled. These three are the names buyers actually weigh ([our browser-agents guide](browser-agents.md) covers the other two lanes: agent libraries and MCP tool servers).

| | [Browserbase](https://www.browserbase.com/) | [Steel](https://steel.dev/) | [Hyperbrowser](https://hyperbrowser.ai/) |
|---|---|---|---|
| What it sells | Managed cloud browsers with automation, search, and extraction APIs | An open-source browser API you run hosted or self-hosted | Cloud browsers with agent-focused scraping and extraction APIs |
| Open source? | No: closed infra; its open source is the [Stagehand SDK](https://github.com/browserbase/stagehand) and an [MCP server](https://github.com/browserbase/mcp-server-browserbase) | Yes: [steel-dev/steel-browser](https://github.com/steel-dev/steel-browser) (7.5k stars, Apache-2.0) is the actual product core, Docker-deployable | No: closed infra; peripheral MIT tooling only |
| Steward | Browserbase ($40M Series B, [company blog](https://www.browserbase.com/blog/series-b-and-beyond)) | Nen Labs | Hyperbrowser (Y Combinator) |
| Anti-bot and CAPTCHAs | Via partnerships (Cloudflare, Fingerprint), per its [docs](https://docs.browserbase.com/introduction/what-is-browserbase) | Stealth mode, residential proxies, CAPTCHA solving, per its [docs](https://docs.steel.dev) | "Ultra Stealth Mode"; CAPTCHA solving priced into agent steps, per its [docs](https://hyperbrowser.ai/docs/introduction) |
| Session recording | Yes, every agent step plus live view | Yes, session replays | Yes, video recordings |
| Logged-in state between runs | Context-style persistence | Persistent profiles | Profiles via API |
| Pricing shape (checked 2026-08-12) | Base fee plus usage: free tier, then [$20 and $99/month tiers](https://www.browserbase.com/pricing) with browser-hour overage around $0.10 to $0.12/hr | Usage-based with plan fees: [$0/month plus $30 one-time credit](https://steel.dev/pricing) to start, $250/month at scale | Pure credits (1,000 = $1): [$0.10 per browser-hour, $0.02 per agent step](https://hyperbrowser.ai/docs/reference/pricing) |
| Works with | Stagehand natively; LangChain, CrewAI, Mastra | Framework-agnostic: any Playwright or Puppeteer stack | Claude, OpenAI, Gemini, and browser-use agents built in |

_Only Steel has an open-source core, so this page carries no star-ranked table; Steel's list entry lives in [Libraries and SDKs](../README.md#libraries-and-sdks). Pricing shapes summarize each vendor's own pricing page on the date shown; check them before committing._

## Pick by situation

Three independent comparison write-ups ([APIScout](https://apiscout.dev/guides/browserbase-vs-steel-vs-hyperbrowser-browser-infrastructure-2026), [PkgPulse](https://www.pkgpulse.com/guides/browserbase-vs-hyperbrowser-vs-steel-cloud-browsers-ai-2026), and [Steel's own head-to-head](https://steel.dev/blog/steel-vs-browserbase-a-practical-comparison), bias noted) land on the same split, which is rare enough to trust:

- **You want the safest managed default with the most polish** → **Browserbase**. The most mature tooling story (Stagehand is its SDK), session visibility on every agent step, and the most generous named free tier of the three. The trade: fully closed, and features track plan level.
- **Openness or self-hosting is a buying criterion** → **Steel**. The only one whose core you can read, run in your own Docker, and walk away from without losing the integration; the hosted product is the same code with the ops handled. This is also the pick when procurement rules out closed infra handling logged-in sessions.
- **You're scraping or crawling at volume, agent-first** → **Hyperbrowser**. Credit pricing that meters agent steps rather than plans, stealth as the headline feature, and built-in support for driving it from browser-use and the major model providers.

Also in the lane, one tier out: [Browserless](https://www.browserless.io/comparison), the long-running self-hosted headless-Chrome standard, and Anchor Browser, a newer managed entrant focused on agent identity. Neither shows up in buyer comparisons as often as these three yet.

## The question to ask first

*Does your agent hold credentials?* A hosted browser that logs into your users' accounts concentrates real risk with the vendor: recordings, cookies, and persistent profiles all live on someone else's infrastructure. If the answer is yes and that sentence made you uncomfortable, Steel's self-hosted mode is the escape hatch the other two don't offer. If the answer is no (public-web scraping, testing, research), pick on developer experience and unit economics, which is the fight Browserbase and Hyperbrowser are actually having.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# Claude Code skill packs: superpowers vs GStack vs get-shit-done vs Anthropic Skills

A skill is a folder of instructions (a SKILL.md file, plus any scripts it needs) that a coding agent loads only when the task matches, instead of carrying every instruction all the time. A skill pack is a curated bundle of them: someone else's working habits for your agent, installed as files. Nothing to run, nothing to migrate off, which is why [How to pick a harness](how-to-pick-a-harness.md) says to start at this tier.

The cost is behavioral, not financial. A pack rewrites how your agent plans, tests, and commits, so the wrong pack means weeks of fighting a workflow you didn't choose, and stacking two packs that disagree is worse than running neither. One [published test of popular skills](https://www.firecrawl.dev/blog/best-claude-code-skills) found many made output worse. Install one, watch what changes, prune what never fires.

If you're deciding between a skill and the other extension points, the split is: files like CLAUDE.md hold what the agent must always know, skills hold procedures loaded on demand, subagents are separate workers with their own context, and MCP servers add external tools ([Anthropic's own explainer](https://claude.com/blog/skills-explained) covers this; so does our [context files](progressive-disclosure.md) page). The four packs below are different bets on what belongs in the skill folders.

| | [superpowers](https://github.com/obra/superpowers) | [GStack](https://github.com/garrytan/gstack) | [get-shit-done](https://github.com/open-gsd/gsd-core) | [Anthropic Skills](https://github.com/anthropics/skills) |
|---|---|---|---|---|
| ⭐ Stars | 289k | 134k | 9.7k | 177k |
| The bet | **Process**: test-driven development, systematic debugging, verification before claiming done | **Roles**: 23 slash-command modes (CEO review, eng review, design, QA, ship) that structure one assistant as a virtual team | **Plans**: goal-backward plans on disk, executed in waves over fresh context windows | **The format**: the official reference skills, plus document production (docx, pdf, pptx, xlsx) |
| Runs on | Claude Code plus 13 other harnesses (Codex, Cursor, OpenCode, Gemini CLI, more) | Claude Code | Claude Code, OpenCode, Gemini CLI | Claude Code, Claude.ai, the API |
| How it enforces itself | A startup hook re-injects its rules when a session begins and after compaction (when a long session gets compressed and standing instructions usually fall out) | Checkpoint mode auto-commits work as it goes; /freeze and /careful guardrails; its browser tool only allows pre-approved commands | The plan is a file on disk, so execution survives a dead session and picks back up | It doesn't: pure content, the host platform decides |
| Memory between sessions | Basic: an optional companion plugin adds recall | Strong: per-repo learnings plus an optional GBrain knowledge backend | The plan files are the memory | None |
| License | MIT | MIT | MIT | ⚠️ Per-skill Anthropic terms ("all rights reserved") |

_Stars as captured for the main list; rating definitions live in the [guide to rankings](../README.md#guide-to-rankings). Enforcement and memory rows come from this list's [deep-dive research](../attributes/RUBRIC.md) and each repo's own documentation._

## Pick by situation

- **You want the agent to work like a disciplined engineer** → **superpowers**. The most-starred pack of the category. Its skills are process (write the test first, debug systematically, verify before claiming done), and its startup hook re-injects that discipline exactly when packs usually lose it: after compaction.
- **You want a product team, not just an engineer** → **GStack**. Garry Tan's daily-driver stack turns one assistant into reviewers, QA, and a ship pipeline, with the strongest memory story of the four: what a project teaches the agent feeds its later sessions.
- **Your projects outlive single sessions** → **get-shit-done**. Its whole design is plans as files: work is planned backward from the goal, executed in fresh context windows wave by wave, and a crash or a cleared session resumes from the plan instead of starting over.
- **You're learning the format, or you need documents produced** → **Anthropic Skills**. The first-party reference implementations everything else builds on, plus the docx/pdf/pptx/xlsx workhorses. Read the licensing before you redistribute anything: these are not standard open source; each skill ships an all-rights-reserved Anthropic-terms license file.

## They compose, up to a point

Skills are folders, so nothing stops you from installing two packs, and a [combining guide](https://dev.to/imaginex/a-claude-code-skills-stack-how-to-combine-superpowers-gstack-and-gsd-without-the-chaos-44b3) exists for exactly the trio above. The failure mode is trigger overlap: two packs claiming the same moment ("before any bug fix, use MY debugging skill") leave the model choosing between rulebooks mid-task. Start with the pack whose bet matches your gap, add single skills from the others, and prune anything that never fires.

Beyond these four: [addyosmani/agent-skills](https://github.com/addyosmani/agent-skills) (84.9k stars) optimizes for portability, with 24 senior-dev workflow skills installable across 70+ agents; [ECC](https://github.com/affaan-m/ECC) is the breakout mega-pack (68 specialized subagents and 284 skills at last count); [wshobson/agents](https://github.com/wshobson/agents) is the cross-harness marketplace of drop-in agent definitions; and [awesome-claude-code](https://github.com/hesreallyhim/awesome-claude-code) is the most-followed catalog of the whole genre, the place to discover packs this page doesn't name.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# Eval and observability platforms: Langfuse vs LangSmith vs Braintrust vs Phoenix

Benchmarks tell you how a model ranks; your production agent still fails in ways no public exam covers. An eval and observability platform is where teams watch what their agent actually did (tracing: recording every step, tool call, and token of a run) and score it continuously (evals: checks that run on those traces, in CI or on live traffic). This is also where the eval budget actually gets spent, which makes it the most-compared purchase in the whole evaluation space. Our [Agent evals](agent-eval-harnesses.md) page covers the open-source benchmarks and frameworks one layer down.

| | [Langfuse](https://github.com/langfuse/langfuse) | [LangSmith](https://www.langchain.com/langsmith) | [Braintrust](https://www.braintrust.dev/) | [Phoenix](https://github.com/Arize-ai/phoenix) |
|---|---|---|---|---|
| What it is | Open-source tracing, evals, and prompt management in one platform | LangChain's commercial platform for the full agent lifecycle | Eval-first platform: versioned datasets, scoring, CI release gates | Arize's local-first tracing and eval layer, with managed Arize AX above it |
| Open source? | Yes: MIT core (33k stars; enterprise folders separately licensed) | No: closed platform, open client SDK only | No: closed platform, open SDKs only | Source-available: Elastic License 2.0 (11k stars), self-hostable but not OSI open source |
| Self-hosting | Free, first-class (Docker Compose, Helm) | Enterprise-only, paid | Enterprise-only, paid | Free, local-first by design |
| Center of gravity | Traces first, evals on top | Lifecycle: tracing, evals, deployment, tuned for LangChain/LangGraph | Scores first: "what reaches production" gates | Traces and experiments, on your own machine |
| Free tier (checked 2026-08-12) | Hobby cloud: 50k units/month, plus unlimited free self-host ([pricing](https://langfuse.com/pricing)) | Developer: 5k traces/month, 1 seat ([pricing](https://www.langchain.com/pricing)) | Starter: $10 credits, 14-day retention ([pricing](https://www.braintrust.dev/pricing)) | Self-host free with no event caps; managed AX from $0 ([pricing](https://arize.com/pricing/)) |
| Steward | Langfuse, acquired by [ClickHouse in January 2026](https://clickhouse.com/blog/clickhouse-acquires-langfuse-open-source-llm-observability) | LangChain | Braintrust | Arize AI |

_No star row here because two of the four have no open-source core to count. Langfuse, Phoenix, and Opik (below) carry entries in this list's [observability category](../README.md#observability-and-eval-ops)._

## Pick by situation

The vendors' own head-to-head pages, read against each other, agree more than you'd expect:

- **You want open source, self-hosting, and predictable cost** → **Langfuse**. The cleanest open-source credential of the four (MIT core) and the one whose [own positioning](https://langfuse.com/faq/all/best-phoenix-arize-alternatives) leads with exactly that. The cost gap it advertises is real but dated: a widely cited mid-2026 calculation put a million events near [$101/month on Langfuse Core against roughly $2,514/month on LangSmith Plus](https://www.morphllm.com/comparisons/langfuse-vs-langsmith); the units aren't equivalent and LangSmith's pricing model has since changed, so treat that as an order-of-magnitude signal, not current math.
- **You build on LangChain or LangGraph** → **LangSmith**. First-party tracing for that stack with the least integration work; [LangChain's own comparison](https://www.langchain.com/resources/langsmith-vs-langfuse) pitches it as covering the full lifecycle. The trades: closed platform, and self-hosting is an Enterprise feature, not an option you can default to.
- **Evals gate your releases** → **Braintrust**. Its bet is that scoring, not tracing, is the product: versioned datasets, experiments, and regression gates deciding what ships. [Its own LangSmith comparison](https://www.braintrust.dev/articles/langsmith-vs-braintrust) draws the same line. Closed platform; on-prem is Enterprise-only.
- **You want tracing on your laptop before you buy anything** → **Phoenix**. Local-first and free to self-host with no event caps; [Arize's framing](https://arize.com/pricing/) is stay local, graduate to the managed AX platform when you outgrow it. Mind the license: Elastic 2.0 is source-available, not open source, and restricts reselling Phoenix as a hosted service.

The fifth name that keeps appearing in these comparisons: [Opik](https://github.com/comet-ml/opik) (Comet, 21.3k stars, Apache-2.0), whose whole core feature set is free to self-host under a plain open-source license; the pick when Langfuse's enterprise-folder split bothers you.

## How this layer relates to the benchmarks

These platforms score *your* traffic continuously; benchmark frameworks like [SWE-bench and inspect_ai](agent-eval-harnesses.md) score fixed task sets once. Mature setups use both: a benchmark to choose the base model, a platform to catch the regressions your users would otherwise find. Dev-time eval libraries (promptfoo, DeepEval, Ragas) plug into either layer; worth knowing that [OpenAI acquired promptfoo in March 2026](https://openai.com/index/openai-to-acquire-promptfoo/), which has non-OpenAI teams re-checking their neutrality assumptions.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# How to pick a harness

This is the decision guide for [best-of-Agent-Harnesses](../README.md), a curated, ranked list of the runtimes that turn an AI model into a working agent: the software that decides what the model's thinking is allowed to touch. Six questions, in order. Each one eliminates most of the list; by the end you should be choosing between two or three projects instead of the entire list. The [use-case index](../README.md#pick-by-use-case), the [landscape charts](../README.md#the-landscape-at-a-glance), and the head-to-head comparison pages linked throughout do the heavy lifting.

## The chart to internalize first

[![Same model, different harness: coding-benchmark pass rates when only the harness changes](../assets/harness-spread.svg)](../assets/harness-spread.svg)

The most-repeated line in this month's harness debates is that ["the harness matters more than the model itself"](https://x.com/amy05sa6/status/2087347665311019129), and the measurements back the slogan. [Cursor's benchmarking research](https://www.mindstudio.ai/blog/agent-harness-scaffolding-matters-more-than-model) found the same model passing 46% of tasks in one harness and 80% in another. On SWE-bench Pro, [swapping only the harness](https://x.com/joelniklaus/status/2085725862142623875) moved GLM-5.2 from 23% to 52%; a [2026 line of research](https://arxiv.org/abs/2605.27922) now calls the harness the binding constraint on agent performance. And because harness rankings barely transfer between models (rank correlation about -0.05), the pick is a *pairing*, not a single winner. Two consequences for everything below: your harness choice deserves at least as much care as your model choice, and it needs re-asking whenever your model changes. When the shortlist is down to two or three, [test-drive them on your own work](how-to-test-drive-a-harness.md). The gap is even wider on long-horizon work: a bare frontier model was verified at about 30% on ARC-AGI-3, and [Prime Agent](https://arxiv.org/abs/2608.23552)'s harness took Opus 5 to 95.5%, the result the [YC Paper Club talk on why the harness matters more than the model](https://www.youtube.com/watch?v=n9xKblqyQ28) walks through; [who says so, and what the claim does not mean](why-the-harness-matters.md), is its own page.

## 1. What do you actually want it to do?

Don't start from frameworks; start from the job. The [Pick by use case](../README.md#pick-by-use-case) index maps 14 reader intents ("turnkey coding agent today", "drop-in memory layer", "always-on personal agent in my chat apps") to 2-7 curated picks each. If your job is on that list, you're already down to a handful of candidates.

## 2. How much do you want to adopt?

Every project carries a simplicity-to-capability tier measuring how much you take on by adopting it: **super simple** (a file format, nothing to run) → **mostly simple** (a small library or content bundle) → **slightly complex** (a real tool with moving parts) → **complex** (a platform with its own runtime and ecosystem). Rule of thumb: pick the *lowest* tier that solves the job, because every tier you go up is something you'll maintain, secure, and eventually migrate off. A skill pack on a harness you already run beats a new framework; a library beats a platform.

## 3. How much rope does it need?

The autonomy rating (★ in the tables = ready for unattended runs) describes how much a harness is designed to do without you watching: **step-gated** (asks before every action) → **checkpoint-gated** (asks at milestones) → **bounded** (runs free inside limits you set) → **headless** (built to run with nobody watching; here that's about supervision, not whether it has an interface). Match it to your actual risk tolerance, not your ambition: if you'll review every change anyway, a step-gated tool like Cline wastes nothing; if you want overnight runs, only bounded and above qualify, and the [Autonomy × Recovery grid](../README.md#the-landscape-at-a-glance) shows which. Autonomy is also shaped by configuration as much as by the model: the harness's approval defaults decide what actually happens on your machine.

## 4. What happens when it breaks?

Runs die: rate limits, crashed sandboxes, closed laptops. The recovery rating (✱ = durable) describes what survives: **none** (start over) → **retry** (it re-attempts failed steps) → **resumable** (a dead session can pick up where it stopped) → **durable** (execution state is persisted; even a process restart or redeploy doesn't lose the run). For anything long-running or unattended, treat **resumable** as the floor and **durable** as the bar for production; only a handful of projects clear it. A headless harness with no recovery story is an incident generator.

## 5. Who pays for the tokens?

The question 2026 keeps re-asking. In April 2026, Anthropic banned third-party agents from running on flat-rate Claude subscriptions; it then announced a separate metered credit pool for such usage, and on June 15, [paused that change the day it was due to take effect](https://thenewstack.io/anthropic-pauses-claude-agent-sdk-subscription-change/). As of August 2026, programmatic and third-party usage draws from normal subscription limits again, and Anthropic says any revised plan will come with notice. The lesson stands regardless of where the policy lands: **an always-on agent's economics can change under you, so price the usage shape, not the plan.** Heartbeat-driven agents (ones a timer wakes up all day, like OpenClaw and Hermes) burn tokens while idle; session-based tools (terminal coding agents) don't. Field reports put one unoptimized always-on setup near $360/month and the same workload under $10 after three fixes, in priority order: **wake less** (scripts that only invoke a model on a match), **route by tier** (cheap models for background work, never a frontier model on a timer), and **slim the tool list** (large tool registries tax every request). Your insurance policy is provider flexibility: harnesses differ sharply in how well they support metered API keys with budget caps and open-weight models on your own hardware. [OpenClaw vs Hermes](openclaw-vs-hermes.md) covers all of this in depth.

## 6. Can you walk away from it?

Prefer harnesses where your investment ports: instructions in open file formats (AGENTS.md for repo briefings, SKILL.md for on-demand procedures; see [Context files](progressive-disclosure.md)), standard protocols (MCP, the Model Context Protocol, the standard way tools plug into agents), permissive licenses (the license column in every table), and state you can export. Switching costs between terminal coding agents are deliberately low; trying two is cheap ([comparison](terminal-coding-agents.md)). Switching costs between platforms are not, which is one more reason question 2 says to start low.

## Worked examples

- *"I want code reviews while I sleep"* → use case: coding agent; autonomy: headless ★; recovery: resumable or better → opencode headless or OpenHands.
- *"I want a personal assistant in Telegram"* → use case: always-on personal agent; billing shape: always-on (watch question 5) → [OpenClaw vs Hermes](openclaw-vs-hermes.md), budget decision first.
- *"My multi-agent pipeline must survive deploys"* → recovery: durable ✱ → LangGraph (see the [orchestration comparison](multi-agent-orchestration.md)), or n8n, Letta, and Cloudflare Agents from the main list, all rated durable.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). The same data is queryable by agents via the [MCP server](../mcp/): `pick_harness(use_case, max_complexity, min_autonomy, min_recovery)` automates questions 1-4._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# How to test-drive a harness

Spec sheets cannot answer "which harness should I use," because an agent's performance is a property of the *pairing* between harness and model, not of either alone: the same model passes [46% of tasks in one harness and 80% in another](https://www.mindstudio.ai/blog/agent-harness-scaffolding-matters-more-than-model) in Cursor's benchmarking research, harness swaps moved [SWE-bench Pro scores by 21 to 29 points](https://x.com/joelniklaus/status/2085725862142623875), and harness rankings barely transfer between models (rank correlation about -0.05). Public benchmarks don't rescue you either: OpenAI [stopped reporting SWE-bench Verified](https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/) after auditing its tasks, and a leaderboard position measures someone else's repos, not yours. The only comparison that transfers to your work is running your work. This page is the protocol: an afternoon to set up, about two weeks to run, numbers at the end.

## Step 1: shortlist two or three, no more

Use [How to pick a harness](how-to-pick-a-harness.md) and the [use-case index](../README.md#pick-by-use-case) to get to 2-3 candidates. Trialing more than three divides your task set until no candidate gets enough data; the decision guides exist so you don't have to trial five.

## Step 2: make the race fair

- **Hold the model constant where you can.** If both harnesses accept the same model, use it, so you're measuring the harness. Where a candidate is first-party-only, accept the confound and write it down: you're comparing stacks, not harnesses.
- **Port your instructions once.** Write one AGENTS.md briefing and give every candidate the same one (the [terminal-agents guide](terminal-coding-agents.md) covers the two tools that need a setting flipped). Same MCP tools on each side.
- **Pin versions and write them down.** The pairing you test is the pairing you ship; harnesses ship weekly and behavior moves.
- **Fresh git worktree per candidate per task**, so no candidate inherits another's leftovers.
- **Define what "pass" means per task before running anything.** Written down, so the goalposts can't move after you've seen an output you like.

## Step 3: build the task set from your own work

Generic prompts measure nothing. The [error-analysis-first rule](https://www.lennysnewsletter.com/p/building-eval-systems-that-improve-your-ai-product) from Hamel Husain and Shreya Shankar's eval playbook applies fully here: you can't know what to test until you look at how work actually fails in your shop. The practical version is a golden-replay set, [pulled from your own recently merged work](https://futureagi.com/blog/evaluating-coding-agents-2026/): take 8-12 real, completed tasks from the last month (merged PRs, closed tickets) where you know what good looked like. Cover the spread:

- one small bugfix and one multi-file feature (the bread and butter)
- one refactor with tests (does it keep them green or delete them?)
- one question about unfamiliar code (comprehension, not generation)
- one long task that will blow past a single context window (compaction behavior)
- one task with destructive potential, run deliberately (does the permission model catch it?)

## Step 4: measure these seven things

The 2026 harness-effects research ([Harness-Bench](https://arxiv.org/abs/2605.27922)) scores configurations on success, token cost, robustness, and traceability; the columns below are that list extended with the two things buyers report caring about most, integration friction and exit cost.

| Dimension | Record per task | Why it predicts |
|---|---|---|
| Output quality | Human accept / accept-with-rework / reject | The only score that matters. Never let the agent grade itself: [one builder's overnight run](https://x.com/Skaly__Bull/status/2087662420672332087) self-passed 31 of 40 tasks; a human reading the same outputs passed 18 |
| Interventions | Times you had to redirect mid-task | The real autonomy number, whatever the marketing says |
| Plan drift | Did the final diff match the stated plan? | Catches harnesses that narrate one thing and do another |
| Wall-clock | Minutes per accepted task | Latency compounds across a team |
| Cost | Dollars per *accepted* task, not per token | Cheap tokens on rejected work is expensive work; note the bill shape too ([question 5](how-to-pick-a-harness.md)) |
| Robustness | Kill it mid-task; note what resumes | The recovery axis, observed instead of read |
| Setup friction | Minutes from clean machine to first accepted task, plus MCP/tool wiring effort | The adoption-surface tier, observed instead of read |

## Step 5: score against your own baseline

Absolute scores mean little; compare against how the same work goes without the candidate. Your repo already knows your baseline: time-to-merge, acceptance rate, and rework rate for recent human-authored changes. [A two-week trial with three to five engineers on real tickets](https://ucstrategies.com/news/autonomous-coding-agents-evaluation-framework/) produces enough accepted-task data for a fair comparison; a solo trial needs the full task set above instead.

Copy-paste scorecard:

```
| Task | Pass criteria | Harness A: verdict / interventions / min / $ | Harness B: verdict / interventions / min / $ |
|------|---------------|----------------------------------------------|----------------------------------------------|
| 1.   |               |                                              |                                              |
```

## The walk-away test

Before deciding, run one more test on the *winner*: export what you built during the trial (the briefing file, any skills, memory, traces) and time how long it takes to make the runner-up work with it. That number is your future switching cost, and it's the practical version of the [vendor-decoupling argument](progressive-disclosure.md): instructions, skills, memory, and tools should outlive any one harness or model, which is exactly why they belong in open formats. LangChain's Harrison Chase has argued the same from the other side: harness configuration is model-specific tuning, so when your model changes, the trial needs re-running, and portable assets are what make the re-run cheap.

## Pitfalls, all field-reported

- **The novelty window.** The first week flatters every new tool. The second week is where update churn and permission fights show up; the [OpenClaw vs Hermes field reports](openclaw-vs-hermes.md) found release breakage, not capability, was the top reason people switched.
- **Leaderboard transfer.** A 70% benchmark score doesn't survive contact with your repo's conventions; that's what [Agent evals](agent-eval-harnesses.md) calls the model-plus-harness problem, and it's why this page exists.
- **Grading your own homework.** Applies to the humans too: whoever ran the trial wants their favorite to win, so have a second person judge accept/reject on the diffs alone.
- **Deciding on price-per-token.** The spread that matters is dollars per accepted task; a harness that's 20% pricier per token and 40% better on acceptance is the cheap one.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# Managed vs self-hosted always-on agents

An always-on agent keeps working between your messages, and in 2026 you can rent one (Grok Bot, Claude Managed Agents), run one for your whole team (QM), run one for yourself (OpenClaw, Hermes), or keep one on your own laptop (OpenJarvis). This guide says who owns the computer in each case, who pays for idle time, and which one to pick.

| | [Grok Bot](https://docs.x.ai/grok-bot/approvals-security-and-privacy) | [Claude Managed Agents](https://platform.claude.com/docs/en/about-claude/pricing) | [QM](https://github.com/yc-software/qm) | [OpenClaw](https://github.com/openclaw/openclaw) / [Hermes](https://github.com/NousResearch/hermes-agent) | [OpenJarvis](https://github.com/open-jarvis/OpenJarvis) |
|---|---|---|---|---|---|
| Who runs it | xAI | Anthropic | You, for your team (Fly, AWS, or your own servers) | You, for yourself (a Mac mini, a VPS) | You, on the device in front of you |
| Where the agent's computer lives | One shared cloud computer per account | Anthropic's sandboxed sessions | Sandboxes your deployment allocates, plus Postgres for state | Your machine | Your machine; no cloud by default |
| Talk to it from | Desktop app (macOS, Windows), iPhone | Your own app, through the API | Slack and a web app, including shared rooms | WhatsApp, Telegram, Slack, Discord, and more | Desktop app and command line |
| Model | Grok, xAI's choice | Claude | Any: Pi, OpenCode, Codex, and Claude Code drive the same core | Any provider or local model | Local models (Qwen, Gemma, GPT-OSS) through Ollama, llama.cpp, or vLLM |
| License | Proprietary | Proprietary | MIT | MIT / MIT | Apache-2.0 |
| Cost shape | Subscription with a weekly usage allowance; extra usage billed from token cost | Tokens at API rates plus $0.08 per active session-hour; idle time is free | Your servers plus your model keys | Your hardware plus keys or a subscription | Your hardware; near-zero marginal cost |
| Approval model | Asks before sending, publishing, deleting, buying, or changing production; passwords and two-factor codes stay with you | Set by your code | Human-reviewed bulk writes; read-mostly by default | Relaxed by default (OpenClaw) or restrictive (Hermes) | Yours to set |

## What is Grok Bot, and is it an agent harness?

Grok Bot is xAI's always-on agent product, in beta since August 11, 2026. A Bot is a persistent, named agent: you message it from the Grok Bot app, give it a job and access to the tools it needs, and it keeps working on a cloud computer with a browser, files, and a command line while your own computer is off ([Composio's guide](https://composio.dev/content/guide-to-frok-bot), [xAI docs](https://docs.x.ai/grok-bot/approvals-security-and-privacy)). It is a harness in this list's sense: xAI owns the loop, the tool wiring, the memory, and the approval rules, and you rent the result. It is not in the ranked list because the list ranks open repositories, and Grok Bot has none.

Two design facts decide whether it fits you. All Bots on one account share the same cloud computer, files, browser sessions, and command-line credentials, so xAI's own docs say not to treat separate Bots as separate security boundaries. And the approval model is fixed by xAI: a Bot asks before sending messages, publishing, deleting, buying, or changing production systems, and passwords, two-factor codes, and CAPTCHAs stay with you. Skills (reusable workflows), routines (scheduled or event-triggered runs), and groups of two to six Bots working together round out the product. Access comes bundled with SuperGrok Heavy, Cursor Ultra, and eligible Cursor team plans, with a weekly usage allowance and extra usage billed from token cost ([pricing notes](https://www.eesel.ai/blog/grok-bot-pricing)).

## Grok Bot vs OpenClaw: who owns the computer?

This is the whole difference. OpenClaw runs on a machine you own, with the accounts you gave it, and its "free reign" defaults are yours to tighten; the [OpenClaw vs Hermes guide](openclaw-vs-hermes.md) covers the security posture, the update churn, and the token bill in depth. Grok Bot runs on a computer xAI owns, with approval gates xAI wrote, and a bill that tracks xAI's plan changes. Pick OpenClaw or Hermes if you want to read the code that touches your accounts, run any model, and own the state. Pick Grok Bot if you want an agent with a computer by tonight and one vendor holding the keys is acceptable.

## QM vs OpenClaw: personal or multiplayer?

QM (Quartermaster) is Y Combinator's own harness, open-sourced under MIT on July 31, 2026, after months of internal use across accounting, legal, events, and engineering ([repo](https://github.com/yc-software/qm), [MarkTechPost](https://www.marktechpost.com/2026/08/03/y-combinator-open-sources-qm-multiplayer-ai-agent-harness/)). Its README draws the line itself: most agents are designed like personal assistants; QM is designed for startups. Every person and every room gets its own scoped memory, files, credentials, permissions, crons, web apps, and sandbox, and people can work with it together in a Slack channel.

The architecture choice, from the team's talk at YC's Paper Club (52:00 in [the video](https://www.youtube.com/watch?v=n9xKblqyQ28)): pull the brain out of the sandbox. OpenClaw and Hermes give the agent its own computer, which is powerful and also traps every session inside that computer, so a fleet of fifty of them became a whack-a-mole of SSH repairs. QM stores everything in Postgres, treats sandboxes as a resource the agent dips into (a small one for simple work, a bigger machine for heavy dev jobs), and keeps the harness to three core tools: run code in a remote sandbox, read and write object storage, publish an internal app. Two lessons from running it: agents give up too early, so QM puts wall-clock and token budgets on goals; and the amount of knowledge you can safely put in a shared brain is bounded by how good your permission system is.

Pick OpenClaw or Hermes for one person and one machine. Pick QM when several people share one agent and you already have the permissions to keep their data apart.

## OpenJarvis: the model runs on your own device

OpenJarvis, from Stanford's Hazy Research and Scaling Intelligence labs, keeps the whole stack on the device: model inference, agent execution, memory, and learning ([repo](https://github.com/open-jarvis/OpenJarvis), [Stanford write-up](https://scalingintelligence.stanford.edu/blogs/openjarvis)). It defines five primitives (the interfaces, the agent logic, the model, the inference engine, and the tools, memory, and learning around them) and lets a cloud model tune that local configuration once, so you get the tuning without paying cloud prices at run time. The team's numbers from the talk (43:00): on-device models trail the frontier by six to twelve months, and an optimized local stack ran personal-assistant and coding tasks at 800 times lower cost than the cloud. Pick it when privacy or cost rules out the cloud and your tasks fit a model that runs on your hardware.

## Claude Managed Agents: the API-shaped version

Anthropic's Managed Agents (public beta since April 8, 2026) rent you the infrastructure, not a chat product: sandboxing, long-running sessions, state, and error recovery behind an API, billed at token rates plus $0.08 per session-hour while the session is active, with idle time free ([pricing](https://platform.claude.com/docs/en/about-claude/pricing)). You bring the interface and the agent logic. It belongs in this comparison because it is the managed answer for teams who want to ship their own product on top rather than adopt someone else's assistant.

## When managed beats self-hosted

Managed wins when you have no one to run servers, when the job needs a browser and a computer today, or when the vendor's approval model is stricter than what you would build. Self-hosted wins when the agent needs your credentials and you want to read the code that uses them, when you want to swap models (the [pairing rule](why-the-harness-matters.md) says the best harness depends on the model), when you need state you can export, or when the workload runs all day and a metered bill would hurt.

## Who pays for idle time?

Always-on means the clock runs while nothing happens. Grok Bot meters a weekly allowance and bills the overage from token cost. Managed Agents charge only while a session is active. QM, OpenClaw, and Hermes cost whatever your server and model keys cost, which the [OpenClaw vs Hermes guide](openclaw-vs-hermes.md#the-billing-scare-april-to-june-2026) shows can be $360 a month unoptimized and under $10 after three fixes. OpenJarvis costs electricity. Price the usage shape before you price the plan.

## Which one should you pick?

- *One person, one machine, a chat-app front end* → OpenClaw or Hermes ([head-to-head](openclaw-vs-hermes.md)).
- *A team sharing one agent in Slack, with per-room permissions* → QM.
- *A computer-using agent by tonight, vendor holds the keys* → Grok Bot.
- *Your own product on rented agent infrastructure* → Claude Managed Agents.
- *Nothing leaves the laptop* → OpenJarvis.

Autonomy and recovery scores for the open-source options are on their project pages in the [ranked list](../README.md#personal-agent-runtimes).

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). Facts checked 2026-09-14; corrections welcome as issues with a source link._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# Agent memory layers: Mem0 vs Zep vs Letta vs claude-mem

Agents forget. A model keeps nothing between sessions, so anything your agent should still know tomorrow (who the user is, what was decided, what failed last time) has to live in a memory system outside the model. "Add memory to my agent" then hides genuinely different products, and picking by star count instead of by shape is how teams end up adopting a whole runtime when they needed a plugin. The three shapes: a **memory layer** you call from any agent (Mem0, Zep), an **agent runtime** where memory is the core abstraction and your agents live inside it (Letta), and a **harness plugin** that gives the coding agent you already run automatic recall (claude-mem).

| | [Mem0](https://github.com/mem0ai/mem0) | [Graphiti / Zep](https://github.com/getzep/graphiti) | [Letta](https://github.com/letta-ai/letta) | [claude-mem](https://github.com/thedotmack/claude-mem) |
|---|---|---|---|---|
| ⭐ Stars | 65.7k | 31k | 24.8k | 94.3k |
| Shape | Memory layer / API | Memory layer / engine | Agent runtime with built-in memory | Harness plugin |
| Works with | Any agent or framework | Any agent or framework | Agents you build *inside* Letta | Claude Code, Codex, OpenClaw, Gemini, Copilot, and more |
| Memory model | Extracted facts, scoped per user, agent, or session, retrieved on demand | A temporal knowledge graph: facts about people and things, plus *when* they were true, so answers can change as facts change | Self-editing memory (the MemGPT research lineage): the agent maintains its own memory as part of how it thinks | Captures what a session did, compresses it with AI, and injects the relevant parts when you resume |
| License | Apache-2.0 | Apache-2.0 | Apache-2.0 | Apache-2.0 |
| Adoption surface (list tier) | slightly complex | slightly complex | mostly simple | slightly complex |

_Stars as captured for the main list; the [guide to rankings](../README.md#guide-to-rankings) defines the rating vocabulary. Mem0, Graphiti, and claude-mem run no agent loop of their own (your agent stays in charge); Letta is rated headless autonomy with durable recovery on the list's axes, because your agents run as persistent entities on its server._

## Pick by situation

- **You have an agent and want it to remember users** → **Mem0**. The default drop-in: store and retrieve scoped memories from any framework via an API or library, with hosted and self-managed backends. If you're asking the generic question, this is the generic answer.
- **Your memory questions involve time and relationships** ("what did this customer believe before the refund?") → **Graphiti**, the open-source engine behind Zep's hosted platform. Plain fact-stores overwrite; a temporal knowledge graph keeps the history, which is the difference when facts change and the change matters.
- **You're designing an agent *around* memory** → **Letta**. The MemGPT lineage: agents that manage their own memory as a first-class behavior rather than calling out to a store. You're adopting a runtime, not adding a layer; that's the right trade for long-lived, persistent-persona agents and over-engineering for "remember the user's name."
- **Your "agent" is a coding assistant you already run** → **claude-mem**. Don't build memory infrastructure for a harness that takes a plugin: it captures everything a session does and hands the relevant context to future sessions. It started Claude Code-only and now covers Codex, OpenClaw, Gemini, and Copilot too. Its star count, highest on this page, says how common this situation is.

Also in the same space on the main list: [cognee](https://github.com/topoteretes/cognee), which turns your data into a queryable knowledge graph plus vector store through an extract-and-load pipeline; it competes with Mem0 and Graphiti at the layer shape and shows up in most four-way memory comparisons.

## The question to ask first

*Who decides what gets remembered?* If your application code decides (you choose what to store and when to look it up), you want a memory layer: Mem0, Graphiti, or cognee. If the agent itself should decide as part of how it operates, you want a runtime: Letta. If you'd rather nobody has to decide because the tool records and recalls automatically, you want a harness plugin: claude-mem. Answer that one question and the shortlist usually collapses to one shape.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# Multi-agent orchestration: OpenAI Agents SDK vs CrewAI vs AutoGen vs Agent Framework vs LangGraph

Orchestration is the layer that coordinates several AI agents working on one job: who acts next, what they share, and what happens when a step fails. The five frameworks here answer that with genuinely different architectures, and the pick is expensive to get wrong because the coordination model shapes your whole codebase, not just one file. One of the five also changed status in a way most comparison articles haven't caught up with: AutoGen, still the most-starred name in the category, is officially in maintenance mode.

| | [openai-agents-python](https://github.com/openai/openai-agents-python) | [CrewAI](https://github.com/crewAIInc/crewAI) | [AutoGen](https://github.com/microsoft/autogen) | [Agent Framework](https://github.com/microsoft/agent-framework) | [LangGraph](https://github.com/langchain-ai/langgraph) |
|---|---|---|---|---|---|
| ⭐ Stars | 29.6k | 58.8k | 61.1k | 13.6k | 42k |
| Coordination model | **Handoffs**: one agent passes the whole conversation to another, like a call-center transfer | **Roles**: agents defined by role, goal, and backstory collaborate in Crews; Flows (its event-driven control layer) steer production paths | **Conversation**: agents talk in a group chat until a stop condition ends it | **Workflows**: graph-based workflows merging the AutoGen and Semantic Kernel lines, in Python and .NET | **Graph**: you draw an explicit map of states and steps; agents are nodes on it |
| Status (checked 2026-08-12) | Active | Active | ⚠️ Maintenance mode: its own README says no new features and points new users at Agent Framework | Active: the designated successor, 1.0 GA in April 2026 | Active |
| How visible is the control flow | Medium: it emerges from the handoff rules you wrote | Low to medium: declarative, the framework decides | Low: it emerges from the dialogue | High: workflows are explicit | Highest: you drew the map yourself |
| Production features | Guardrails (checks that block bad inputs/outputs) and tracing (a step-by-step run log); you own the loop | Flows, hierarchical control | Code execution, human approval points | Graph workflows, checkpointing, .NET + Python | Checkpointing (saving run state so it can restart), durable execution, human approval points |
| Autonomy (list axis) | bounded | bounded | bounded | bounded | headless |
| Recovery (list axis) | resumable | resumable | resumable | resumable | durable |
| Adoption surface (list tier) | mostly simple | complex (product suite) | complex (product suite) | slightly complex | slightly complex |

_Stars as captured for the main list. The list-axis rows use this site's rating ladders: autonomy runs step-gated → headless (how unattended a tool is designed to run), recovery runs none → durable (what survives a crash); definitions in the [guide to rankings](../README.md#guide-to-rankings)._

## Pick by situation

- **You want the least framework between you and the model** → **OpenAI Agents SDK**. Handoffs plus guardrails and almost nothing else; the smallest adoption surface of the five, and multi-provider in practice ([LiteLLM and per-agent model adapters](https://openai.github.io/openai-agents-python/models/) mean it is not OpenAI-only). Start here if unsure: it is the cheapest to abandon if you outgrow it.
- **You think in team structures** → **CrewAI**. Role-based crews are the fastest path to a working multi-agent demo and the most readable to non-engineers. The trade: it is a product suite, so you adopt its worldview, and stepping outside the declarative style means fighting the framework. Flows exist to win back control in production.
- **Your problem is genuinely conversational** → **AutoGen**, with eyes open. Group chat fits problems where agents *should* debate (review panels, negotiation simulations, brainstorming) and fights you when you wanted a pipeline. The status row is the real caveat: [Microsoft's own README](https://github.com/microsoft/autogen) declares maintenance mode, so treat it as stable-but-frozen. The community continuation is [AG2](https://github.com/ag2ai/ag2), which carries the original conversation-based line forward.
- **You're on Microsoft's stack, or leaving AutoGen** → **Agent Framework**. The convergence of AutoGen and Semantic Kernel, generally available since April 2026, with graph workflows and checkpointing in Python and .NET. It is where Microsoft's investment actually goes now.
- **It's going to production and must survive restarts** → **LangGraph**. Explicit graphs, checkpointing, and durable execution make it the infrastructure-grade choice; the same explicitness makes it the most up-front design work of the five. If your "multi-agent system" is really a workflow with LLM steps, LangGraph is the framework that says so out loud.

Worth knowing beyond the table: [Google ADK](https://github.com/google/adk-python) (Google's engineering-first framework, with agent-to-agent interop), [Agno](https://github.com/agno-agi/agno) (performance and multi-modal focus), and [Mastra](https://github.com/mastra-ai/mastra) (the TypeScript-native answer in a Python-first field) all show up in buyer comparisons now, and all three are on the main list.

## The unfashionable default

Most "multi-agent" systems in the wild are one coordinator delegating to workers that don't talk to each other and keep no state of their own. All five frameworks can express that, and so can a plain loop over your provider's SDK. Reach for an orchestration framework when agents need to *interact*: share evolving state, contest each other's outputs, or hand a live task around. If they just need to run in parallel and report back, you may not have a framework problem at all.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# OpenClaw vs Hermes: the always-on personal-agent debate

An always-on personal agent is a program that runs all day on your own machine, talks to you through the chat apps you already use (WhatsApp, Telegram, Slack, Discord), and keeps acting between your messages: watching inboxes, running scheduled jobs, remembering what you told it last month. OpenClaw and Hermes are the two big open-source ways to run one, and two of the most-starred projects on this entire list. The choice matters because it sets three things at once: your security posture (this thing has your accounts), your maintenance load, and, since June 2026, your monthly AI bill. The flame war treats them as rivals; the architecture says they are opposite design philosophies that happen to share a category.

| | [OpenClaw](https://github.com/openclaw/openclaw) | [Hermes](https://github.com/NousResearch/hermes-agent) |
|---|---|---|
| ⭐ Stars | 390k | 247k |
| Steward | OpenClaw Foundation (community) | Nous Research |
| License | MIT | MIT |
| Core language | TypeScript | Python |
| The bet | **Presence**: one event loop where messages, timers, scheduled jobs, and outside events feed a single conversation; it accumulates unbounded memory of *you* and feels like a person | **Discipline**: separated execution domains, a deliberately small (~3k-character) user model, and script-gated wake-ups; it feels like a harness, by design |
| Proactivity | Native heartbeat: a built-in timer wakes the agent, so background events can speak up in your main conversation | No heartbeat. Scheduled jobs run isolated from the conversation, and a `wakeAgent` script decides whether a check is even worth waking the model |
| Ecosystem | 13,700+ community skills (drop-in instruction packs) on ClawHub; the most chat channels and integrations; native multi-agent | Lean: skills are mostly self-generated by its learning loop; checkpoint and rollback built in |
| Security posture | Relaxed by default ("free reign"); users sandbox it themselves | Restrictive by default: permission prompts, confined script locations, per-domain isolation (users report it "works on the third try") |
| Config surface | Fully drivable from chat | Several commands are CLI-only, so it is hard to administer from your phone |
| Release tempo (checked 2026-08-12) | 100+ releases; fast-moving, frequently breaking | 24 releases; conservative |
| Tokens per request | ~48k average ([one user's OpenRouter telemetry](https://old.reddit.com/r/AskClaw/comments/1tkekk3/discussion_hermes_vs_openclaw_no_fanboyism_allowed/)) | ~62k average, same source: heavier per request, lighter when idle |
| Autonomy (list axis) | headless | headless |
| Recovery (list axis) | resumable | resumable (checkpoint/rollback is a marquee feature) |

_Stars as captured for the main list. Both are rated headless (designed to run unattended) and resumable (a dead run can pick back up); the [guide to rankings](../README.md#guide-to-rankings) defines this vocabulary._

## What the field reports actually say

Primary-source notes from the debate threads (r/openclaw, r/hermesagent, r/AskClaw, r/AI_Agents; links inline), claims attributed as claims:

- **The migration driver is update churn, not features.** The recurring OpenClaw complaint isn't capability; it's that frequent releases break working setups ("the last 5 updates in a row", broken Telegram sessions, flaky scheduled jobs; [experience report](https://old.reddit.com/r/openclaw/comments/1t2m2uu/yet_another_openclaw_vs_hermes_experience_sharing/)). Hermes converts say they are "in the repair shop one-eighth of the time." **The confounder, named by an OpenClaw user:** Hermes has shipped a fraction of OpenClaw's release count (24 vs 100+ as of August 2026), so it has had less time to accumulate breakage; its stability record is partly an artifact of age ([thread](https://old.reddit.com/r/openclaw/comments/1sdw7xc/seeing_a_lot_of_migrating_from_openclaw_to_hermes/)). Counter-reports exist in both directions ("I went to Hermes and it was a nightmare… back with OpenClaw").
- **The learning loop is real, and double-edged.** A [three-week side-by-side](https://old.reddit.com/r/AI_Agents/comments/1sh2r25/i_ran_hermes_openclaw_sidebyside_for_3_weeks/) had Hermes turn a daily news-briefing task into a reusable skill plus a scheduled routine, unprompted. The dark side, from a detailed migration report: bad decisions get learned too, "etched in stone", and scraping a mislearned pattern out of skills, memory, and scheduled jobs "reminds me of scraping a virus infection out of your PC manually."
- **Run-both is the power-user consensus, with a twist.** OpenClaw as the coordinator, Hermes as the execution specialist. The most-cited benefit isn't throughput, it's **redundancy**: multiple independent reports of telling one agent to diagnose and fix the other when it bricks. Cost of running both: roughly +30%, for reportedly much more than +30% output.
- **Trust the threads less than usual.** Both camps accuse the other of astroturfing, and the "everyone is migrating" narrative is contested inside OpenClaw's own subreddit. Volume of sentiment is unusually weak evidence here; the mechanism-level reports above are what's load-bearing.

## The billing scare (April to June 2026)

Some context if you're new to this: running an agent on "subscription auth" means logging it in with the flat-rate Claude subscription you already pay for, instead of a metered API key that bills per token. In 2026 that arrangement nearly changed, twice.

- **Early April 2026:** Anthropic banned third-party agents from running on Claude subscriptions (capacity issues; an estimated 135,000+ OpenClaw instances were on subscription auth), then [reinstated them](https://venturebeat.com/technology/anthropic-reinstates-openclaw-and-third-party-agent-usage-on-claude-subscriptions-with-a-catch) alongside an announced catch: a separate metered [Agent SDK credit pool](https://thenewstack.io/anthropic-agent-sdk-credits/) ($20 to $200 per month by plan, billed at API rates) that would end flat-rate always-on agents.
- **June 15, 2026:** Anthropic [paused the credit-pool change the day it was due to take effect](https://thenewstack.io/anthropic-pauses-claude-agent-sdk-subscription-change/). As of August 2026, third-party and programmatic usage draws from normal subscription limits, and Anthropic says any revised plan will come with notice.

The episode is the lesson: an always-on agent's economics sit on policy that can move under you. What the ops threads add: the durable cost problem is usage shape, not harness choice.

- The trap is **frontier models on background work**. One thread's summary: running a top-tier model on heartbeats is "hiring a PhD physicist to check whether the fridge door is closed" every 15 minutes. One lightweight monitoring setup ran [$0.50/hour, about $360/month](https://old.reddit.com/r/AI_Agents/comments/1tjs12x/how_are_people_keeping_openclawhermes_agents/) before optimization.
- The fix that recurs across threads, in priority order ([worked example](https://old.reddit.com/r/better_claw/comments/1tgfbig/dont_quite_openclawhermes_because_of_api_costs_do/), claimed ~80% reduction in an afternoon): **(1) wake less**: deterministic scripts and narrow watches that only invoke a model on a match (Hermes's `wakeAgent` and OpenClaw's model-bypassing `message send` both support this); **(2) route by tier**: cheap or free models for background work, a mid-tier model for conversations, never a frontier model on a timer; **(3) slim the tool list**: big tool registries silently tax every request. Field claims put monitoring setups under $5 to $10 per month after all three.
- The hedges people adopted during the scare, and mostly kept: open-weight models (run locally or via OpenRouter), and spreading load across other providers' plans. Both harnesses are provider-flexible; Hermes most natively, since open weights are Nous's founding thesis.

## Our read

Forget the flame war's framing of better and worse. **OpenClaw optimizes for presence; Hermes optimizes for discipline.** Almost every observed difference (heartbeat vs isolated scheduled jobs, unbounded vs bounded memory, chat-everything vs CLI-first admin, relaxed vs restrictive security) falls out of that one fork in the design tree. Pick by which failure you'd rather live with: an exuberant agent you have to contain and patch weekly, or a careful one you have to coax and can't fully drive from your phone. If you genuinely can't choose, the coordinator-plus-specialist pattern is real, the two interoperate over ACP (a protocol that lets one agent hand work to another), and roughly +30% cost is a fair price for agents that can repair each other. Whatever you pick, your bill is determined by questions 3 to 5 of [How to pick a harness](how-to-pick-a-harness.md), not by this page.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# Context files for agents: AGENTS.md vs CLAUDE.md vs skills vs MCP tool search

A model has a context window: a fixed amount of text it can consider at once. Everything competes for that space: your instructions, the definitions of every tool the agent could call, and the output of every tool it already called. Context bloat is what happens when the "might need it" pile crowds out the actual task, and it makes every request slower, dumber, and more expensive. The fix has one name across all its forms: progressive disclosure. Give the model a map first, and load details only when they're needed.

Bloat enters through three doors: instructions, tool definitions, and tool output. The four things in the table guard the first two doors, and they're the ones people actually weigh against each other: the [AGENTS.md vs CLAUDE.md question](https://news.ycombinator.com/item?id=44957443) has produced some of the biggest agent-tooling threads on Hacker News (837 points; a [files-vs-skills follow-up](https://news.ycombinator.com/item?id=46809708) drew 524 more).

| | [AGENTS.md](https://agents.md) | CLAUDE.md | [Skills](https://github.com/anthropics/skills) | [MCP tool search](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool) |
|---|---|---|---|---|
| What it is | An open, cross-tool briefing file at the repo root; 60k+ repos use it, and stewardship moved to the Linux Foundation in December 2025 | Claude Code's native briefing file: same job, one tool | Folders of instructions and scripts (SKILL.md) the agent loads only when the task matches | A built-in Claude Code feature: with many MCP servers connected, tool definitions load on demand instead of upfront |
| Door it guards | Instructions | Instructions | Instructions (on demand) | Tool definitions |
| Loaded | Every session | Every session | When triggered | When the agent searches for a tool |
| Read by | 20+ tools (Codex, Cursor, Copilot, Zed, more) | Claude Code | Claude Code, Claude.ai, the API, plus compatible harnesses | Claude Code (shipped January 2026, on by default; Anthropic claims ~85% fewer tool-definition tokens) |
| Scales by | Nesting: a file per directory scopes instructions to that part of the repo | Hierarchy: global, project, and local files stack | Adding skills costs almost nothing until triggered | Automatically |

_MCP is the Model Context Protocol, the standard way tools plug into agents. The [guide to rankings](../README.md#guide-to-rankings) defines this site's rating vocabulary._

## Which layer owns which instruction

This is the question people actually ask ("I have CLAUDE.md, AGENTS.md, skills, subagents, and MCP servers, and I don't know where anything belongs"), and it has a short answer:

- **The agent must always know it** (build commands, conventions, hard rules) → **AGENTS.md**, and keep it a map, not an encyclopedia. The community's hard-won rule: [a good one reads like a model upgrade, a bad one is worse than no docs](https://news.ycombinator.com/item?id=47938417).
- **Only Claude Code needs it** (your personal workflow rules) → **CLAUDE.md**. In mixed-tool teams the common pattern is AGENTS.md as the single source with CLAUDE.md pointing at it; Claude Code reading AGENTS.md natively is a [long-running feature request](https://github.com/anthropics/claude-code/issues/6235).
- **It's a procedure, only sometimes needed** (how to cut a release, how to write a PDF) → **a skill**. This is progressive disclosure in its purest form, and whole [packs of them exist](claude-code-skill-packs.md). The one caution from the field: skills only help when they trigger, which is exactly what the 524-point files-vs-skills thread found teams struggling with.
- **It's a tool** → an MCP server, and let **tool search** load the definitions. Before that feature, connecting many servers could eat a meaningful slice of the window before the first instruction; that pain built a whole cottage industry (below). For heavy tool users Anthropic also documents a stronger pattern: [give the agent a code environment that calls tools programmatically](https://www.anthropic.com/engineering/code-execution-with-mcp) instead of loading schemas at all.
- **Tool output floods the window** (huge JSON, logs, scraped pages) → summarize, offload to files, or use an output-side layer (below). This third door has no standard answer yet.

## The shelf: research and third-party layers

Worth knowing, mostly not worth building on today. [MCP-Zero](https://arxiv.org/abs/2506.01056) is the research result for tool-schema routing (~98% token reduction on the APIBank benchmark), but its [code](https://github.com/xfey/MCP-Zero) has been frozen since July 2025: cite the paper, don't build on the repo. [ToolGen](https://arxiv.org/abs/2410.03439) (ICLR 2025, a major machine-learning conference) showed retrieval and invocation can be one generative step; also frozen. [langgraph-bigtool](https://github.com/langchain-ai/langgraph-bigtool) is the maintained version of retrieve-then-load for LangGraph (LangChain's agent framework); [spring-ai-tool-search-tool](https://github.com/spring-ai-community/spring-ai-tool-search-tool) did the same for Java's Spring and has since been folded into Spring AI's core. On the output door, [context-mode](https://github.com/mksglu/context-mode) (19.7k stars) intercepts tool results and hands the model a summary, and [Headroom](https://github.com/headroomlabs-ai/headroom) compresses tool output as a library, proxy, or MCP server. Two cautions on context-mode: it's source-available under the Elastic License 2.0 rather than open source, and its npm installs have fallen by roughly a third from their May 2026 peak (per npm registry data) since the native tool-search feature shipped. The pattern to internalize: platform defaults keep absorbing this category, so prefer the built-in fix and treat third-party layers as bridges. For public websites, [llms.txt](https://llmstxt.org) is the same map-first idea applied to a site instead of a repo.

## Pick by situation

- **One repo, several tools touching it** → **AGENTS.md**, nested per directory in big repos. Free, no software, every major harness reads it.
- **Claude Code is your only harness** → **CLAUDE.md** is equivalent and native; adopt AGENTS.md the day a second tool shows up.
- **Procedures are bloating your briefing file** → move them to **skills**; the briefing file keeps the map.
- **Dozens of MCP servers** → the built-in **tool search** already fixed most of it; the code-execution pattern is the heavyweight option beyond that.
- **Output is the leak** → compaction plus file offloading first, then context-mode or Headroom if you need a dedicated layer.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# Agent sandboxing: what it is and how to pick

An AI agent does not just suggest code. It runs code, opens web pages, and edits files on a real computer. Agent sandboxing means making that computer a disposable one, so that when the agent makes a mistake, or a malicious web page talks it into one, the damage lands in a box you can throw away instead of on your laptop or your production servers. In August 2026 [an agent hacked a gym's booking system](https://www.bbc.com/news/articles/cn0nww2qlp7o) to get its user into a full pilates class; nobody asked it to. Docker's answer to the category, launched the same month to a [687-point Hacker News thread](https://news.ycombinator.com/item?id=49239751), was pitched as the fix for "what happens when the agent does something you didn't ask for."

This page gives you the concepts first, because the products only make sense once you have them. Then it compares the field.

## The problem, in one sentence

You are combining untrusted code with a trusted machine. The untrusted code comes from two places: the model itself, which writes and runs programs nobody reviewed, and the internet, because any web page or file an agent reads can contain instructions that steer it (this is called prompt injection). The fix is the same for both: give the agent its own machine.

## Two different jobs share the name "sandboxing"

Vendors use one word for two different products, and most confusion in this space traces back to that. A [widely shared ranking thread on r/AI_Agents](https://old.reddit.com/r/AI_Agents/comments/1sh2x4p/i_compared_sandbox_options_for_ai_agents_heres_my/) landed on the same conclusion: "best sandbox" depends on which job you mean.

1. **Sandbox the agent.** The whole agent program runs inside the box: it can only see and break what the box contains. This is what [Docker Sandboxes](https://www.docker.com/products/docker-sandboxes/) does for Claude Code, Codex CLI, Copilot CLI, and Gemini CLI, and what [Agent Sandbox](https://github.com/kubernetes-sigs/agent-sandbox) does on Kubernetes.
2. **Give the agent a sandbox.** The agent runs wherever it already runs, and gets a disposable remote computer as a tool for executing the code it writes. This is E2B's and Modal's product, often called a code interpreter.

Careful setups do both: the thread's author runs generated code in a sandbox even when the agent itself is already in one.

## The concepts that decide the pick

- **Isolation boundary.** How hard is the wall? From weakest to strongest: a process sandbox (tools like bubblewrap restrict what one program can touch; no infrastructure, but the operating system kernel is shared), a container (its own filesystem, still a shared kernel, so a kernel bug is an escape route), a user-space kernel (gVisor, which Modal uses: syscalls hit a decoy kernel first), and a microVM (a lightweight virtual machine with its own kernel and a hardware-enforced boundary; E2B and Vercel Sandbox run Firecracker, the microVM engine built for AWS Lambda, and [Docker built its own equivalent](https://www.docker.com/blog/why-microvms-the-architecture-behind-docker-sandboxes/) so it works on macOS and Windows).
- **Lifecycle.** Agents create and destroy machines constantly, so start-up time is a per-turn tax: [one 2026 benchmark write-up](https://particula.tech/blog/modal-vs-e2b-vs-daytona-vs-vercel-sandbox-ai-code-execution) measured cold starts around 90ms for Daytona and 150ms for E2B. Snapshot and fork let you checkpoint a machine's exact state and branch it; pause and resume lets a long job idle without paying for compute. Warm pools (pre-booted sandboxes waiting for work) are how Kubernetes-based setups dodge the cold start entirely.
- **The network boundary.** The most-missed axis. A sandbox can isolate the filesystem perfectly and still let the code inside open a connection and send your secrets out. Look for outbound-network controls: a domain allowlist at minimum (Docker's Sandbox Kits declare allowed network domains in the sandbox spec), inspection of what is actually sent at best.
- **State.** Does the box keep files and installed packages between turns and sessions, or is every run a fresh machine? Code-interpreter products lean stateless; agent-workspace products lean stateful.
- **Where it runs.** A hosted API (fastest to adopt, someone else's cloud), your own cluster (data stays home, you operate it), or your own laptop (for running coding agents unsupervised locally).

## The field, August 2026

Start with the open-source anchors this list tracks:

| | [E2B](https://github.com/e2b-dev/E2B) | [Daytona](https://github.com/daytonaio/daytona) | [Agent Sandbox](https://github.com/kubernetes-sigs/agent-sandbox) |
|---|---|---|---|
| ⭐ Stars | 13.9k | 71.7k | 4k |
| License | Apache-2.0 | ⚠️ AGPL-3.0 at v0.190.0, the final open release | Apache-2.0 |
| Maintenance (checked 2026-08-12) | Active | ⚠️ Unmaintained since June 2026 | Active |

_Stars as captured for the main list (see the [guide to rankings](../README.md#guide-to-rankings), which also defines the autonomy, recovery, and adoption-surface ratings used across this site). None of these three runs an agent loop of its own; they are the floor an agent stands on._

The wider field includes closed products, and in this category they matter. Isolation and deployment claims below come from each vendor's own documentation or the linked write-ups.

| Product | Runs | Isolation | The one thing to know |
|---|---|---|---|
| [E2B](https://github.com/e2b-dev/E2B) | Hosted API (open-source infra) | Firecracker microVMs | The default code-interpreter pick; its [cookbook runs all of Claude Code inside a sandbox](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/anthropic-claude-code-in-sandbox-python) |
| [Modal](https://modal.com) | Hosted API | gVisor | The pick when the sandbox side needs GPUs; E2B's [most-compared rival](https://northflank.com/blog/e2b-vs-modal) |
| [Daytona](https://github.com/daytonaio/daytona) | Hosted | Containers, optional stronger runtimes | Fastest measured cold starts; ⚠️ public repo unmaintained since June 2026 (below) |
| [Vercel Sandbox](https://vercel.com/docs/vercel-sandbox) | Hosted | Firecracker microVMs | Bills active CPU time rather than sandbox lifetime |
| [Cloudflare Sandboxes](https://developers.cloudflare.com/sandbox) | Hosted (edge) | Containers | Sandboxes next to Workers, for edge-shaped apps |
| [AWS Bedrock AgentCore](https://aws.amazon.com/bedrock/agentcore/) | Managed, your AWS account | Managed sandbox | The enterprise-AWS answer, code interpreter included |
| [GKE Agent Sandbox](https://cloud.google.com/blog/products/containers-kubernetes/agentic-ai-on-kubernetes-and-gke/) | Your GKE cluster | gVisor | GA May 2026; Google's launch claims 300 sandbox starts per second |
| [Agent Sandbox](https://github.com/kubernetes-sigs/agent-sandbox) | Your Kubernetes cluster | Pod plus a runtime you choose | The open Kubernetes-native primitive: Sandbox resource, warm pools, claims |
| [Docker Sandboxes](https://www.docker.com/products/docker-sandboxes/) | Your laptop | MicroVMs (works on macOS/Windows) | Run coding agents unsupervised locally; the August 2026 launch that put the category on front pages |
| [microsandbox](https://github.com/superradcompany/microsandbox) | Your laptop or self-host | MicroVMs | Local-first and open source (Apache-2.0) |
| bubblewrap and friends | Your laptop | Process sandbox | The zero-infrastructure baseline; better than nothing, weakest wall |

Newer hosted entrants (Blaxel, Runloop, Morph, Beam) compete mostly on idle pricing and cold starts; they fill the ["E2B alternatives" lists](https://www.beam.cloud/blog/best-e2b-alternatives) worth scanning before you commit.

## Two neighbors that are not sandboxes

[smolagents](https://github.com/huggingface/smolagents) (28.7k stars) is not a sandbox: it is an agent library whose actions are Python code, and it plugs *into* E2B, Modal, Docker, or Blaxel to run that code. Its real rivals are agent frameworks like LangGraph and CrewAI. [OpenHands](https://github.com/OpenHands/OpenHands) is a full coding agent with the Docker sandbox already wired, for when assembling agent plus sandbox yourself is the part you don't want.

## Pick by situation

- **You're building a product whose agent writes and runs code** → **E2B**, or **Modal** if that code needs GPUs. Hosted, per-second billing, strong isolation.
- **You're a platform team and the code must stay on your infrastructure** → **Agent Sandbox** on any Kubernetes, **GKE Agent Sandbox** on Google, or **AgentCore** on AWS.
- **You want to run Claude Code or Codex unsupervised on your own machine** → **Docker Sandboxes**, or **microsandbox** if you want it open source.
- **You were evaluating Daytona** → read the [repo banner](https://github.com/daytonaio/daytona#readme) first. Core development moved to a private codebase in June 2026 and the public repo gets no further updates. The 72k stars measure what it was. Choose it today only as the hosted product, or fork v0.190.0 and accept AGPL plus sole maintenance.
- **Watch the bill shape**, not just the price: E2B and Daytona bill the full time a sandbox is alive; Vercel bills active CPU only. An agent that thinks for minutes between commands pays very differently on each.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# Terminal coding agents: opencode vs Codex vs Gemini CLI vs crush vs goose

The most-asked pick in this list: *"I want a turnkey coding agent in my terminal today."* A terminal coding agent is a program you run in your shell that takes a plain-language request, then works in a loop: the model proposes an action (edit this file, run this command, search the repo), the tool executes it and shows the model the result, and the loop continues until the job is done. That loop, plus the provider wiring, the sandboxing, and the extension model around it, is the **harness**, and it is what actually differs between these five; the chat-in-a-terminal experience on top has converged. The closed first-party products (Claude Code, Cursor's agent) aren't list entries, but they define the workflow all five implement.

Why the pick matters: these tools edit your files and run commands on your machine, so you are choosing a default safety posture and a provider relationship, not just an interface.

| | [opencode](https://github.com/anomalyco/opencode) | [Gemini CLI](https://github.com/google-gemini/gemini-cli) | [Codex](https://github.com/openai/codex) | [goose](https://github.com/aaif-goose/goose) | [crush](https://github.com/charmbracelet/crush) |
|---|---|---|---|---|---|
| ⭐ Stars | 209k | 107k | 125k | 54.5k | 28.2k |
| Steward | Anomaly (company, formerly SST) | Google (first-party) | OpenAI (first-party) | Linux Foundation AAIF (Agentic AI Foundation) | Charm |
| License | MIT | Apache-2.0 | Apache-2.0 | Apache-2.0 | ⚠️ FSL-1.1-MIT (Functional Source License; each release converts to MIT after two years) |
| Core language | TypeScript | TypeScript | Rust | Rust | Go |
| Model lock-in | None: 75+ providers, including local models | Gemini-first | OpenAI-first, other providers configurable | None: provider choice by design | None: multi-provider |
| Autonomy (list axis) | headless | bounded | bounded | headless | bounded |
| Recovery (list axis) | resumable | resumable | resumable | resumable | resumable |
| Distinctive harness bet | Client/server split: the agent runs as a server your terminal, IDE, or another machine connects to | First-party Gemini integration and a generous free tier | Sandboxed execution as the default posture | Extensions and recipes over a fixed experience; desktop app and CLI both first-party | Session persistence and terminal-interface polish |

_Stars as captured for the main list. The list-axis rows: autonomy is how unattended the tool is designed to run (step-gated → headless; "headless" means built for unsupervised runs, not that it lacks an interface), recovery is what survives a dead session (none → durable). Definitions: [guide to rankings](../README.md#guide-to-rankings)._

## Pick by situation

- **You want maximum freedom and the biggest community** → **opencode**. No model lock-in ([75+ providers](https://opencode.ai/docs/providers/), local models included), the largest star and contributor count in the category, and a [client/server design](https://opencode.ai/docs/server/) that lets the agent run somewhere other than the terminal displaying it: your laptop's terminal today, a remote box or IDE tomorrow. The default pick in this list's [use-case index](../README.md#pick-by-use-case).
- **You live on Gemini, or want the free tier** → **Gemini CLI**. Google's [personal-account tier](https://github.com/google-gemini/gemini-cli#readme) (60 requests/minute, 1,000/day) is the most generous first-party allowance in the category; the reason to choose it is Gemini integration and Google's pace of investment, not harness novelty.
- **You care most about safe autonomous execution** → **Codex**. Its defining bet is running the agent's actions inside a sandbox by default ([sandboxing docs](https://developers.openai.com/codex/concepts/sandboxing)): the strongest default isolation of the five. Best experience on OpenAI models and plans.
- **You're embedding an agent into your own tooling** → **goose**. Foundation-governed, built around extensions using MCP and ACP (the Model Context Protocol and Agent Client Protocol, the standard plugs for tools and for editor-agent connections) plus reusable task recipes. It ships both a desktop app and a CLI; the reason to pick it is the extension architecture, not the absence of one.
- **You want the nicest terminal experience, and the license is acceptable** → **crush**. Charm builds the terminal-UI tooling much of the ecosystem uses, and it shows, plus persistent per-project sessions. The FSL license is the one structural caveat: fine for individual use, check it before redistributing.

Three more names people shortlist, all on the main list: [aider](https://github.com/Aider-AI/aider) (48k stars, the original git-native terminal pair programmer, though its repo has been quiet since May 2026), [pi](https://github.com/earendil-works/pi) (the minimal multi-provider agent toolkit this list's oh-my-pi entry builds on), and [Qwen Code](https://github.com/QwenLM/qwen-code) (a Gemini CLI fork tuned for open-weight Qwen models).

## What they share

All five: the tool loop described above over file, shell, and search tools; MCP support for adding more tools; permission prompts before destructive actions; and a briefing file for custom instructions. Switching costs are low by design. All five read AGENTS.md, the shared instructions-file format (Gemini CLI defaults to its own GEMINI.md and needs one setting flipped; goose reads it in its developer extension), so trying two is cheap.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). New to this decision? Start with [How to pick a harness](how-to-pick-a-harness.md). Spot an error or a stale claim? [Open an issue](https://github.com/RyanAlberts/best-of-Agent-Harnesses/issues)._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._


---

# Why the harness matters more than the model

The same model weights score about 30% on ARC-AGI-3 as a bare model and 95.5% inside a good harness, and that gap is the number agent builders keep quoting. This page collects who says the harness matters more than the model, what each of them measured, and what the claim does not mean.

[![YC Paper Club: Why the harness matters more than the model (video, 60 minutes)](https://i.ytimg.com/vi/n9xKblqyQ28/hqdefault.jpg)](https://www.youtube.com/watch?v=n9xKblqyQ28)

_Y Combinator's Paper Club session of September 7, 2026, with the authors of Prime Agent, OpenJarvis, and QM. One hour; the history of harnesses runs from 7:00 to 17:00._

## The claim

A model answers; an agent acts. The harness is the runtime that turns one into the other ([definition](../README.md#what-is-an-agent-harness)): the tool list, the approval rules, what the model sees each turn, what survives a crash, and, since 2026, what the agent is allowed to change about itself. "The harness matters more than the model" is shorthand for a measurable fact: hold the weights fixed, change only the harness, and benchmark scores move more than most model upgrades move them.

## The measurements

- **ARC-AGI-3, same weights, 30% to 95.5%.** A bare frontier model was verified at about 30% on the private ARC-AGI-3 set. [Prime Agent](https://arxiv.org/abs/2608.23552), a self-improving harness from Prime Intellect, reached 95.5% with Opus 5, above the reported human-expert baseline. The [YC talk](https://www.youtube.com/watch?v=n9xKblqyQ28) walks through the runs at 30:00, and the host adds that an NVIDIA harness has since reported 100%.
- **SWE-bench Pro, harness-only swap, 23% to 52%.** [@joelniklaus](https://x.com/joelniklaus/status/2085725862142623875) held the model fixed and changed the harness: GLM-5.2 went from 23% to 52% pass@1, and Gemma 4 26B from 15% to 36%. Latent Space's summary: swapping the harness "changed pass@1 more than many model upgrades do" ([AINews, Aug 8 2026](https://www.latent.space/p/ainews-zawinskis-law-of-multiagents)).
- **Cursor's benchmark, 46% to 80%.** [Cursor's benchmarking research](https://www.mindstudio.ai/blog/agent-harness-scaffolding-matters-more-than-model) found the same model passing 46% of tasks in one harness and 80% in another.
- **Rankings do not transfer.** [Harness-Bench](https://arxiv.org/abs/2605.27922) measured a rank correlation of about -0.05 between harness rankings on different models. The best harness for one model says almost nothing about the best harness for another.
- **Harness search beats hand design.** [Meta-Harness](https://arxiv.org/abs/2603.28052), from Stanford's IRIS lab, searched over harness code end to end and beat a state-of-the-art context manager by 7.7 points while using four times fewer context tokens.
- **Two harnesses, identical weights, 18 points apart.** The YC host's opening slide (1:00) shows an 18% gap between two harnesses on the same weights, the number he uses to answer the claim that harness work is not research.

The chart that opens the [decision guide](how-to-pick-a-harness.md#the-chart-to-internalize-first) plots the coding-benchmark spreads.

## Who says so

| Who | Role | What they say | Evidence | Source | Date |
|---|---|---|---|---|---|
| Y Combinator Paper Club | talk | Harnesses were "belittled as subpar research"; the numbers say otherwise | measurement, practitioner | [video](https://www.youtube.com/watch?v=n9xKblqyQ28) | 2026-09 |
| Prime Intellect (Seth Karten et al.) | paper | Prime Agent: 95.5% on ARC-AGI-3 with Opus 5 through a recursive-language-model harness | measurement | [arXiv:2608.23552](https://arxiv.org/abs/2608.23552), [blog](https://www.primeintellect.ai/blog/prime-agent) | 2026-08 |
| Karten et al., Continual Harness | paper | The agent edits its own prompt, skills, memory, and sub-agents mid-episode | paper | [arXiv:2605.09998](https://arxiv.org/abs/2605.09998) | 2026-05 |
| Stanford IRIS (Lee, Finn et al.), Meta-Harness | paper | End-to-end search over harness code: +7.7 points, four times fewer context tokens | measurement | [arXiv:2603.28052](https://arxiv.org/abs/2603.28052) | 2026-03 |
| Harness-Bench | paper | Harness rankings barely transfer across models (rank correlation about -0.05) | measurement | [arXiv:2605.27922](https://arxiv.org/abs/2605.27922) | 2026-05 |
| @joelniklaus | practitioner analysis | SWE-bench Pro: 23% to 52% and 15% to 36% from the harness alone | measurement | [x.com](https://x.com/joelniklaus/status/2085725862142623875) | 2026-08 |
| Cursor (via MindStudio) | vendor benchmark | Same model, 46% in one harness, 80% in another | measurement | [write-up](https://www.mindstudio.ai/blog/agent-harness-scaffolding-matters-more-than-model) | 2026 |
| Endor Labs | practitioner report | Claude Fable 5: same model, different harness, very different result | measurement | [post](https://www.endorlabs.com/learn/claude-fable-5-take-two-same-model-different-harness-and-a-very-different-result) | 2026-06 |
| SWE-agent (Princeton) | paper | Coined the agent-computer interface: how tools are presented changes what a model can do | paper | [arXiv:2405.15793](https://arxiv.org/abs/2405.15793) | 2024-05 |
| Anthropic | lab guidance | Effective harnesses for long-running agents; build simple before you build frameworks | lab guidance | [long-running agents](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents), [building effective agents](https://www.anthropic.com/engineering/building-effective-agents) | 2024-2026 |
| OpenAI | lab guidance | "Harness engineering": environment design, feedback loops, the repo as system of record | lab guidance | [post](https://openai.com/index/harness-engineering/) | 2026 |
| LangChain (Harrison Chase, Vivek Trivedy) | lab guidance | Better Harness: hill-climb the harness with evals as the training signal | lab guidance | [post](https://blog.langchain.com/better-harness-a-recipe-for-harness-hill-climbing-with-evals/), [Chase](https://x.com/hwchase17/status/2041929684741747171) | 2026-04 |
| Andrej Karpathy | practitioner | The model is "the kernel process of a new Operating System"; the harness is the rest of the OS | practitioner | [x.com](https://x.com/karpathy/status/1707437820045062561) | 2023-09 |
| Karpathy's autoresearch, field reports | practitioner | Overnight research loops lived or died on harness affordances, not raw model quality | practitioner | [repo](https://github.com/karpathy/autoresearch), [AINews](https://www.latent.space/p/ainews-autoresearch-sparks-of-recursive) | 2026-03 |
| Simon Willison | practitioner | "An LLM agent runs tools in a loop to achieve a goal"; the harness is everything around the loop | definition | [post](https://simonwillison.net/2025/Sep/18/agents/) | 2025-09 |
| swyx, Latent Space | practitioner | Asked "Is Harness Engineering real?", then declared "Meta-Harness Summer" | practitioner | [AINews](https://www.latent.space/p/ainews-is-harness-engineering-real) | 2026 |
| Jerry Liu (LlamaIndex) | practitioner | The framework era is over; what matters now is skills, tools, and context quality | practitioner | [VentureBeat](https://venturebeat.com/infrastructure/the-ai-scaffolding-layer-is-collapsing-llamaindexs-ceo-explains-what-survives) | 2026 |
| Aakash Gupta | commentary | "2025 was agents, 2026 is agent harnesses" | commentary | [Medium](https://aakashgupta.medium.com/2025-was-agents-2026-is-agent-harnesses-heres-why-that-changes-everything-073e9877655e) | 2026 |
| Awesome-Agent-Harness survey | survey | 110+ papers and 23 systems organized into a harness taxonomy | paper | [GitHub](https://github.com/Gloriaameng/Awesome-Agent-Harness) | 2026 |
| The 2026 explainer wave | commentary | "The harness matters more than the model" as a blog genre | commentary | [Medium](https://medium.com/@Micheal-Lanham/building-agents-in-2026-why-the-harness-matters-more-than-the-model-eb72448fee04), [dev.to](https://dev.to/max_quimby/harness-engineering-the-developer-skill-that-matters-more-than-your-ai-model-in-2026-47ke), [Tencent Cloud](https://www.tencentcloud.com/techpedia/147786?lang=en), [Infralovers](https://www.infralovers.com/blog/2026-03-13-harness-engineering-rahmen-wichtiger-als-modell/) | 2026 |

Measurement rows are the evidence; commentary rows show how far the claim has spread. Every link was checked on 2026-09-14. Missing an authority? Open an issue with the primary source.

## What the claim does not mean

The YC talk adds the nuance the slogan drops. Timestamps point into [the video](https://www.youtube.com/watch?v=n9xKblqyQ28).

1. **Two eras, and the gains are in the second one.** The static harness (7:00 to 13:00) ran from GPT-2's sampling loop in 2019 through few-shot prompts, chain of thought, tools, editable memory, skills, reflection, sub-agents, and recursive language models: a fixed program around a fixed model. The self-improving harness (14:00 to 17:00) lets the agent change its own prompt ([DSPy](https://github.com/stanfordnlp/dspy), [GEPA](https://arxiv.org/abs/2507.19457)), its own code ([Darwin Gödel Machine](https://arxiv.org/abs/2505.22954)), or its whole harness state ([Continual Harness](https://arxiv.org/abs/2605.09998), [Meta-Harness](https://arxiv.org/abs/2603.28052)). The recent jumps come from the second era.
2. **More harness is not the claim. The right harness is.** YC's QM runs on three core tools and calls itself an "AGI-anticipating harness" (57:00). Prime Agent's rule (25:00) is to expose what the model cannot do for itself, such as compaction, a persistent REPL, and programmatic sub-agents, and to drop imposed procedure like fixed plan-act-critique loops, which models now run on their own.
3. **The harness is a cost lever, not only a score lever.** One popular harness spent about $5,000 on ARC-AGI-3 without progress before the run was cut off (33:00), while Prime Agent finished the set. Working on context programmatically, instead of stuffing it into the prompt, is what saved the money.
4. **Long-horizon failure is a harness problem.** Agents "give up way too early" (57:30), so QM sets wall-clock and token budgets on goals and will not let the agent quit before the budget is spent. The QM team notes that OpenAI and Anthropic used a similar technique on open math problems.
5. **Automatic self-improvement still needs a person in the loop.** QM's team tried hill-climbing on their trace set and hit "main character syndrome" (53:00): each agent fixes the piece of the elephant it can see.
6. **It is a pairing, not a winner.** Because rankings barely transfer across models, "best harness" only means something for a given model, and the choice must be re-asked whenever the model changes.

## What to do about it

- Pick the harness with the same care as the model, and re-pick when the model changes. The [six questions](how-to-pick-a-harness.md) turn this list into that decision; the [two-week test drive](how-to-test-drive-a-harness.md) checks it on your own repos.
- Treat the harness as a first-class variable in every benchmark you read or run. A score without the harness named is half a number.
- Budget long-horizon runs explicitly: wall-clock time and tokens, per goal.
- Prefer thin harnesses that expose capabilities over thick ones that impose procedure, and expect the right amount of harness to shrink as models improve.
- Let your agent choose. The [MCP server](../mcp/) in this repo exposes `recommend` and `pick_harness`, so a coding agent can pick a harness matched to its model and task instead of inheriting whichever one someone else benchmarked.

---

_Part of [best-of-Agent-Harnesses](https://github.com/RyanAlberts/best-of-Agent-Harnesses). Corrections and missing sources: open an issue with the primary link._

_Agents can query this page's data directly: `claude mcp add agent-harnesses -- uvx agent-harnesses-mcp` ([MCP server](../mcp/))._