Observability and eval-ops

Tracing, monitoring, and production evaluation for live agent runs: capture every step, tool call, and token, then score and debug in the loop. Distinct from the fixed-task benchmarks above—this is what you run against your own traffic.

#	Project	Stars	Tier	OSS	Description
1	Langfuse evalstypescript	34.9k	slightly complex	✅	Open-source LLM engineering platform: full-trace observability, online and offline evals, prompt management, and cost metrics for agent runs in production—the monitoring layer most harnesses lack out of the box.
2	MLflow evalspython	28.1k	complex	✅	Mature ML platform now covering GenAI: MLflow Tracing captures every agent step, tool call, and token, with built-in LLM evals and prompt versioning—observability for teams already standardized on MLflow.
3	Opik evalspython	22.2k	slightly complex	✅	Comet's open-source agent observability and evaluation platform: tracing, scoring, and experiment comparison with the whole core feature set free to self-host under Apache-2.0.
4	Arize Phoenix evalspython	11.5k	slightly complex	⚠️ Elastic-2.0	Arize's source-available, local-first tracing and eval layer: run it on your laptop or your own infra, and graduate to the managed Arize AX platform only when you need it.