Laminar

Laminar (lmnr) is an open-source observability platform purpose-built for AI agents and LLM applications. It provides comprehensive tracing, evaluation, and analytics capabilities that help developers understand, debug, and improve their agent systems in development and production.

The platform captures detailed traces of every agent execution — including LLM calls, tool invocations, retrieval operations, and custom spans — with automatic instrumentation for popular frameworks like LangChain, LlamaIndex, CrewAI, and OpenAI. Each trace includes input/output data, token counts, latency measurements, and cost calculations, giving developers full visibility into what their agents are doing and how much it costs.

Laminar's evaluation system lets developers define custom evaluation functions and run them against traces or datasets. Evaluations can be LLM-as-judge assessments, deterministic checks, or custom Python functions. Results are tracked over time, enabling teams to measure quality trends and catch regressions before they reach users.

The dataset management feature allows teams to curate collections of inputs and expected outputs from production traces, creating golden datasets for testing and evaluation. This production-to-test feedback loop is critical for systematically improving agent quality.

Laminar can be self-hosted via Docker or used as a managed cloud service. The open-source version includes all core features — tracing, evaluation, datasets, and the analytics dashboard. The managed version adds team collaboration, higher retention, and support.

The platform integrates via a lightweight SDK (Python and TypeScript) that adds minimal overhead to agent execution. Auto-instrumentation means most frameworks work out of the box with just an import statement.

For teams building production agent systems, Laminar fills a critical gap between generic observability tools (which don't understand LLM-specific metrics) and framework-specific tools (which lock you into one ecosystem). Its open-source nature, broad framework support, and focus on the development-to-production lifecycle make it a strong choice for teams that want observability without vendor lock-in.