Llama Stack

Llama Stack is Meta's open-source toolchain and standardized API for building AI applications and agents using Llama models. It provides a unified interface that standardizes the core building blocks of agent development — inference, safety, memory, tool use, and evaluation — into a consistent API that works across different deployment environments from local development to cloud production.

The stack is designed around a distribution model where different providers implement the standardized APIs. A local development distribution might use Ollama for inference and ChromaDB for memory, while a production distribution could use AWS Bedrock for inference and PostgreSQL for persistence. The API remains the same, making it easy to develop locally and deploy to production without code changes.

Llama Stack includes built-in safety features through Llama Guard, Meta's content safety model that provides input and output filtering for agent interactions. This is integrated at the API level, so safety checks happen automatically without additional integration work. The safety system covers categories including violence, sexual content, criminal planning, and more.

The Agents API provides a complete framework for building tool-using agents with support for function calling, code execution, web search, and custom tools. The memory API supports both vector-based retrieval (for RAG) and conversation history management. An evaluation API enables testing agent performance with standardized benchmarks.

Llama Stack supports multiple client languages including Python and TypeScript, and provides REST APIs for language-agnostic integration. Distributions are available for local development (with Ollama), cloud deployment (with AWS, Azure, Fireworks, Together), and on-device inference. The project represents Meta's effort to create a standardized, portable agent development stack around the Llama model family.