TL;DR — LangGraph is the low-level runtime underneath LangChain's create_agent(). Where LangChain gives you a one-liner agent, LangGraph lets you define the agent as an explicit graph — nodes are functions, edges are conditional transitions, state is typed and checkpointed. Use it when you need branching, parallelism, human-in-the-loop gates, or multi-agent coordination that a linear loop can't express.
What it is
LangGraph is a framework for building stateful, multi-step AI applications as directed graphs. Each node is a Python function (or coroutine); edges carry typed state between them; conditional edges let the model's output decide where to go next. The runtime handles persistence, streaming, fault tolerance, and time-travel debugging.
It sits in AI Agent › Agent Framework — the layer where you design the control flow of an agent, deciding exactly which steps happen, in what order, and under what conditions.
Fig 1 — A basic ReAct agent as a LangGraph: call model → conditional branch → tools or respond.
Why it exists
Simple agents follow a loop: call model → run tools → repeat. But real-world agents need more: parallel tool execution, human approval before dangerous actions, sub-agents that coordinate, error recovery with fallback paths. A flat loop can't express these. LangGraph gives you a graph so every branch, gate, and retry is an explicit, visible node — not hidden inside callbacks.
Core concepts
- State — a typed dict (usually a
TypedDictor Pydantic model) that flows between nodes. The graph's "memory" for the current run. - Node — a function
(state) → partial state update. Does one thing: calls a model, runs tools, formats output. - Edge — connects nodes. Normal edges always fire; conditional edges call a router function to pick the next node.
- Checkpointer — persists state after each node, enabling conversation memory, time-travel, and human-in-the-loop interrupts.
Install & setup
pip install langgraph langchain-openai
export OPENAI_API_KEY=sk-...Building a basic agent
from typing import Annotated, TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langchain_openai import ChatOpenAI
from langchain.tools import tool
# 1. Define state
class State(TypedDict):
messages: Annotated[list, add_messages]
# 2. Define tools
@tool
def search(query: str) -> str:
"""Search the web."""
return f"Results for: {query}"
# 3. Define nodes
model = ChatOpenAI(model="gpt-4o").bind_tools([search])
def call_model(state: State):
response = model.invoke(state["messages"])
return {"messages": [response]}
def run_tools(state: State):
tool_map = {"search": search}
last = state["messages"][-1]
results = []
for tc in last.tool_calls:
result = tool_map[tc["name"]].invoke(tc["args"])
results.append({"role": "tool", "content": result,
"tool_call_id": tc["id"]})
return {"messages": results}
# 4. Build graph
def should_continue(state: State):
if state["messages"][-1].tool_calls:
return "tools"
return END
graph = StateGraph(State)
graph.add_node("model", call_model)
graph.add_node("tools", run_tools)
graph.add_edge(START, "model")
graph.add_conditional_edges("model", should_continue, {"tools": "tools", END: END})
graph.add_edge("tools", "model")
agent = graph.compile()
result = agent.invoke({"messages": [("user", "Search for vLLM news")]})Persistence & memory
Add a checkpointer and every node's output is saved. Pass a thread_id to maintain conversation history across calls:
from langgraph.checkpoint.memory import MemorySaver
agent = graph.compile(checkpointer=MemorySaver())
config = {"configurable": {"thread_id": "user-42"}}
agent.invoke({"messages": [("user", "My name is Shivam")]}, config=config)
r = agent.invoke({"messages": [("user", "What's my name?")]}, config=config)
# -> "Your name is Shivam"Human-in-the-loop
Interrupt the graph before a dangerous node. The runtime pauses, waits for approval, then resumes from the checkpoint:
agent = graph.compile(
checkpointer=MemorySaver(),
interrupt_before=["tools"] # pause before running tools
)
result = agent.invoke({"messages": [("user", "Delete old records")]}, config=config)
# graph pauses — inspect result, approve, then:
agent.invoke(None, config=config) # resume from checkpointSubgraphs & multi-agent
A node can itself be a compiled graph. This is how you build multi-agent systems — each sub-agent is a subgraph with its own state, and a parent graph coordinates them:
researcher = build_researcher_graph().compile()
writer = build_writer_graph().compile()
parent = StateGraph(ParentState)
parent.add_node("research", researcher)
parent.add_node("write", writer)
parent.add_edge(START, "research")
parent.add_edge("research", "write")
parent.add_edge("write", END)
app = parent.compile()Streaming
for event in agent.stream({"messages": [("user", "Search AI news")]},
stream_mode="updates"):
for node_name, update in event.items():
print(f"[{node_name}]", update)LangGraph Platform
For production deployment, LangGraph Platform (formerly LangGraph Cloud) gives you a managed runtime with built-in task queues, cron jobs, a REST API, and LangSmith integration. You deploy your graph; it handles scaling, persistence, and long-running background agents. Self-hosted option available.
When to use, when to skip
Use it when your agent needs branching logic, parallel paths, human gates, sub-agents, or any control flow that a linear tool-calling loop can't express. Also when you want full visibility into exactly what path the agent took.
Skip it for simple agents — create_agent() from LangChain wraps LangGraph and is simpler. If you don't need LangChain's ecosystem at all, consider lighter alternatives like Pydantic AI or raw provider SDKs.
vs the alternatives
| Tool | Best for | Trade-off |
|---|---|---|
| LangGraph | Custom graphs, branching, multi-agent, max control | More wiring; steeper learning curve |
LangChain create_agent | Simple tool-calling agents, quick start | Less control over flow |
| AutoGen | Multi-agent conversations, research | Different paradigm (chat-based) |
| CrewAI | Role-based agent teams | Opinionated, less low-level |
| Temporal / Hatchet | Durable workflow orchestration | Not AI-specific |
Verified against the LangGraph docs (langchain-ai.github.io/langgraph), May 2026.
Where LangGraph fits: the mental model
LangGraph is an application-layer component that turns model calls into controlled, multi-step software behavior. The useful question is not simply “can it run the demo?” It is whether the component gives your team a clear ownership boundary, predictable failure behavior, and enough evidence to operate changes safely. Treat it as one replaceable layer in a larger system rather than letting it quietly become the architecture.
Start by drawing the request and data path. Mark where untrusted input enters, where identity is checked, where durable state changes, and where retries can repeat work. That diagram tells you which guarantees belong to LangGraph and which still belong to your application, platform, cloud provider, or database. The distinction matters during incidents: a healthy process is not proof that the end-to-end task is correct.
Core concepts you should understand first
The vocabulary below is more important than any single SDK method. It lets application engineers, platform engineers, security reviewers, and incident responders describe the same system without confusing a framework feature with an end-to-end guarantee.
| Concept | Meaning in this layer | Design question |
|---|---|---|
| Control loop | The repeated plan, call, observe, and decide cycle. Bound it with explicit stop conditions and budgets. | Write down how LangGraph represents or enforces this before production. |
| Tool contract | A typed name, description, input schema, output schema, timeout, and error model exposed to the model. | Write down how LangGraph represents or enforces this before production. |
| State | Data required between steps or turns. Separate durable business state from disposable prompt context. | Write down how LangGraph represents or enforces this before production. |
| Context window | The finite model input assembled for a step. Retrieval and summarization are policies, not infinite memory. | Write down how LangGraph represents or enforces this before production. |
| Determinism boundary | Code should own authorization, money movement, deletion, and invariants; the model may propose actions. | Write down how LangGraph represents or enforces this before production. |
| Checkpoint | A recoverable snapshot used to resume long-running or human-approved work without replaying side effects. | Write down how LangGraph represents or enforces this before production. |
From quick start to a production deployment
The earlier quick start proves that the package or service runs. Production readiness is a different exercise. Build the smallest vertical slice that crosses every real boundary—identity, network, persistence, upstream provider, telemetry, and rollback—before broadening the feature set.
- Pin the compatibility envelope. Record the LangGraph release, language/runtime version, client SDK version, model or backend version, and—where applicable—Kubernetes API or driver requirements. Use a lock file, immutable image digest, or chart version; floating “latest” tags prevent repeatable rollback.
- Define contracts before configuration. Write the accepted input, successful output, error classes, timeout, idempotency behavior, and ownership of durable state. Validate at the boundary so corrupt work fails early instead of surfacing deep in a workflow.
- Create separate development, staging, and production identities. Do not copy a broad personal API key into every environment. Prefer workload identity or short-lived credentials, scope access by tenant and operation, and verify denial cases as part of deployment.
- Add bounded failure behavior. Every remote call needs a deadline. Retry only transient, idempotent operations with exponential backoff and jitter. Set concurrency and queue limits so an upstream slowdown becomes controlled backpressure rather than resource exhaustion.
- Instrument the complete path. Emit a correlation ID, component and release version, duration, outcome, retry count, and resource or cost dimensions. Keep sensitive prompt, document, and credential values out of ordinary logs.
- Ship through a reversible rollout. Run compatibility and regression tests, deploy to a canary or isolated workload, compare service-level indicators, then increase exposure. Preserve the previous artifact and configuration until rollback has been exercised.
Production configuration checklist
- Pin artifacts by version and, where possible, digest.
- Set connect, request, and total workflow deadlines.
- Bound retries, concurrency, queue length, and payload size.
- Separate read-only operations from mutations.
- Use idempotency keys for replayable mutations.
- Persist canonical state outside disposable workers.
- Encrypt traffic and durable data with managed keys.
- Redact secrets, tokens, prompts, and personal data.
- Apply per-tenant quotas and authorization filters.
- Expose readiness separately from process liveness.
- Back up metadata and test restore, not only backup.
- Document owner, escalation path, RPO, and RTO.
Failure modes and the response you should design
| Failure mode | What you observe | Engineering response |
|---|---|---|
| Unbounded loop | The agent keeps revising or calling tools. | Set maximum steps, token/cost budgets, and a terminal failure state. |
| Duplicate side effect | A retry repeats an email, charge, or write. | Give mutations idempotency keys and persist completion before retrying. |
| Prompt injection | Retrieved or web content instructs the agent to cross a trust boundary. | Treat content as data, allow-list tools, and re-authorize every sensitive action. |
| Context drift | Summaries omit a requirement or stale state wins. | Keep canonical state outside the prompt and rebuild context from versioned records. |
| Provider degradation | Rate limits or model errors stall the workflow. | Use bounded exponential backoff, circuit breakers, and an explicitly tested fallback. |
| Schema mismatch | The model emits arguments a tool cannot accept. | Validate at the boundary and return a small, machine-readable repair error. |
Turn these rows into runbook entries with an alert, first diagnostic query, safe mitigation, and escalation owner. Test at least one failure in staging every release cycle. If the system cannot be forced into a failure safely, it is usually not yet observable or isolated enough.
Security, privacy, and tenant isolation
Place LangGraph in a threat model, not just an architecture diagram. Identify human users, workload identities, administrators, upstream services, model providers, artifact registries, and data stores. For each edge, document authentication, authorization, encryption, audit evidence, and the consequence of credential compromise.
Apply least privilege at the operation and resource level. A component that only retrieves documents should not be able to delete the index; an evaluation worker should not inherit production mutation credentials; a model-serving pod should not need cluster-admin. In multi-tenant systems, enforce the tenant boundary before retrieval or execution and include tenant identity in quotas and audit events. Never rely on a prompt instruction, namespace string supplied by the client, or UI filtering as authorization.
Decide what data is permitted in telemetry. Prompts, retrieved chunks, tool arguments, model responses, notebooks, and traces can contain secrets or regulated data. Redact close to collection, keep high-sensitivity payload capture opt-in, encrypt exports, restrict support access, and give each class an explicit retention period. Verify deletion across caches, replicas, indexes, backups, and derived evaluation datasets.
Observability and service-level objectives
A useful dashboard follows the user-visible unit of work and then decomposes it by component, release, tenant tier, backend, and failure class. Start with these signals for LangGraph:
- task success rate — graph both rate and distribution, then compare with the previous release and traffic mix.
- steps per successful task — graph both rate and distribution, then compare with the previous release and traffic mix.
- tool-call error rate — graph both rate and distribution, then compare with the previous release and traffic mix.
- model tokens and cost per task — graph both rate and distribution, then compare with the previous release and traffic mix.
- p50/p95/p99 end-to-end latency — graph both rate and distribution, then compare with the previous release and traffic mix.
- human escalation and override rate — graph both rate and distribution, then compare with the previous release and traffic mix.
Choose an SLO at the boundary your users experience, such as “99% of accepted tasks complete correctly within five minutes over 28 days.” Availability alone is insufficient for AI systems because a fast but incorrect or ungrounded result is still a failure. Pair latency and completion objectives with a reviewed quality or policy indicator. Page on rapid error-budget burn; use tickets for slow capacity trends.
Testing and release strategy
Use four layers. Unit tests cover deterministic adapters, schemas, policy, and error mapping without a live external service. Contract tests exercise the pinned integration boundary—API, CLI, SDK, protocol, or ephemeral service—and verify its exact surface. Scenario tests exercise representative end-to-end cases, including permissions and state. Load and resilience tests establish saturation, queue behavior, retry amplification, and recovery after dependency loss.
Keep a small blocking suite for every commit and a broader scheduled suite for expensive or probabilistic checks. Store results with the application version, LangGraph version, configuration hash, model/backend version, dataset version, and random seed. A score without that provenance cannot explain a regression. Before upgrading, read the migration notes, run both versions against the same replay set, and explicitly test rollback across any schema or state transition.
How to decide whether LangGraph is the right tool
| Question | Evidence to collect | Red flag |
|---|---|---|
| Does it remove a real constraint? | A measured bottleneck, missing guarantee, or repeated custom component. | Adoption is based only on a demo or feature count. |
| Can the team operate it? | Named owner, upgrade path, alerts, runbooks, backup, restore, and on-call skills. | Only the original prototype author understands failure behavior. |
| Is the interface portable? | Your domain contracts wrap vendor-specific APIs; data and state have an export path. | Business objects are inseparable from framework internals. |
| Does it meet the envelope? | Benchmarks using your payloads, concurrency, topology, quality bar, and cost model. | Published benchmark hardware or workload does not resemble production. |
| Is failure affordable? | Tested degraded mode, bounded blast radius, rollback, RPO, and RTO. | A component outage blocks unrelated tenants or irreversible actions. |
Prefer the smallest component that satisfies the required guarantees. A provider SDK, relational table, background job, or standard Kubernetes controller is often better than another platform when the workload is small and predictable. Choose LangGraph when its specific abstraction removes sustained engineering work and the team is willing to own its lifecycle.
A focused 90-minute validation lab
- Minutes 0–15: run the documented quick start in a disposable environment with pinned dependencies. Save the exact commands and a known-good input/output fixture.
- Minutes 15–35: replace the toy input with one representative case from your system. Add schema validation, a deadline, and a correlation ID.
- Minutes 35–55: force invalid credentials, a timeout, malformed input, and one dependency failure. Record the observed errors and whether retries are safe.
- Minutes 55–75: run a small concurrency test and capture latency, throughput, saturation, and unit cost. Do not extrapolate beyond the tested range.
- Minutes 75–90: write the adoption decision: required guarantees met, open risks, owner, next experiment, and the simplest credible alternative.
Frequently asked questions
Should we standardize on LangGraph for every team?
Standardize the contracts, telemetry, security controls, and release evidence first. Standardizing one implementation is useful only when workloads share requirements and a platform team owns upgrades and support.
Can we use the hosted version and skip operations work?
Hosted service removes part of the control-plane burden, not architecture ownership. You still own identity, tenant isolation, data classification, quotas, dependency failure, observability, export, and an exit plan.
What should be pinned for reproducibility?
Pin the tool/server, client SDK, runtime, configuration, model or backend, container image digest, and test dataset. Record these values with every benchmark and evaluation result.
When is a proof of concept ready for production?
After representative success and failure tests pass, sensitive data paths are approved, limits and SLOs are defined, telemetry and runbooks exist, restore or rollback is rehearsed, and an accountable owner accepts the remaining risk.
Official sources and freshness
This guide was reviewed for architecture and operational guidance on 10 July 2026. Projects evolve quickly: verify installation syntax, supported versions, feature maturity, and upgrade notes against the exact release you deploy.