Integration · LangGraph

LangGraph with an OpenAI-compatible API

LangGraph owns the state machine; the model is just a node. Point ChatOpenAI at an OpenAI-compatible base URL, bind tools, add a checkpointer, and cap the loop before it burns your budget.

Short answer. Use langchain_openai.ChatOpenAI(model="plugsky-plus", base_url="https://api.plugsky.com/v1", api_key=...). Bind tools, add a ToolNode, route with tools_condition, and compile with a checkpointer plus a thread_id. Set a recursion limit and a tool-call cap; a graph with tools can loop without one.

Install

pip install -U langgraph langchain-openai

Pin versions in production. LangGraph's APIs have changed across releases; a working notebook is not a deployment.

Minimal agent graph

from typing import Annotated
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, MessagesState, START
from langgraph.prebuilt import ToolNode, tools_condition
from langgraph.checkpoint.memory import MemorySaver

@tool
def get_ticket_status(ticket_id: str) -> str:
    """Return the status of a support ticket."""
    return {"T-100": "open", "T-101": "resolved"}.get(ticket_id, "unknown")

tools = [get_ticket_status]

llm = ChatOpenAI(
    model="plugsky-plus",
    base_url="https://api.plugsky.com/v1",
    api_key="YOUR_PLUGSKY_KEY",
    temperature=0,
)
llm_with_tools = llm.bind_tools(tools)

def agent(state: MessagesState):
    return {"messages": [llm_with_tools.invoke(state["messages"])]}

graph = StateGraph(MessagesState)
graph.add_node("agent", agent)
graph.add_node("tools", ToolNode(tools))
graph.add_edge(START, "agent")
graph.add_conditional_edges("agent", tools_condition)
graph.add_edge("tools", "agent")

app = graph.compile(checkpointer=MemorySaver())

Run it with a thread

config = {"configurable": {"thread_id": "ticket-123"}, "recursion_limit": 12}

result = app.invoke(
    {"messages": [{"role": "user", "content": "Is ticket T-100 still open?"}]},
    config=config,
)
print(result["messages"][-1].content)

The checkpointer stores state per thread_id. Use a durable checkpointer in production so a restart does not lose the conversation.

Streaming

for event in app.stream(
    {"messages": [{"role": "user", "content": "Summarise T-101."}]},
    config=config,
    stream_mode="updates",
):
    print(event)

stream_mode="updates" emits node updates; token streaming requires the model to stream and a token-level stream mode. Pick the mode that matches your UI; mixing them without a plan produces duplicated text.

Tool design that survives

  • Narrow schemas: one action per tool, typed arguments, no free-form JSON strings.
  • Idempotency: a retried tool call must not create a second refund or ticket.
  • Errors as data: return a structured error the model can recover from, not an exception that kills the graph.
  • Authorization: check the user's permission inside the tool, not in the prompt.
  • Timeouts: cap each tool call and route timeouts to a fallback.

Stop the loop

config = {
    "configurable": {"thread_id": "ticket-123"},
    "recursion_limit": 12,
}

A recursion limit is necessary but not sufficient. Add a per-turn tool-call cap and a failure edge to a human queue. If the model calls the same tool with the same arguments twice, stop and escalate.

Production checklist

  • Durable checkpointer (Postgres or the platform's store), not MemorySaver.
  • Prompt and model version recorded with each run.
  • Token and tool-call budgets enforced outside the prompt.
  • Retry with backoff for 429/5xx, and a fallback model or provider.
  • Trace IDs that connect the graph run to the model request and the tool call.

FAQ

Can LangGraph use an OpenAI-compatible API?

Yes. Use ChatOpenAI with base_url and api_key; LangGraph orchestrates the graph.

How do I add tools?

Bind them with bind_tools, add a ToolNode, and route with tools_condition.

How do I persist a conversation?

Compile with a checkpointer and pass a thread_id. Use a durable store in production.

How do I stream output?

Use stream/astream with the stream mode that matches your UI; token streaming requires a streaming model.

How do I stop an infinite tool loop?

Set recursion_limit, cap tool calls per turn, and add a failure edge to a human.

Reviewed 20 November 2026. LangGraph and langchain-openai APIs change; pin versions and test the graph before deploying. The code is illustrative and omits production persistence and secrets management.