How do you use Plugsky with LangChain?

Key facts

Plugsky endpoint	POST https://api.plugsky.com/v1/chat/completions
LangChain class	ChatOpenAI(base_url="https://api.plugsky.com/v1", model="plugsky-pro")
Embeddings	OpenAIEmbeddings with model plugsky-embed
Structured output	llm.with_structured_output(PydanticModel) uses JSON mode or tool calling
Streaming	chain.stream() drives SSE streaming under the hood
Models	30+ models; route cheap steps to plugsky-lite and hard ones to plugsky-pro
Compatibility	Any LangChain OpenAI-compatible component works with a base_url override
Product status	Live; assistants and responses endpoints are coming soon, so keep agents on chat completions

TL;DR

• Change one parameter: base_url in ChatOpenAI.
• LCEL, streaming, tools and structured output keep working.
• OpenAIEmbeddings points at plugsky-embed for RAG.
• Prefer explicit parameters over global OPENAI_* env vars.
• Agents run on chat completions plus tools today.

How it works, step by step

1. Install langchain, langchain-openai and export PLUGSKY_API_KEY.
2. Create ChatOpenAI with the Plugsky base URL and a model id such as plugsky-pro.
3. Run llm.invoke() and confirm the response before building chains.
4. Add an LCEL chain and stream results with chain.stream().
5. Define Pydantic models and use with_structured_output for extraction tasks.
6. Create OpenAIEmbeddings with plugsky-embed and build a retriever for RAG.
7. Bound retries, timeouts and model selection per chain step.

Original data

Try it yourself

Open the RAG sandbox →

Point ChatOpenAI at Plugsky

Install the OpenAI integration package and pass the base URL explicitly:

pip install langchain langchain-openai

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="plugsky-pro",
    api_key="sk-live-…",
    base_url="https://api.plugsky.com/v1",
    temperature=0,
    timeout=60,
    max_retries=2,
)

print(llm.invoke("Explain HNSW in two sentences").content)

LangChain also honors OPENAI_API_KEY and OPENAI_BASE_URL environment variables, but those apply to every OpenAI client in the process. Passing explicit arguments avoids surprising another integration later.

LCEL chains and streaming

Because the model is a normal LangChain runnable, LCEL composition is untouched:

from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a terse release-notes editor."),
    ("human", "{changelog}"),
])

chain = prompt | llm | StrOutputParser()

for token in chain.stream({"changelog": raw_changelog}):
    print(token, end="", flush=True)

Streaming uses the OpenAI SSE format under the hood. Swap llm for a second model instance bound to plugsky-lite to run the same chain cheaper on high-volume traffic.

Tools and structured output

Tool binding and structured output are the two features most agent code depends on. Both work through the OpenAI-compatible schema. Structured extraction:

from pydantic import BaseModel, Field

class Ticket(BaseModel):
    title: str = Field(description="Short summary")
    priority: str = Field(description="low, medium or high")

structured = llm.with_structured_output(Ticket)
ticket = structured.invoke("Production API is returning 500s for all users")
print(ticket.priority)

For explicit tool control, use llm.bind_tools([...]) and inspect tool_calls in the response. When the model asks for a tool, execute it, append the result as a tool message and invoke again. This is the same loop as raw OpenAI function calling, with LangChain handling serialization. Pydantic validation failures should trigger one retry with the error included, then a fallback path.

Embeddings and RAG

Embeddings use the same base URL and key:

from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    model="plugsky-embed",
    api_key="sk-live-…",
    base_url="https://api.plugsky.com/v1",
)

Wire them into any LangChain vector store, index your documents, then use the retriever as a chain step. Two practical notes: rebuild the index when you change embedding providers, because vector spaces are not comparable; and keep chunk sizes stable during evaluation so a relevance change is attributable to the model or prompt, not the preprocessing.

Production tips

Set timeout and max_retries on every model instance, and remember that LangChain retries are separate from the SDK's own retries. Log response_metadata from each call: it carries the model that actually answered, which matters when you route requests across several Plugsky models. For multi-step agents, prefer LangGraph-style explicit state over an unbounded ReAct loop so you can cap steps and inspect intermediate state.

The assistants and responses endpoints are coming soon on Plugsky, so keep agent state in your own store and drive the loop through chat completions today. A flat self-serve plan plus per-step model selection is the simplest way to keep agent economics predictable.

Honest comparison

Capability	Plugsky + LangChain	LangChain with OpenAI only	Custom orchestration
Model access	30+ models via one base_url	OpenAI catalogue	Whatever you integrate
Agents	Tool loop on chat completions	Native plus Responses API	You build the loop
Structured output	with_structured_output on supported models	Native	Manual parsing
Embeddings	plugsky-embed with the same key	OpenAI embeddings	Run your own model
Cost control	Flat self-serve plans	Per-token	Your own metering
Ops	Managed API	Managed API	You operate everything

Frequently asked questions

Which LangChain class calls Plugsky?

ChatOpenAI from langchain-openai. Pass base_url="https://api.plugsky.com/v1", your model id and your Plugsky key.

Do chains and streaming still work?

Yes. Plugsky is OpenAI-compatible, so LCEL chains, streaming, async invocation and callbacks behave exactly as they do with OpenAI.

How do I use structured output?

Call llm.with_structured_output(PydanticModel). LangChain maps it to JSON mode or tool calling, and validation happens on the Pydantic model.

Can I use tools and agents?

Yes. bind_tools() and LangChain agent constructors work through the standard function-calling schema. Keep loop caps for production.

How do I add embeddings for RAG?

Create OpenAIEmbeddings with model plugsky-embed, the Plugsky base URL and your key, then use any LangChain vector store.

Should I set OPENAI_BASE_URL?

It works, but it affects every OpenAI client in the process. Prefer explicit base_url arguments on each model instance.

Which model should each chain step use?

Use plugsky-lite or plugsky-micro for extraction and routing, and plugsky-pro or plugsky-frontier for reasoning and final answers.

What about the Responses API?

It is coming soon on Plugsky. LangChain's chat model path targets /v1/chat/completions, which is live today with tools, streaming and JSON mode.