How do you use Plugsky with LlamaIndex?

Key facts

Plugsky endpoint	POST https://api.plugsky.com/v1/chat/completions
LlamaIndex LLM	OpenAI(model="plugsky-pro", api_base="https://api.plugsky.com/v1")
Embeddings	OpenAIEmbedding(model="plugsky-embed", api_base=…)
Index types	VectorStoreIndex, SummaryIndex and query engines work unchanged
Streaming	as_query_engine(streaming=True) iterates tokens over SSE
Models	30+ models; plugsky-lite for extraction and plugsky-pro for synthesis
Re-indexing	Embedding dimensions differ from other vendors — rebuild the index when switching
Product status	Live; audio, images, batch and fine-tuning are coming soon

TL;DR

• Two configuration objects connect LlamaIndex to Plugsky.
• api_base is the parameter name in LlamaIndex's OpenAI classes.
• Vector and summary indexes work without modification.
• Rebuild the index when changing embedding providers.
• Streaming query engines work through the same endpoint.

How it works, step by step

1. Install llama-index and export PLUGSKY_API_KEY.
2. Create the OpenAI LLM class with api_base pointing at Plugsky.
3. Create OpenAIEmbedding with model plugsky-embed and the same api_base.
4. Assign both to Settings so all components inherit them.
5. Load documents and build a VectorStoreIndex.
6. Query with a streaming query engine and inspect source nodes.
7. Add re-ranking, evaluation and persistence before production.

Original data

Try it yourself

Open the RAG sandbox →

Configure the LLM and embeddings

LlamaIndex's OpenAI classes accept a custom API base through the api_base parameter. Configure the LLM and embedder once, then assign them to Settings so every index and query engine uses Plugsky:

pip install llama-index

from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

llm = OpenAI(
    model="plugsky-pro",
    api_key="sk-live-…",
    api_base="https://api.plugsky.com/v1",
)

embed_model = OpenAIEmbedding(
    model="plugsky-embed",
    api_key="sk-live-…",
    api_base="https://api.plugsky.com/v1",
)

Settings.llm = llm
Settings.embed_model = embed_model

Test llm.complete("hello") before indexing anything. A 404 there means the model id is wrong; a 401 means the key or header is wrong.

Index a folder and query it

With Settings configured, a local RAG index is a few lines:

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex

documents = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine(streaming=True, similarity_top_k=4)
response = query_engine.query("What is our refund policy?")

for token in response.response_gen:
    print(token, end="", flush=True)

Streaming comes from the same chat completions endpoint, so a streaming query engine and a blocking one share the same credentials and model config. Use similarity_top_k to control how much context reaches the model, and print response.source_nodes while tuning retrieval — the model can only answer from what the retriever returns.

Chat engines and agents

Chat engines keep conversation state in LlamaIndex rather than on the provider, which fits Plugsky's chat-completions-first model:

chat = index.as_chat_engine(chat_mode="context", verbose=True)
print(chat.chat("And for international orders?"))

Agents use function calling through the same endpoint. Define tool specs with LlamaIndex's FunctionTool, give the agent a model that handles tools well — plugsky-pro is a solid default — and cap iterations so a confused agent cannot loop indefinitely. Because Plugsky's assistants and responses endpoints are coming soon, keeping state in LlamaIndex is the supported pattern today and keeps your data portable.

Embeddings, re-indexing and evaluation

Embedding hygiene is the most common source of silent RAG failures. Keep the embedding model fixed across indexing and querying, and rebuild the index whenever you change providers — dimensions and vector spaces differ, and mixed indexes degrade retrieval without raising errors.

Persist the index so you do not re-embed the corpus on every deploy, and record the model and chunking parameters in the index metadata. When answers regress, evaluate retrieval and synthesis separately: retrieval metrics on a small golden set of question-document pairs, then answer quality on the generated responses. If retrieval is fine but answers are not, adjust the prompt or move from plugsky-lite to plugsky-pro. If retrieval is weak, fix chunking, add a re-ranker or increase similarity_top_k before touching the model at all.

Honest comparison

Capability	Plugsky + LlamaIndex	LlamaIndex with OpenAI only	Custom RAG stack
LLM configuration	api_base override	Default endpoint	Write your own client
Embeddings	plugsky-embed with the same key	OpenAI embeddings	Host an embedding model
Indexing	Any LlamaIndex index and retriever	Same	Build ingestion yourself
Agents	Function calling on chat completions	Native tools support	Custom loop
Residency	Region pin plus private deployment options	US and EU options	You control everything
Ops	Managed API	Managed API	You run and scale it

Frequently asked questions

Which parameter sets the Plugsky endpoint in LlamaIndex?

Use api_base on the OpenAI LLM and OpenAIEmbedding classes, set to https://api.plugsky.com/v1.

Do I need a special integration package?

No. The standard llama-index OpenAI integration works because Plugsky is OpenAI-compatible; only the base URL and model id change.

Does streaming work?

Yes. Build the query engine with streaming=True and iterate response.response_gen for tokens as they arrive.

Can I use plugsky-embed for retrieval?

Yes. Set OpenAIEmbedding(model="plugsky-embed") and keep that model fixed for both indexing and querying.

Why should I re-index when switching providers?

Embedding dimensions and vector spaces differ between models. Reusing an old index with new query embeddings produces unreliable retrieval.

Can agents call my tools?

Yes. LlamaIndex FunctionTool definitions use the OpenAI function-calling schema, and tool-capable Plugsky models handle the loop.

How do I keep costs predictable?

Use plugsky-lite for extraction and plugsky-pro for final synthesis, cap similarity_top_k and agent iterations, and monitor usage per index.

What is coming soon?

Audio, images, moderation, files, batch, fine-tuning, assistants and the responses endpoint are roadmap items; chat, embeddings, tools and JSON mode are live.