How do you use Plugsky with Haystack?

Key facts

Plugsky endpoint	POST https://api.plugsky.com/v1/chat/completions
Haystack generator	OpenAIChatGenerator(model="plugsky-pro", api_base_url="https://api.plugsky.com/v1")
Auth	Secret.from_env_var("PLUGSKY_API_KEY") or an explicit sk-live-… key
Embeddings	OpenAIDocumentEmbedder and OpenAITextEmbedder with model plugsky-embed
Live capabilities	Streaming, tool calling and JSON mode on the chat endpoint
Retrieval stack	Any Haystack 2.x document store or retriever; only the LLM and embedder change
Models	30+ models; plugsky-lite for cheap generation and plugsky-pro for final answers
Product status	Live; audio, images, batch and fine-tuning are coming soon

TL;DR

• One api_base_url override connects Haystack to Plugsky.
• Use plugsky-embed for indexing and queries on the same key.
• Haystack 2.x pipelines and components stay unchanged.
• Check embedding dimensions before reusing an existing index.
• Debug with a single generator call before wiring the full pipeline.

How it works, step by step

1. Install haystack-ai and export PLUGSKY_API_KEY.
2. Create an OpenAIChatGenerator with api_base_url pointing at Plugsky.
3. Create an embedder with model plugsky-embed and the same base URL.
4. Load documents and index them into your chosen document store.
5. Assemble a pipeline: retriever, prompt builder, generator.
6. Run a test query and inspect the generated answer and retrieved documents.
7. Add evaluation and tracing before production traffic.

Original data

Try it yourself

Open the RAG sandbox →

Install and configure the components

Haystack 2.x splits generation and embedding into separate components, and both accept a custom API base:

pip install haystack-ai

from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.embedders import OpenAIDocumentEmbedder, OpenAITextEmbedder
from haystack.utils import Secret

api_key = Secret.from_env_var("PLUGSKY_API_KEY")
base = "https://api.plugsky.com/v1"

generator = OpenAIChatGenerator(
    model="plugsky-pro",
    api_key=api_key,
    api_base_url=base,
)

doc_embedder = OpenAIDocumentEmbedder(
    model="plugsky-embed",
    api_key=api_key,
    api_base_url=base,
)
query_embedder = OpenAITextEmbedder(
    model="plugsky-embed",
    api_key=api_key,
    api_base_url=base,
)

Test the generator alone before building a pipeline. A single chat completion isolates key, URL and model problems from retrieval problems.

A minimal RAG pipeline

With an in-memory store, a full question-answering pipeline is short:

from haystack import Pipeline, Document
from haystack.components.builders import ChatPromptBuilder
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.dataclasses import ChatMessage

docs = [Document(content="Refunds take five business days."),
        Document(content="Support hours are 9am to 5pm GST.")]
store = InMemoryDocumentStore()
store.write_documents(docs)

template = [
    ChatMessage.from_system("Answer using only the provided context."),
    ChatMessage.from_user("Context: {{documents}}\nQuestion: {{query}}"),
]

pipe = Pipeline()
pipe.add_component("retriever", InMemoryBM25Retriever(document_store=store))
pipe.add_component("prompt", ChatPromptBuilder(template=template, required_variables=["query"]))
pipe.add_component("llm", generator)

pipe.connect("retriever.documents", "prompt.documents")
pipe.connect("prompt.prompt", "llm.messages")

result = pipe.run({"retriever": {"query": "How long do refunds take?"},
                   "prompt": {"query": "How long do refunds take?"}})
print(result["llm"]["replies"][0].text)

Swap the BM25 retriever for a vector retriever once the pipeline works; the generator and prompt stages do not change.

Embeddings and index hygiene

For semantic retrieval, embed documents at write time and queries at read time with the same model. Two rules keep retrieval honest:

• Same model on both sides: plugsky-embed for indexing and querying. Mixing models produces vectors that are not comparable.
• Re-index when you switch providers: embedding dimensions differ between vendors. Building a new index is cheaper than debugging mysterious relevance failures.

Store the embedding model name and version alongside the index metadata. When the model changes, you can rebuild exactly the affected collections rather than the entire corpus.

Debugging and evaluation

When answers look wrong, separate the stages. Print the retrieved documents first — if the right passage is missing, the problem is chunking, retrieval or the index, not the model. If the passage is present but the answer is wrong, tighten the prompt and lower the temperature.

Common errors map cleanly: 401 means the key is missing or malformed, 403 means the key lacks a scope, 404 means the model id is wrong (list /v1/models to confirm), and 400 with context counts means the assembled prompt is too long. Trim retrieved documents or use a long-context model such as plugsky-longctx. Add a small golden set of questions with known answers and run it after every prompt or chunking change — retrieval quality regresses quietly otherwise.

Honest comparison

Capability	Plugsky + Haystack	Haystack with vendor-only LLMs	Custom retrieval stack
Generator setup	One api_base_url override	Vendor defaults	Write your own client
Model choice	30+ models behind one endpoint	Whichever vendors you integrate	You integrate each one
Embeddings	plugsky-embed with the same key	Separate vendor keys and quotas	Run your own embedding model
Pipelines	Haystack 2.x components unchanged	Same	Build orchestration yourself
Residency	Region pin plus private deployment options	Depends on each vendor	You control the stack
Ops	Managed API	Managed per vendor	You run and scale it

Frequently asked questions

Which Haystack component calls Plugsky?

OpenAIChatGenerator and OpenAIChatGenerator variants accept api_base_url. Point it at https://api.plugsky.com/v1 and pass your Plugsky key.

How do I configure the embedding model?

Use OpenAIDocumentEmbedder and OpenAITextEmbedder with model plugsky-embed and the same api_base_url.

Do I need a different retriever?

No. Keep any Haystack 2.x document store and retriever. Only the generator and embedder need the Plugsky base URL.

Does streaming work in pipelines?

The OpenAI-compatible endpoint supports server-sent events. Use the streaming-capable generator when you want incremental output.

Why do I get a 404 for a model?

The model id is not in the catalogue. Call GET /v1/models with your key and use an id returned there.

Can I mix Plugsky and another provider in one pipeline?

Yes. Components are independent, so you can generate with Plugsky while embedding with another provider — but keep the embedding model consistent across index and query.

How should I handle long contexts?

Trim retrieved documents first, then consider plugsky-longctx. Context errors return exact token counts so you can size the cut precisely.

What is not supported yet?

Audio, images, moderation, files, batch, fine-tuning, assistants and the responses endpoint are coming soon; chat, embeddings and tools are live.