Key facts
| API surface | OpenAI-shaped /v1/embeddings and /v1/chat/completions (live) |
| Features | Streaming, JSON mode and function calling (live) |
| Embeddings | plugsky-embed and plugsky-embed-multilingual (live) |
| Models | 30+ models behind one endpoint for generation |
| Migration | Base URL plus model names; re-embed the corpus once |
| Pricing | Flat monthly plans with unlimited fair-use usage on self-serve |
| Deployment | Region-locked planes, VPC, on-prem and air-gapped options |
| Free tier | plugsky-micro and plugsky-lite on the free plan, no card required |
TL;DR
- One client, two calls: embeddings for indexing and chat completions for answers.
- Keep your retrieval stack; re-embed into a new collection when providers change.
- Use JSON mode to return answers with a typed sources array.
- Stream answers to hide retrieval latency from users.
- Validate retrieval recall and citations in a shadow run before cutover.
How it works, step by step
- Audit every model call in your pipeline and confirm each maps to /v1/embeddings or /v1/chat/completions.
- Move base URL, key and model names into configuration, and create scoped Plugsky keys per environment.
- Re-embed the entire corpus with plugsky-embed (or plugsky-embed-multilingual) into a new collection.
- Switch the answer call to the Plugsky client and add a JSON-mode schema with answer and sources fields.
- Keep hybrid retrieval, filters and permission logic exactly where they are.
- Shadow-run both stacks on real questions and compare recall, citation accuracy and latency.
- Cut over by configuration, monitor usage, and retain the previous provider path for rollback.
Try it yourself
Open the OpenAI-compatible API tester →
Two calls, one client
RAG only touches the model at two points, and OpenAI-compatible APIs cover both. Indexing calls the embeddings endpoint per chunk; answering calls chat completions with retrieved context. Because request and response shapes match the SDK you already use, retries, timeouts and fixtures keep working.
Plugsky runs both endpoints live, with streaming, JSON mode and function calling available on the chat side. Managed extras such as files, batch and assistants-style endpoints are coming soon, so bulk ingestion and job orchestration stay in your pipeline — which is where document handling usually belongs anyway.
The migration checklist
Provider changes fail when vector spaces are mixed. Record your current embedding model and dimensions, backfill a new collection with the replacement model, and switch reads only after the new index passes retrieval checks. Software changes are trivial; data changes are the real work.
- Re-embed: never mix vectors from two models in one collection.
- Dimensions: confirm your vector store schema accepts the new size before the backfill.
- Model mapping: keep a table from old model names to Plugsky models so config stays readable.
- Rollback: retain the old collection until the new one is validated.
Citations, evaluation and operations
Ask for structured output: a direct answer plus a sources array of chunk IDs, mapped in your service to titles and links. That structure keeps citations honest and makes the interface simpler. Stream the answer so users see progress while retrieval happens ahead of the call.
Evaluate like a pipeline, not a demo: retrieval recall on a golden set, groundedness and citation accuracy on answers, and refusal correctness when evidence is missing. Compare the two providers on the same questions during the shadow period. Then operate it deliberately — separate keys for indexing and answering, usage alerts, and per-request traces of retrieved IDs. Pin deployments to a region that meets policy, including VPC, on-prem or air-gapped where required.
Honest comparison
| Aspect | Plugsky | Staying on OpenAI | Multiple providers |
|---|---|---|---|
| Client code | One compatible client | No change | Adapter per provider |
| Embeddings | plugsky-embed and plugsky-embed-multilingual (live) | Existing model | Varies |
| Answer format | JSON mode with sources array | Available | Inconsistent |
| Pricing | Flat monthly with unlimited fair use on self-serve | Per-token | Mixed bills |
| Residency | Region choice, VPC, on-prem, air-gapped | Limited options | Varies by vendor |
Frequently asked questions
Does the whole RAG stack need to change?
No. Only the client configuration, model names and the vector collection change. Parsing, chunking, retrieval and filtering remain untouched.
Why does re-embedding matter?
Vectors from different models are not comparable. If documents are embedded with one model and queries with another, similarity search breaks, so backfill a fresh collection before switching.
How do we return citations?
Use JSON mode to require an answer field and a sources array containing chunk IDs. Your service maps those IDs to document titles and URLs for display.
Is streaming supported?
Yes. Streaming is live on chat completions, so answers render progressively while retrieval runs ahead of the request.
What about the files and batch endpoints?
They are coming soon. Keep ingestion and bulk re-embedding in your own pipeline, and use the live chat and embeddings endpoints for model work.
How do we compare providers fairly?
Shadow-run identical questions through both stacks and compare retrieval recall, groundedness, citation accuracy and latency. Quality metrics matter more than price when pipelines differ.
How do we start?
Create a free workspace with plugsky-micro and plugsky-lite and no card, migrate a small collection, and validate before moving volume to a paid plan or the 14-day full-access trial.