Key facts
| API compatibility | OpenAI-shaped /v1/embeddings and /v1/chat/completions (live) |
| Indexing | plugsky-embed and plugsky-embed-multilingual for vector search (live) |
| Answer synthesis | Streaming, JSON mode and function calling (live) |
| Models | 30+ models, route ranking helpers and synthesis independently |
| Migration | Change base URL and model names; re-embed into a new collection |
| Access control | Scoped API keys per service with RBAC and SSO/SCIM |
| Residency | Region-locked planes plus VPC, on-prem and air-gapped options |
| Free tier | plugsky-micro and plugsky-lite on the free plan, no card required |
TL;DR
- Keep search relevance in your stack; replace only the embedding and chat calls.
- Index with plugsky-embed or plugsky-embed-multilingual and re-embed rather than mixing vector spaces.
- Use JSON mode to return answer, sources and confidence as a typed object.
- Scope keys per service so an answer endpoint cannot re-index the corpus.
- Ship to a pilot group first and compare citation accuracy against your current search.
How it works, step by step
- Wrap your search service's model calls behind a small client interface so the provider is a configuration value.
- Create a Plugsky workspace, issue a scoped key for the search service, and set the base URL in your config.
- Re-embed the corpus with plugsky-embed (or plugsky-embed-multilingual) into a new vector collection alongside your existing index.
- Update the answer step to call /v1/chat/completions with the retrieved passages and a JSON-mode schema for answer plus sources.
- Keep ACL resolution and ranking filters in your service, applied before passages reach the model.
- Run a pilot group for two weeks, comparing zero-result rate, citation accuracy and user satisfaction with the old interface.
- Cut over by configuration, and keep the previous provider path one config flip away for rollback.
Try it yourself
Open the AI workload router simulator →
Compatibility in a search service
Enterprise search already has the hard parts: connectors, entitlements, deduplication and ranking. An OpenAI-compatible API changes none of that. It standardises the two model-shaped calls in the pipeline — turning text into vectors and turning retrieved passages into an answer — so the provider becomes an implementation detail behind your client.
Because the request and response shapes match the OpenAI SDK, existing retry logic, timeouts and test fixtures keep working. Where specialist endpoints exist for other workflows, note that files, batch and assistants-style endpoints are coming soon, so ingestion and bulk re-embedding should stay in your pipeline for now.
Wiring embeddings and cited answers
Indexing calls /v1/embeddings per chunk and writes vectors plus ACL metadata. Query time embeds the question with the same model, retrieves allowed chunks, then sends them to /v1/chat/completions with a JSON-mode schema that requires an answer and a sources array. That gives your UI structured citations instead of prose it has to scrape.
- Re-embed on provider change: vectors from different models are not comparable, so backfill a new collection before switching reads.
- Stream the answer, not the retrieval: show passage skeletons while synthesis streams, so perceived latency stays low.
- Budget the prompt: rank and trim passages to fit the model's context rather than truncating blindly.
Operations: keys, regions and rollout
Search is a shared service, so operate it like one. Use separate keys for indexing and answering, cap the answer key's rate and scope, and rotate both on a schedule. Log retrieval decisions and the citation set per request so relevance complaints can be replayed.
Residency usually applies to the corpus, not the interface, but the prompt carries document text to the model. Pin the workspace to a region-locked plane, or deploy to VPC, on-prem or air-gapped when content cannot leave your network. Roll out behind a pilot group, measure citation accuracy and zero-result rate, then widen — and keep provider selection in configuration so future moves are deploys, not migrations.
Honest comparison
| Layer | Plugsky | Staying on OpenAI | Self-hosted models |
|---|---|---|---|
| Client code | OpenAI-compatible, config change only | No change | Rewrite against a new runtime |
| Embeddings | plugsky-embed and plugsky-embed-multilingual (live) | Existing model | You operate the server |
| Answer synthesis | 30+ models behind one endpoint | OpenAI catalogue only | One model per deployment |
| Pricing | Flat monthly with unlimited fair use on self-serve | Per-token | GPU plus operations cost |
| Residency | Region choice, VPC, on-prem, air-gapped | Limited region options | Wherever you deploy |
Frequently asked questions
Does my search index need to change?
Only its vectors. Keep the documents, connectors and ACLs; re-embed chunk text into a new collection with plugsky-embed or plugsky-embed-multilingual, then switch reads.
Can I stream answers in the UI?
Yes — streaming is live on chat completions. A common pattern is to stream the answer while rendering retrieved passage placeholders immediately.
How do I return citations?
Use JSON mode to require an answer field and a sources array containing chunk or document IDs. Your service maps IDs back to titles and deep links.
Where do permissions live?
In your service, applied as filters before retrieval. Plugsky only sees the passages you choose to include in the prompt.
Is this suitable for regulated content?
Yes, with the right deployment. Region-locked planes cover most policy requirements, and VPC, on-prem or air-gapped options cover strict sovereignty rules.
What breaks first in migration?
Embedding dimensionality and model IDs. Record the old dimensions, re-index cleanly, and keep a mapping table for model names.
How should we measure the pilot?
Compare zero-result rate, citation accuracy and time-to-answer against your current interface, using the same query set for both.