Key facts
| API compatibility | Drop-in /v1/chat/completions with streaming, JSON mode and function calling (live) |
| Models | 30+ models behind one API, route routine and complex turns separately |
| Migration | Base URL plus model-name changes; SDK and retry logic unchanged |
| Tool use | Typed function schemas into internal APIs and knowledge sources |
| Pricing | Flat monthly plans with unlimited fair-use usage on self-serve |
| Controls | Scoped API keys, RBAC and SSO/SCIM for user identity |
| Deployment | Region-locked planes, VPC, on-prem and air-gapped options |
| Free tier | plugsky-micro and plugsky-lite on the free plan, no card required |
TL;DR
- Treat the provider as configuration — one client module, one base URL, many environments.
- Stream tokens so the copilot stays responsive while tool calls run.
- Use JSON mode for structured answers your UI and workflows can consume.
- Keep tools narrow and permission-checked; the model never gets direct data access.
- Roll out behind a flag and keep rollback to a config change.
How it works, step by step
- Centralise the model client so base URL, key and model names come from environment configuration.
- Set the base URL to https://api.plugsky.com/v1 with a scoped key, and map model names for routine and complex turns.
- Define tools for internal systems with typed schemas, and enforce user permissions inside each tool implementation.
- Implement streaming in the UI and structured output via JSON mode where the copilot returns records rather than prose.
- Shadow-test against real internal queries, comparing answer quality and latency with the current provider.
- Launch to a pilot team behind a feature flag, then widen access as trust signals improve.
- Review key scopes, retention settings and residency requirements with security before general availability.
Try it yourself
Open the OpenAI compatibility checker →
What a copilot needs from an API
Internal copilots need four things from the model layer: low-friction streaming, reliable tool calls, structured output and predictable billing. All four are available on Plugsky today — streaming, function calling and JSON mode are live on /v1/chat/completions, and self-serve plans are flat monthly with fair-use usage rather than per-token meters.
Roadmap endpoints such as files, batch and assistants-style runs are coming soon. That matters if you planned to outsource conversation state; until then, keep threads and scheduling in your application, which is generally the better design for internal tools anyway.
Adding tools and structured output
The copilot's usefulness comes from tools, not from the model alone. Wrap internal systems — ticketing, CRM, documentation, data warehouse — in narrow functions with clear descriptions and server-side permission checks. Then let the loop do the work: the model chooses tools, your code executes them, results come back and the answer improves.
- JSON mode: return typed objects — summary, category, next action — for workflows that need structure.
- Streaming: render partial answers while tools execute to keep perceived latency low.
- Validation: verify tool arguments before execution and reject anything outside expected ranges.
- Budgeting: cap turns and tool calls per request so one question cannot spiral.
Rollout without replatforming
Because the API is OpenAI-compatible, migration risk is low: the client code, request format and response parsing stay the same. Put base URL and model names in configuration, run both providers in shadow mode on a sample of real queries, and compare usefulness scores, latency and tool-call success before switching.
Scope keys per environment and per function, log model and tool activity for audit, and pin deployment to a region that satisfies policy — including VPC, on-prem or air-gapped options where required. Then iterate on the copilot itself: remove tools nobody calls, tighten descriptions that cause wrong calls, and route routine steps to plugsky-micro or plugsky-lite to keep the experience fast.
Honest comparison
| Aspect | Plugsky | Staying on OpenAI | Self-hosted runtime |
|---|---|---|---|
| Client change | Base URL and model names only | None | New serving stack |
| Streaming | Live | Available | You operate it |
| Tool calls | Function calling live | Available | Self-managed |
| Billing shape | Flat monthly with unlimited fair use on self-serve | Per-token | GPU and ops cost |
| Residency | Region choice, VPC, on-prem, air-gapped | Limited options | Wherever you deploy |
Frequently asked questions
Will our existing copilot code work?
Yes, if it targets OpenAI-style chat completions. Change the base URL and model names; request and response shapes stay compatible, so parsing and retry logic carry over.
How do we stream tool-based answers?
Stream the chat completion and render tool progress separately. The answer streams token by token while your service executes tool calls between turns.
Is JSON mode useful for an internal copilot?
Yes. It lets the copilot return structured records — summaries, categories, next actions — that your UI and downstream workflows can consume without parsing prose.
What about conversation state?
Keep it in your application. Assistants-style managed endpoints are coming soon, but owning thread state gives you better control over retention, audit and permissions.
How do we handle residency?
Pin the workspace to a region-locked plane, or deploy to VPC, on-prem or air-gapped when internal data cannot leave your network.
How do we start cheaply?
Use the free plan with plugsky-micro and plugsky-lite and no card to build and validate the tool loop, then move to a paid plan or the 14-day full-access trial.
What if quality drops after migration?
Run shadow comparisons on real queries and tune prompts and tool descriptions. Model behaviour differs between providers, so prompt tuning is normal, not a sign of incompatibility.