Key facts
| API contract | Identical OpenAI-compatible endpoints offline and hosted |
| Supported features | Chat completions, streaming, JSON mode, function calling and embeddings are live |
| Models | Open-weight models suitable for offline operation from the catalogue |
| Toolchain | Local package mirrors, artifact registries and controlled update bundles |
| Environment parity | Staging instance inside the perimeter mirrors production configuration |
| Debugging | Request IDs and local logs replace hosted dashboards in disconnected networks |
| Endpoint status | Assistants and responses endpoints are coming soon and should not be assumed offline |
| Starting point | Free plan with plugsky-micro and plugsky-lite for prototyping the same code path |
TL;DR
- Write normal OpenAI-compatible code; only the base URL changes.
- Plan the artifact supply chain — packages, models, updates — before the network does not exist.
- Keep a staging instance inside the perimeter so tests match production.
- Use request IDs and structured logs; there is no vendor dashboard to fall back on.
- Prototype in the cloud, deploy the same code path inside the network.
How it works, step by step
- Prototype against the hosted API using plugsky-micro or plugsky-lite so your integration and tests are settled early.
- Inventory every dependency — SDKs, tokenisers, model artefacts, vector libraries — and arrange internal mirrors.
- Stand up a staging instance inside the perimeter that matches production configuration and model versions.
- Pin model and runtime versions explicitly, and record them in a manifest you can diff between environments.
- Add request IDs, structured logs and token metrics to your code, since hosted observability will not be available.
- Package updates as verified bundles with checksums and a documented rollback path.
- Rehearse a full deploy, rollback and restore on the disconnected staging instance before touching production.
Try it yourself
Open the private LLM deployment estimator →
The code does not care about the network
From your application's perspective, air-gapped Plugsky is the same API on a different host. Chat completions, streaming, JSON mode, function calling and embeddings are available through the same request shapes, so existing SDKs and prompt code work unchanged. Keep a single base URL constant in configuration and the same integration serves cloud, VPC, on-prem and disconnected environments.
What differs is everything around the code: dependency installs, model pulls and updates cannot reach the internet. Treat the disconnected network as a deployment target with its own supply chain, not as an exotic special case.
Managing the artifact pipeline
Plan the intake of four artefact classes: packages and base images, model weights, runtime updates, and configuration. Each needs provenance, checksums, scanning and a staging validation step before production. A manifest that maps environment to model version and runtime version turns drift into a diff you can review.
- Mirror early: internal package registries and container registries prevent mid-project surprises.
- Version everything: model aliases should resolve to explicit versions inside the perimeter.
- Test the bundle path: a bundle that installs on a laptop may fail behind your controls.
Debugging without a dashboard
Disconnected development removes the safety net of hosted analytics, so build observability into the application. Emit request IDs, model version, prompt version, token counts, latency and outcome for every call, and aggregate them locally. That data powers both debugging and capacity planning.
Keep the test suite identical across environments: the same evaluation set should run against cloud, staging and production models, with results recorded in the manifest. When an update bundle arrives, the suite is your acceptance gate; when quality drops, per-version history tells you which artefact changed.
Honest comparison
| Developer concern | Plugsky air-gapped | Hosted model API | DIY serving stack |
|---|---|---|---|
| API compatibility | OpenAI-compatible offline | Native vendor API | You define it |
| Feature set | Chat, streaming, JSON mode, tools, embeddings live | Broadest feature set first | Whatever you implement |
| Dependency supply | Internal mirrors and bundles | Public registries | You own it end to end |
| Observability | Your logs and metrics | Vendor dashboards | Your build |
| Environment parity | Same code path across tiers | Cloud only | Manual parity work |
| Effort | Deployment and pipeline work | Almost none | Serving, tooling, upgrades |
Frequently asked questions
Is the API really identical?
Yes. The same chat completions, streaming, JSON mode, function calling and embeddings endpoints are available offline. Only the base URL and model mapping differ.
How do we get packages and models inside?
Through internal mirrors and verified bundles with checksums, scanning and staging validation. Design the intake process before the network is disconnected, not after.
Can we develop on laptops and deploy offline?
Yes, if you maintain a staging instance inside the perimeter that matches production. Run your evaluation suite there before promoting any artefact.
What replaces hosted analytics?
Your own telemetry. Emit request IDs, model and prompt versions, token counts, latency and outcomes, then aggregate them locally for debugging and capacity planning.
Which model should we start with?
An open-weight model sized to your hardware and task. Prototype against plugsky-micro or plugsky-lite in the cloud to settle your integration and evals first.
Do assistants-style endpoints work offline?
Assistants and responses endpoints are coming soon, and offline support should not be assumed until they are documented. Build on chat completions today.
How do we test upgrades safely?
Pin versions in a manifest, run the same evaluation suite against staging and production, and package every update with checksums and a rollback path.