How should architects plan air-gapped AI on Plugsky?

Key facts

API parity	OpenAI-compatible endpoints offline; applications unchanged across tiers
Runtime	On-prem and air-gapped deployments with open-weight models
Capacity	GPU sizing driven by model size, concurrency and node-failure headroom
Key management	Customer-managed keys via KMS or on-prem HSM
Update pipeline	Controlled intake with provenance, scanning, staging and rollback
Identity	SSO/SCIM for people; scoped keys for services inside the perimeter
Observability	Inference, key and admin events exported to internal SIEM
Status	Live platform; enterprise deployment terms per engagement

TL;DR

• Design four tiers: inference, control, artifact pipeline and observability.
• Size GPUs for peak plus failure headroom — you cannot burst to cloud.
• Make the update pipeline a reviewed, versioned software supply chain.
• Keep keys in your KMS or HSM with separated duties.
• Rehearse restore and failover before go-live; air gaps fail operationally.

How it works, step by step

1. Decide which workloads truly require no egress, and which can use VPC or on-prem with controlled egress at lower cost.
2. Model GPU capacity: target models, expected concurrency, context lengths, and headroom for node failure plus peaks.
3. Define the topology — inference nodes, gateway, model registry, key management and log pipeline — inside the perimeter.
4. Specify the artifact pipeline for packages, models and runtime updates, including checksums, scanning and rollback.
5. Design identity and authorisation: SSO/SCIM for staff, scoped service keys, and separation between key and platform administration.
6. Plan disaster recovery: artefact backups, key escrow, restore runbooks and periodic game days.
7. Write an exit and upgrade strategy, then prove the design with one production workload and an evaluation suite.

Try it yourself

Open the LLM GPU capacity calculator →

Reference topology for a disconnected estate

Four tiers, each with a clear owner. The inference tier runs the OpenAI-compatible runtime and open-weight models on GPU nodes sized for your concurrency. The control tier holds the gateway, model registry, policy and key integration. The artifact pipeline moves packages, weights and updates from the connected world into the perimeter under review. The observability tier collects identity, key and inference events for SIEM and capacity analysis.

Because the API contract matches the hosted platform, application architecture stays portable: the same clients, prompts and evaluation harnesses work in both worlds, which keeps skills and tooling reusable.

Capacity, performance and failure

Air-gapped capacity is a fixed commitment, so model the tail, not the average. Factor model size and quantisation, context length, concurrency, and the headroom needed to survive a node failure without breaching latency targets. Smaller models may serve classification and extraction while a larger model handles reasoning, which spreads load across the estate.

• No bursting: peak demand must be provisioned, or gracefully queued.
• Upgrades cost capacity: scheduling a rollout with zero spare nodes forces an outage window.
• Measure honestly: instrument latency and queue depth from day one and review against the model.

Governance of artefacts and keys

In a connected platform, updates are the provider's problem. In an air gap they are yours. Run the artifact pipeline as a software supply chain with provenance, checksums, scanning, staging validation and rollback, and keep a manifest of exactly what runs where. Model version drift between staging and production is the most common cause of unexplained quality changes.

Keys deserve the same rigour: customer-managed keys in your KMS or HSM, dual control for destructive operations, escrow and tested recovery. Combined with SIEM-bound audit events, this gives auditors a story they can verify inside the perimeter without an internet connection.

Honest comparison

Architecture axis	Plugsky air-gapped	VPC deployment	Cloud-only platform
Egress	None required	Controlled	Inherent
API portability	OpenAI-compatible across tiers	Same	Vendor-specific
Capacity model	Fixed estate with headroom planning	Elastic with limits	Fully elastic
Update control	Your pipeline and cadence	Shared	Provider-scheduled
Key control	KMS or on-prem HSM	KMS integration	Provider-managed
Operational burden	Highest	Moderate	Lowest

Frequently asked questions

When is an air gap justified over a VPC?

When regulation, data classification or network architecture forbids any external path. VPC and on-prem tiers with controlled egress satisfy most residency and isolation requirements at lower cost and effort.

How do we size GPU capacity?

Model target models, concurrency, context lengths and failure headroom, then validate with load tests. Air-gapped estates cannot burst, so peaks must be provisioned or queued.

How do updates reach the estate?

Through your controlled artifact pipeline: verified media, checksums, scanning, staging validation and rollback. You control the cadence and the acceptance gate.

What stays the same as the hosted platform?

The API contract. Chat completions, streaming, JSON mode, function calling and embeddings use the same request shapes, so applications and evaluations transfer unchanged.

How do we prevent model drift?

Keep a manifest mapping environments to explicit model and runtime versions, run the same evaluation suite against staging and production, and treat updates as reviewed releases.

What is the biggest operational risk?

Update and recovery discipline, not inference itself. Rehearse restore, failover and bundle installation before go-live, then repeat on a schedule.

How do we start?

Choose one bounded workload with a named owner and an evaluation set, deploy to the disconnected staging instance first, then promote. Enterprise engagements cover deployment review and support terms.