Key facts
| Architect fit | One API surface across cloud, VPC, on-prem and air-gapped topologies |
| Deployment | Gateway inside a controlled zone; approved subnets only |
| Models | 30+ models; bundles licensed per deployment agreement |
| Live endpoints | Chat, streaming, function calling, JSON mode and embeddings |
| Identity | Scoped keys per service, SSO with SCIM and RBAC for operators |
| Observability | Auth, key and admin events exportable to SIEM; metrics for capacity |
| Operations | Enterprise owns capacity, upgrades, patching and on-call |
| Pricing | Flat monthly self-serve plans; enterprise deployment scoped on the pricing page |
TL;DR
- Treat the deployment as a topology decision, not a product decision: the API stays the same.
- Design the gateway, identity and log paths before sizing hardware.
- Plan peak capacity, upgrade headroom and dual-site DR as one exercise.
- Keep release management explicit — who patches what, and on which calendar.
- Compare total cost of ownership against VPC and flat monthly plans honestly.
How it works, step by step
- Write the topology decision record: which workloads run cloud, VPC, on-prem or air-gapped, and the policy reason for each.
- Define the network zone, gateway placement, egress rules and the subnets allowed to reach the endpoint.
- Design identity: scoped keys per service and environment from the corporate secret store, plus SSO with SCIM and RBAC for operators.
- Size capacity from peak concurrency, context length and the next model upgrade, and model day-two operations costs.
- Design disaster recovery with a second site, documented failover and restore tests, and key availability in both sites.
- Specify observability: authentication, key and admin events to the SIEM, plus performance metrics, queue depth and error budgets.
- Agree the release and patch calendar with the vendor, then run a limited pilot with a named owner and exit criteria.
Try it yourself
Open the private LLM deployment estimator →
A topology decision, not a product decision
The strongest argument for treating on-prem Plugsky as a topology choice is that applications keep using the same OpenAI-compatible API. The cloud, VPC, on-prem and air-gapped variants differ in where inference and logs live, who operates capacity, and how updates arrive — not in the interface your services call. That lets architecture govern placement per workload while platform engineering maintains one integration pattern.
It also keeps options open: a workload can move from shared cloud to VPC to on-prem as its classification changes, without a rewrite or a new evaluation baseline.
Design the hard parts first
Three concerns dominate on-prem programmes, and all three are architecture problems before they are procurement problems. Capacity must be sized for peak concurrency and worst-case context length with headroom for upgrades. Resilience needs a second site, tested failover and key availability in both. Release management must name who applies platform updates, when, and how rollback works under the licensing agreement.
- Gateway: one policy point for quota, model allow-lists and logging.
- Secrets: scoped keys per service and environment, rotated on schedule.
- Evidence: auth, key and admin events to the SIEM; metrics retained for capacity reviews.
- Change control: model bundle and platform releases on a published calendar.
Cost, trade-offs and boundaries
On-prem converts a variable token cost into capital, licensing and headcount, so compare like for like. Low-utilisation workloads rarely justify it; regulated or high-utilisation ones often do. Keep VPC deployment and flat monthly plans on the table as intermediate options — see the live pricing page for current tiers — and revisit the decision as usage grows.
State boundaries precisely in the architecture pack: Plugsky provides the model API, authentication surface, audit events and deployment variants. It is not your identity provider, policy engine, vector database or release train for your own applications. Endpoint status is uniform across deployments — chat, streaming, JSON mode, function calling and embeddings are live; audio, images, files and batch are coming soon.
Honest comparison
| Concern | Plugsky on-prem | VPC deployment | Shared cloud API |
|---|---|---|---|
| Inference location | Your data centre or air-gapped site | Your cloud account | Vendor region |
| Operational owner | Your platform team | Mostly vendor, some network work | Vendor |
| Capacity model | Capex plus headroom planning | Contract capacity | Flat monthly, fair use |
| DR design | Dual site, your runbooks | Multi-zone by provider | Vendor-managed |
| Ideal workload | Classified, latency-critical, high utilisation | Regulated with cloud allowance | General product and internal tools |
Frequently asked questions
Is on-prem a different API?
No. All deployment variants expose the same OpenAI-compatible API, so services, SDKs and evaluations stay unchanged across cloud, VPC, on-prem and air-gapped.
How do we decide per workload?
Use classification, latency requirements and utilisation. Low-sensitivity, low-volume workloads rarely justify on-prem; regulated or high-utilization ones usually do.
What capacity should we plan for?
Peak concurrency and worst-case context length, plus headroom for a model upgrade. Averages mislead because long documents and tool loops dominate resource use.
Who applies upgrades?
The enterprise operates its deployment. Platform and model bundle releases arrive on an agreed calendar, and rollback procedures should be documented before the first production workload.
How is identity handled?
Scoped keys per service and environment, stored in the corporate secret store, plus SSO with SCIM and RBAC for operator access.
What evidence goes to the SIEM?
Authentication, key lifecycle and administrative events, joined with inference metadata such as model, tokens and latency, exported from every deployment variant.
Should we standardise on one deployment?
No. Standardise the integration pattern and let workload policy choose the topology; that keeps resilience and cost options open.