Key facts
| Deployment | On-premises enterprise option |
| API surface | OpenAI-compatible chat completions and embeddings |
| Hardware | Customer data-centre servers and GPUs; sizing guidance in docs |
| Models | Open-weight catalogue served locally; hosted frontier models not included |
| Operations | Self-managed, or vendor-managed under an agreement |
| Connectivity | Restricted egress supported; full isolation via air-gapped mode |
| Key custody | Yours — BYOK/HSM options available in private deployments |
| Product status | Live via enterprise agreement |
TL;DR
- On-prem is a first-class deployment, not a fork of the product.
- Your data centre, your keys and your network boundary.
- Open-weight models only — hosted frontier models stay in the cloud.
- You or a managed agreement carries GPU operations and upgrades.
- Start with the API already in place; migrate workloads after capacity is proven.
How it works, step by step
- Confirm why on-prem is required — residency, network isolation or procurement rules.
- Size GPU capacity for your peak concurrency and chosen open-weight models.
- Pick the operating model: your team runs it, or a managed agreement with Plugsky.
- Install the OpenAI-compatible gateway and model server on your infrastructure.
- Validate with existing clients, tests and evals against the on-prem endpoint.
- Set the maintenance cadence for model updates, patches and capacity headroom.
Try it yourself
Open the private LLM cost estimator →
What running fully on-prem includes
An on-prem deployment puts the whole serving path inside your data centre: the API gateway, model runtime, request logs and storage. Your OpenAI-compatible endpoint is reachable on your network, and no prompts or outputs leave your perimeter. Keys can stay in your own KMS or HSM, which matters when encryption-at-rest is audited. The software stack is the same product surface as the cloud deployment, so the API your developers test against locally is the one they deploy against in production.
The honest trade-offs
On-prem trades managed convenience for control:
- Model catalogue: only open-weight models run locally; hosted frontier models require the cloud.
- Capacity: you buy GPUs for peak load, and idle capacity is still capital.
- Operations: upgrades, monitoring and incident response are yours unless covered by a managed agreement.
- Refresh cadence: new models arrive when you deploy them, not the day they launch.
For regulated workloads, those costs are usually acceptable and often mandatory. For teams that only need privacy, a VPC deployment is a lighter middle ground. Benchmark total cost of ownership over three years, including GPUs, power, staff and refresh cycles, before comparing against a managed plan.
What we do and what we do not do
What we do: provide the deployable stack, sizing guidance, support and a managed option; keep the API OpenAI-compatible so applications do not change; and support BYOK and isolated networking. What we do not do: supply server hardware or data-centre space, promise hosted-model parity on-prem, or run your environment without an agreement that defines who operates what. Decide the operating model early — it changes the runbook, the SLA and the cost profile. Review the SLA with your team before committing.
Honest comparison
| Capability | Plugsky on-prem | Plugsky VPC | Plugsky shared cloud |
|---|---|---|---|
| Data location | Your data centre | Your cloud account | Plugsky managed region |
| Hardware owner | You | You | Plugsky |
| Model catalogue | Open-weight models | Open-weight plus opt-in routing | 30+ models |
| Keys | Your KMS/HSM | Your cloud KMS | Plugsky-managed |
| Ops burden | Highest | Moderate | Lowest |
| Best fit | Strict physical control | Network isolation with cloud scale | Fast self-serve start |
Frequently asked questions
Do I need GPUs to run on-prem?
Yes. On-prem deployments run on your servers and GPUs; the docs and your account team help size capacity for your concurrency and chosen open-weight models.
Can I use frontier hosted models on-prem?
No. Hosted frontier models require an external path; on-prem deployments use locally served open-weight models.
Does the API change on-prem?
No. The gateway is OpenAI-compatible, so SDK code, tests and evals work the same against the on-prem endpoint.
Who operates the deployment?
Either your team with vendor support, or Plugsky under a managed agreement. The operating model is defined during scoping and reflected in the SLA.
Can I keep my own encryption keys?
Yes. Private deployments support your KMS or HSM, so keys never leave your control.
What is the difference between on-prem and air-gapped?
On-prem may still allow restricted egress for updates and support. Air-gapped removes the network path entirely for stricter isolation.
Is on-prem available on the free plan?
No. The free plan runs on the shared managed cloud with two free models; on-prem is scoped as an enterprise deployment. See the live pricing page.