Key facts
| Deployment shapes | Managed cloud, VPC, on-prem and air-gapped |
| Interface | OpenAI-compatible /v1 so existing SDKs and tools keep working |
| Models | 30+ models across chat, reasoning, embeddings and coding tiers |
| Governance | SSO, RBAC, audit export, DPA and SLA available at enterprise tier |
| Keys | Customer-managed keys for VPC and on-prem deployments |
| Endpoints | Chat, streaming, JSON mode, function calling, embeddings, RAG and agents are live |
| Roadmap endpoints | Audio, image, moderation, batch and fine-tuning are coming soon |
TL;DR
- Start with one workload, defined success metrics and a fixed evaluation set.
- Choose the deployment shape from residency and latency requirements, not preference.
- Budget for GPUs, power, networking and staff as well as licences.
- Standardise identity, audit and key management before scaling workloads.
- Keep the API OpenAI-compatible so application teams do not rewrite integrations.
How it works, step by step
- Document data classification and residency requirements for the target workload.
- Pick one pilot: a bounded use case with measurable quality and latency targets.
- Select the deployment shape: VPC, on-prem or air-gapped, based on residency and network constraints.
- Size GPUs and memory from concurrency and context, not from model size alone.
- Integrate identity, RBAC and audit logging on day one rather than retrofitting.
- Run acceptance tests on real data and compare against the incumbent provider.
- Plan the rollout: shared platform, cost model and support ownership for additional workloads.
Try it yourself
Open the private LLM deployment estimator →
Deployment shapes and what they imply
A VPC deployment keeps the control plane managed while inference and data stay inside your network boundary, which suits teams that need residency without running everything. On-prem moves the full stack into your data centre, giving complete control at the cost of owning hardware lifecycle, patching and capacity planning. Air-gapped removes internet egress entirely for classified or critical infrastructure environments, with offline update channels and physical media handling.
Each step inward raises cost and operational load. Pick the least restrictive shape that satisfies the requirement, because every constraint you add buys governance and costs agility. Document who operates what: who patches, who rotates keys, who owns incident response.
Governance and platform requirements
Local deployment does not remove the need for governance. Identity, access control and audit become yours to run, and regulators will ask for the same evidence whether inference is local or hosted. Build these capabilities into the platform before onboarding multiple teams.
- SSO integration and per-team role-based access.
- Audit logs covering prompt metadata, model version and key usage.
- Key management with rotation and separation of duties.
- Model registry with versions, evaluations and rollback.
- Data retention and deletion policies that match your DPA commitments.
Plugsky enterprise terms cover DPA, SLA and audit expectations, and the platform supports customer-managed keys in private deployments.
TCO, pilot design and scaling out
Total cost of ownership has four parts: GPUs and servers, data-centre power and cooling, platform operations staff, and the software or support contract. Hardware is visible; staffing is usually underestimated. Include model updates, evaluation runs and re-indexing in the operational budget, because they recur.
Design the pilot to be defensible: one workload, a fixed evaluation set, quality and latency targets, and a comparison against the current provider. Choose a workload with clear success criteria rather than the most exciting one. When the pilot passes, scale by workload, not by department, and reuse the same gateway, key management and observability. Plugsky keeps the interface OpenAI-compatible across 30+ models, so applications that already integrate stay portable, and enterprise deployments can move from cloud to VPC to on-prem without rewriting client code. Review current terms on the live pricing page.
Honest comparison
| Deployment | Data location | Operations | Best for |
|---|---|---|---|
| Managed cloud | Provider regions | Provider-managed | Fast pilots and low-risk workloads |
| VPC | Your network boundary | Shared | Residency without full ownership |
| On-prem | Your data centre | You operate the stack | Strict control and integration |
| Air-gapped | Isolated network | You operate with offline updates | Classified and critical infrastructure |
| Hybrid | Sensitive data local, bursts hosted | Split responsibilities | Mixed sensitivity and load |
Frequently asked questions
When does enterprise local AI make sense?
When residency rules, latency requirements, vendor concentration risk or per-token cost at scale outweigh the operational convenience of a public API.
What deployment options does Plugsky offer?
Plugsky supports managed cloud, VPC, on-prem and air-gapped deployments. Chat, streaming, JSON mode, function calling, embeddings, RAG and agents are live; audio, image, moderation, batch and fine-tuning endpoints are coming soon.
How long does an on-prem deployment take?
Timelines depend on hardware procurement and network constraints. Most organisations begin with a scoped pilot on one workload and expand after acceptance tests pass.
Do existing applications need changes?
No, if they already speak the OpenAI-compatible API. The base URL and model name change; SDK code stays the same.
What governance features are available?
Enterprise deployments support SSO, RBAC, audit logging, customer-managed keys, DPA and SLA commitments. Confirm specifics with your account team.
How should we size GPUs?
Start from concurrency, context length and model size. Weights, KV cache and runtime overhead all consume memory, and throughput depends on batching.
Can we keep using public APIs for some workloads?
Yes. A hybrid policy keeps sensitive workloads local and sends lower-risk or burst traffic to a hosted API through the same gateway.
How do we prove value from the pilot?
Fix an evaluation set and targets before starting, then report quality, latency and cost per task against the incumbent rather than anecdotes.