Guide · private deployment

Private VPC AI deployment

Run inference inside your cloud account, reach it over a private endpoint, and keep model traffic off the public internet. The network is the easy part; identity, keys and observability decide whether the deployment is operable.

Short answer. Put the inference data plane in your VPC, expose it through AWS PrivateLink, Azure Private Link or Google Private Service Connect, and remove the default internet route. Use workload identity, customer-managed keys and private observability. Measure latency in your own region. A VPC is an architecture, not a compliance certificate; verify the contract and counsel-review the regulated claims.

Peering versus PrivateLink

PropertyVPC peeringPrivateLink / PSC
DirectionBidirectional routesConsumer to a specific service
CIDRMust not overlapNo overlap required
ScopeWhole VPC relationshipOne service endpoint
GovernanceRoute tables and security groupsEndpoint policy and service acceptance
Best forFull network integrationNarrow, auditable service access

Peering is simpler when both sides need broad access and the CIDRs are planned. PrivateLink is easier to defend when a regulator asks what the AI provider can reach: the consumer can reach one endpoint, not the whole VPC. Peering is not transitive; a hub-and-spoke design needs a transit gateway or a mesh. Choose the narrowest path that still meets the latency target.

Egress control

ControlWhat it preventsHow to verify
No internet gateway or NATDefault outbound pathRoute table audit; flow logs
VPC endpointsPublic S3, KMS, logs trafficEndpoint policy and DNS resolution
Allowlisted proxyUnapproved package downloadsProxy logs; deny-by-default
DNS controlsResolution to public servicesPrivate hosted zones; resolver rules
Network firewallLateral movement and exfiltrationPolicy hits; denied flows

The inference path should not need the internet. If the model container pulls packages at startup, pin the image and mirror the registry inside the VPC. If observability exports to a SaaS endpoint, use a private endpoint or a local collector. A VPC that still has a NAT gateway for “just updates” is a private network with an exit.

Identity, keys and observability

LayerPatternWhat to avoid
User identitySAML or OIDC through your IdP; group mappingLocal accounts per team
Workload identityIAM roles for service accounts, managed identity, workload identity federationStatic keys in environment variables
Service-to-servicemTLS; short-lived tokens; per-team keysOne shared key for every service
Key custodyCustomer KMS/HSM; envelope encryptionProvider-held key with no audit trail
ObservabilityVPC flow logs, audit logs, metrics and traces to your SIEMPayload logging on by default

Use separate keys per environment and per region, and keep the key policy in infrastructure as code. Key custody is covered in more depth in what BYOK means for AI; the short version is that a VPC does not protect data if the provider can unwrap the key without your policy. For observability, agree on what is logged: request metadata is usually enough; prompt and response bodies should be opt-in and retained deliberately.

Deployment steps

  1. Landing zone. Confirm accounts, regions, CIDRs and the network account that owns the shared services.
  2. Private path. Create the peering or endpoint, accept the service, and update route tables and security groups.
  3. DNS. Resolve the API hostname to the private endpoint; test from a workload subnet, not a laptop.
  4. Identity. Map IdP groups to roles; create workload identities; remove static keys.
  5. Keys. Create the KMS key, policy and break-glass procedure; test unwrap from the data plane.
  6. Deploy. Apply the data plane with infrastructure as code; pin image digests and model versions.
  7. Verify. Run a smoke test, a load test and a failure test; confirm no public egress in flow logs.
  8. Cut over. Move traffic gradually with a rollback path and a named on-call owner.
# AWS: create an interface endpoint for the AI service
aws ec2 create-vpc-endpoint \
  --vpc-id vpc-0123456789abcdef0 \
  --service-name com.amazonaws.vpce.me-central-1.vpce-svc-0123456789abcdef0 \
  --vpc-endpoint-type Interface \
  --subnet-ids subnet-aaaa subnet-bbbb \
  --security-group-ids sg-0123456789abcdef0

# Verify the endpoint is available before changing DNS
aws ec2 describe-vpc-endpoints \
  --vpc-endpoint-ids vpce-0123456789abcdef0 \
  --query 'VpcEndpoints[0].State'

The service name and endpoint type differ by cloud; the pattern is the same: private path, DNS, application.

Latency and cost

PathRelative latencyWhat to measure
Same availability zoneLowestNetwork RTT, TTFT, p95
Cross-zone in regionLow, with a hopZone pair, p95, error rate
PrivateLink / PSCLow, endpoint hopEndpoint ENI, connection reuse
Cross-regionHighestRTT, data transfer, residency

Measure time to first token and total latency at the concurrency you expect, not a single warm request. Reuse connections; a new TLS handshake per request adds latency. Keep the client and endpoint in the same region.

Cost driverWhat to estimateWhy it grows
GPU capacityHours, reserved or on-demandIdle headroom for peak concurrency
PrivateLinkEndpoint hours and data processedPer-AZ endpoints and cross-zone traffic
StorageModel weights, logs, backupsMultiple versions and retention
ObservabilityLog volume, SIEM ingestionPrompt logging and long retention
OperationsPlatform and security staffPatching, key rotation, on-call

A private deployment usually costs more than SaaS before volume is high, because you pay for idle capacity and network plumbing. Model the crossover with real traffic, not a list price. See pricing for the hosted comparison and private AI endpoints for the endpoint model.

When SaaS is enough

Use the hosted API when the data is standard business content, the residency requirement is satisfied by the provider's region, and the team does not want to operate GPU capacity or a private network. Use a VPC when data cannot leave your cloud account, when a regulator or contract requires customer-controlled keys and audit, when you need a custom model or fine-tune that is not offered hosted, or when network latency to a public endpoint is the bottleneck.

A VPC is not automatically more secure. It moves responsibility to your team: patch the data plane, rotate keys, watch the logs and test failover. If you cannot staff that, a private endpoint to a managed service may be the better middle ground. For the decision framework, read when to deploy AI in your own VPC.

FAQ

What is the difference between VPC peering and PrivateLink?

Peering connects two VPCs bidirectionally and requires non-overlapping CIDRs and route tables. PrivateLink exposes a specific service through an endpoint, is unidirectional and avoids CIDR overlap. Peering is simpler for a full network relationship; PrivateLink is narrower and easier to govern.

Where do the encryption keys live?

In a customer-controlled KMS or HSM. The provider's control plane should call your key service through a private endpoint and never hold the plaintext key. Verify the key-wrap protocol, regions and operator access in the contract.

How much latency does PrivateLink add?

It depends on the region, availability zone and path. PrivateLink adds a network hop compared with same-zone peering, but avoids the public internet. Measure time to first token and p95 in your own VPC rather than relying on a generic number.

Is a VPC deployment automatically HIPAA compliant?

No. Network isolation is one control. A BAA, access review, audit logging, retention terms and a security assessment are separate. Verify the current terms in the contract with counsel; this page is not a certification claim.

Sources

AWS PrivateLink documentation for interface endpoints and service access, checked 20 November 2026: docs.aws.amazon.com/vpc/latest/privatelink. AWS VPC peering guide for CIDR and routing constraints, checked 20 November 2026: docs.aws.amazon.com/vpc/latest/peering. Azure Private Link overview, checked 20 November 2026: learn.microsoft.com/azure/private-link. Google Private Service Connect overview, checked 20 November 2026: cloud.google.com/vpc/docs/private-service-connect.

Reviewed 20 November 2026. Network behaviour and deployment scope were checked against the hyperscaler documentation and the Plugsky enterprise pages on that date. Latency and cost depend on your region and traffic; measure before committing. Regulated claims require contract verification and counsel review.