Short answer. Put the inference data plane in your VPC, expose it through AWS PrivateLink, Azure Private Link or Google Private Service Connect, and remove the default internet route. Use workload identity, customer-managed keys and private observability. Measure latency in your own region. A VPC is an architecture, not a compliance certificate; verify the contract and counsel-review the regulated claims.
Peering versus PrivateLink
| Property | VPC peering | PrivateLink / PSC |
|---|---|---|
| Direction | Bidirectional routes | Consumer to a specific service |
| CIDR | Must not overlap | No overlap required |
| Scope | Whole VPC relationship | One service endpoint |
| Governance | Route tables and security groups | Endpoint policy and service acceptance |
| Best for | Full network integration | Narrow, auditable service access |
Peering is simpler when both sides need broad access and the CIDRs are planned. PrivateLink is easier to defend when a regulator asks what the AI provider can reach: the consumer can reach one endpoint, not the whole VPC. Peering is not transitive; a hub-and-spoke design needs a transit gateway or a mesh. Choose the narrowest path that still meets the latency target.
Egress control
| Control | What it prevents | How to verify |
|---|---|---|
| No internet gateway or NAT | Default outbound path | Route table audit; flow logs |
| VPC endpoints | Public S3, KMS, logs traffic | Endpoint policy and DNS resolution |
| Allowlisted proxy | Unapproved package downloads | Proxy logs; deny-by-default |
| DNS controls | Resolution to public services | Private hosted zones; resolver rules |
| Network firewall | Lateral movement and exfiltration | Policy hits; denied flows |
The inference path should not need the internet. If the model container pulls packages at startup, pin the image and mirror the registry inside the VPC. If observability exports to a SaaS endpoint, use a private endpoint or a local collector. A VPC that still has a NAT gateway for “just updates” is a private network with an exit.
Identity, keys and observability
| Layer | Pattern | What to avoid |
|---|---|---|
| User identity | SAML or OIDC through your IdP; group mapping | Local accounts per team |
| Workload identity | IAM roles for service accounts, managed identity, workload identity federation | Static keys in environment variables |
| Service-to-service | mTLS; short-lived tokens; per-team keys | One shared key for every service |
| Key custody | Customer KMS/HSM; envelope encryption | Provider-held key with no audit trail |
| Observability | VPC flow logs, audit logs, metrics and traces to your SIEM | Payload logging on by default |
Use separate keys per environment and per region, and keep the key policy in infrastructure as code. Key custody is covered in more depth in what BYOK means for AI; the short version is that a VPC does not protect data if the provider can unwrap the key without your policy. For observability, agree on what is logged: request metadata is usually enough; prompt and response bodies should be opt-in and retained deliberately.
Deployment steps
- Landing zone. Confirm accounts, regions, CIDRs and the network account that owns the shared services.
- Private path. Create the peering or endpoint, accept the service, and update route tables and security groups.
- DNS. Resolve the API hostname to the private endpoint; test from a workload subnet, not a laptop.
- Identity. Map IdP groups to roles; create workload identities; remove static keys.
- Keys. Create the KMS key, policy and break-glass procedure; test unwrap from the data plane.
- Deploy. Apply the data plane with infrastructure as code; pin image digests and model versions.
- Verify. Run a smoke test, a load test and a failure test; confirm no public egress in flow logs.
- Cut over. Move traffic gradually with a rollback path and a named on-call owner.
# AWS: create an interface endpoint for the AI service
aws ec2 create-vpc-endpoint \
--vpc-id vpc-0123456789abcdef0 \
--service-name com.amazonaws.vpce.me-central-1.vpce-svc-0123456789abcdef0 \
--vpc-endpoint-type Interface \
--subnet-ids subnet-aaaa subnet-bbbb \
--security-group-ids sg-0123456789abcdef0
# Verify the endpoint is available before changing DNS
aws ec2 describe-vpc-endpoints \
--vpc-endpoint-ids vpce-0123456789abcdef0 \
--query 'VpcEndpoints[0].State'
The service name and endpoint type differ by cloud; the pattern is the same: private path, DNS, application.
Latency and cost
| Path | Relative latency | What to measure |
|---|---|---|
| Same availability zone | Lowest | Network RTT, TTFT, p95 |
| Cross-zone in region | Low, with a hop | Zone pair, p95, error rate |
| PrivateLink / PSC | Low, endpoint hop | Endpoint ENI, connection reuse |
| Cross-region | Highest | RTT, data transfer, residency |
Measure time to first token and total latency at the concurrency you expect, not a single warm request. Reuse connections; a new TLS handshake per request adds latency. Keep the client and endpoint in the same region.
| Cost driver | What to estimate | Why it grows |
|---|---|---|
| GPU capacity | Hours, reserved or on-demand | Idle headroom for peak concurrency |
| PrivateLink | Endpoint hours and data processed | Per-AZ endpoints and cross-zone traffic |
| Storage | Model weights, logs, backups | Multiple versions and retention |
| Observability | Log volume, SIEM ingestion | Prompt logging and long retention |
| Operations | Platform and security staff | Patching, key rotation, on-call |
A private deployment usually costs more than SaaS before volume is high, because you pay for idle capacity and network plumbing. Model the crossover with real traffic, not a list price. See pricing for the hosted comparison and private AI endpoints for the endpoint model.
When SaaS is enough
Use the hosted API when the data is standard business content, the residency requirement is satisfied by the provider's region, and the team does not want to operate GPU capacity or a private network. Use a VPC when data cannot leave your cloud account, when a regulator or contract requires customer-controlled keys and audit, when you need a custom model or fine-tune that is not offered hosted, or when network latency to a public endpoint is the bottleneck.
A VPC is not automatically more secure. It moves responsibility to your team: patch the data plane, rotate keys, watch the logs and test failover. If you cannot staff that, a private endpoint to a managed service may be the better middle ground. For the decision framework, read when to deploy AI in your own VPC.
FAQ
What is the difference between VPC peering and PrivateLink?
Peering connects two VPCs bidirectionally and requires non-overlapping CIDRs and route tables. PrivateLink exposes a specific service through an endpoint, is unidirectional and avoids CIDR overlap. Peering is simpler for a full network relationship; PrivateLink is narrower and easier to govern.
Where do the encryption keys live?
In a customer-controlled KMS or HSM. The provider's control plane should call your key service through a private endpoint and never hold the plaintext key. Verify the key-wrap protocol, regions and operator access in the contract.
How much latency does PrivateLink add?
It depends on the region, availability zone and path. PrivateLink adds a network hop compared with same-zone peering, but avoids the public internet. Measure time to first token and p95 in your own VPC rather than relying on a generic number.
Is a VPC deployment automatically HIPAA compliant?
No. Network isolation is one control. A BAA, access review, audit logging, retention terms and a security assessment are separate. Verify the current terms in the contract with counsel; this page is not a certification claim.
Sources
AWS PrivateLink documentation for interface endpoints and service access, checked 20 November 2026: docs.aws.amazon.com/vpc/latest/privatelink. AWS VPC peering guide for CIDR and routing constraints, checked 20 November 2026: docs.aws.amazon.com/vpc/latest/peering. Azure Private Link overview, checked 20 November 2026: learn.microsoft.com/azure/private-link. Google Private Service Connect overview, checked 20 November 2026: cloud.google.com/vpc/docs/private-service-connect.
Reviewed 20 November 2026. Network behaviour and deployment scope were checked against the hyperscaler documentation and the Plugsky enterprise pages on that date. Latency and cost depend on your region and traffic; measure before committing. Regulated claims require contract verification and counsel review.