Key facts
| Telco fit | In-perimeter inference for sovereign sites; one API for all systems |
| Deployment | Same OpenAI-compatible API in your network, private cloud or air-gapped site |
| Data path | Call records, documents and logs stay inside the telco perimeter |
| Models | 30+ models; bundles licensed per deployment agreement |
| Live endpoints | Chat, streaming, function calling, JSON mode and embeddings |
| Identity | Scoped keys per OSS, BSS and care system; SSO with SCIM for staff |
| Audit | Authentication, key lifecycle and admin events exportable to SIEM |
| Pricing | Flat monthly self-serve plans; enterprise deployment scoped on the pricing page |
TL;DR
- Run inference inside the network where sovereignty, latency or classification requires it.
- Keep one OpenAI-compatible API so care, billing and network systems share an integration pattern.
- Issue scoped keys per system and environment, with central rotation and revocation.
- Plan capacity for peak-hour concurrency and dual-site failover from the start.
- Own the release calendar: platform and model bundle updates need telco change control.
How it works, step by step
- Identify the workloads that must stay in-perimeter — call analytics, field operations, network assurance, fraud review — and the sites they serve.
- Design network zones and egress rules so only approved systems reach the model endpoint and only approved destinations are reachable from it.
- Size capacity by peak-hour concurrency and long-context workloads, with headroom for upgrades and a second site.
- Standardise identity: scoped keys per system and environment from the telco secret store, SSO with SCIM for staff, RBAC for policy changes.
- Integrate observability: authentication, key and admin events to the SOC SIEM, plus performance and queue metrics for capacity reviews.
- Agree the platform and model bundle release calendar, and rehearse rollback before first production traffic.
- Pilot one site and one workflow, prove handling-time and accuracy gains, then replicate through the same runbook.
Try it yourself
Open the LLM GPU capacity calculator →
Why telcos self-host AI
Telecoms carry data that regulators and customers treat as sensitive: call detail records, location-adjacent telemetry, network configuration and field imagery. When rules or policy prevent external processing, on-prem deployment keeps inference inside the perimeter without giving up the applications, because the API surface stays OpenAI-compatible.
There is a latency argument as well. Network assurance and field tools run close to operations, and an in-perimeter endpoint removes public-internet variance from the response budget. The same deployment can serve multiple business units behind one gateway.
Reference deployment for a telco
Place the platform behind an internal gateway in a controlled zone, reachable only from approved system subnets. Applications call the OpenAI-compatible endpoint with scoped keys; retrieval uses embeddings against a telco-hosted vector store; function calling reaches internal APIs through the same gateway policy.
- Capacity: peak-hour concurrency, long-context requests, upgrade headroom.
- Resilience: dual-site deployment with tested failover and key availability in both sites.
- Identity: scoped keys for systems, SSO and SCIM for staff, RBAC for operators.
- Evidence: auth, key, admin and inference events to the SOC SIEM.
Operations, cost and precise scope
On-prem shifts the operational burden to the telco: patching windows, model bundle updates under the licensing agreement, capacity reviews and incident response. Budget for those alongside infrastructure, and compare the total against VPC deployment and flat monthly plans on the live pricing page; self-hosting is chosen for control, not automatically for savings.
Keep capability statements exact for internal stakeholders: chat, streaming, JSON mode, function calling and embeddings are live in every deployment, while audio, images, files, batch, moderation, fine-tuning and assistants endpoints are labelled coming soon. Multi-modal pipelines should keep their extraction stage swappable until those endpoints ship.
Honest comparison
| Concern | Plugsky on-prem | VPC deployment | Shared cloud API |
|---|---|---|---|
| Data boundary | Telco network or air-gapped site | Telco cloud tenancy | Vendor regions |
| Latency profile | LAN-local to operations | Region-local | Public-internet dependent |
| Capacity owner | Telco platform team | Shared with provider | Vendor |
| Change control | Telco release calendar | Provider plus telco | Vendor cadence |
| Best fit | Sovereign, high-utilisation, latency-critical | Regulated with cloud allowance | General internal workloads |
Frequently asked questions
Does the API differ from the cloud version?
No. The on-prem deployment exposes the same OpenAI-compatible API, so applications, SDKs and evaluations do not change when you relocate a workload.
How do we plan capacity?
Size on peak-hour concurrency and worst-case context length, add headroom for the next model upgrade, and duplicate capacity at the secondary site for failover.
Who owns upgrades?
The telco does, on an agreed calendar. Platform and model bundle releases arrive under the licensing agreement, and rollback must be tested before production changes.
How do we integrate with OSS and BSS?
Use function calling so the model emits tool calls your middleware executes against approved internal APIs, keeping credentials and permissions inside telco systems.
What audit evidence is available?
Authentication, key lifecycle, administrative and inference events export to the SOC SIEM, giving one reconstruction path across business units and sites.
Can some workloads stay on cloud?
Yes. Treat on-prem as a topology for the workloads that need it, and keep lower-sensitivity workloads on cloud or VPC with the same integration pattern.
How should we start?
One site, one workflow, one gateway. Prove handling-time and accuracy gains before replicating the runbook across markets.