What does a private AI reference architecture for banks look like?

Key facts

Inference boundary	Region pinning, VPC, on-prem or air-gapped deployment
Identity	SAML 2.0 / OIDC SSO and SCIM provisioning on Enterprise
Authorization	Workspace/role/resource RBAC with scoped per-service keys
Key custody	BYOK via AWS KMS, GCP KMS, Azure Key Vault, HashiCorp Vault or HSM
Data controls	no-PII, detect-only and passthrough modes; region pinning
Audit	Key/admin events plus per-request logs, exportable to SIEM
Resilience	Failover behaviour, multi-region options and tested DR
Compliance posture	SOC 2 Type II and ISO 27001 readiness in progress (not yet certified)

TL;DR

• Put inference inside the boundary your regulator expects — region, VPC or on-prem.
• Federate identity and automate deprovisioning before any rollout.
• Hold encryption keys yourself so revocation does not depend on a support ticket.
• Redact PII at the gateway and keep derived vectors in scope.
• Prove the architecture with configuration exports, audit samples and a DR test.

How it works, step by step

1. Classify workloads by data sensitivity and map each to a regulator.
2. Choose the isolation tier: in-region cloud, VPC, on-prem or air-gapped.
3. Connect SSO/SCIM and define roles for analysts, developers, operations and auditors.
4. Define key custody and revocation authority with your security team.
5. Set PII mode and gateway redaction policy per workload, and test on real data.
6. Wire audit export to your SIEM and define the alerting rules.
7. Run a DR rehearsal and a model-deprecation drill, then document the results.

Try it yourself

Open the sovereign AI readiness score →

The reference diagram in words

1. Client channels: branch, mobile, contact centre and back-office apps call your AI gateway, never the model directly.
2. Gateway: authenticates services with scoped keys, applies redaction policy, rate limits and quotas, and logs every request.
3. Inference boundary: an in-region Plugsky deployment, your VPC, or on-prem cluster, serving an OpenAI-compatible API.
4. State: vector stores, prompt history and logs placed deliberately with residency, retention and deletion rules.
5. Identity and keys: SSO/SCIM for people, RBAC for roles, BYOK/HSM for encryption keys.
6. Assurance: audit export to the SIEM, access reviews, model governance and human review for high-impact actions.

Controls a bank examiner will ask about

• Segregation of duties: who can deploy a model, change a prompt, grant a scope or revoke a key — and how those approvals are recorded.
• Data lineage: which documents trained or informed a response, and how the answer is reconstructed later.
• Model governance: an approved model list, evaluation gates before promotion and a deprecation path.
• Human oversight: mandatory review for credit, fraud, complaints and anything customer-facing with legal effect.
• Resilience: failover behaviour, capacity headroom and evidence that DR was tested rather than documented.
• Third parties: subprocessor inventory and flow-down expectations from your regulator.

Mapping to SAMA, CBUAE and DIFC expectations

Financial regulators in the region converge on similar themes: risk assessment before adoption, data protection, outsourcing controls, auditability and exit plans. Treat each expectation as a design input rather than a compliance annex. In practice that means choosing an isolation tier that keeps data inside the required boundary, documenting key custody, ensuring the vendor can provide audit evidence, and writing an exit plan that includes data export formats and deletion evidence. Plugsky's documented deployment matrix maps these needs to region-pinned, VPC, on-prem and air-gapped options behind the same OpenAI-compatible API.

The honest gaps to manage

Record what is not yet in place instead of glossing over it: SOC 2 Type II and ISO 27001 are readiness in progress rather than completed certificates, audio, images, moderation, files and fine-tuning endpoints are coming soon, and the final control mapping to your specific regulator is your review, not the vendor's claim. Mitigate with compensating controls — private deployment, BYOK, audit export, SSO/SCIM — and contractual milestones. Anchor commitments in /legal/terms and /legal/sla, and keep the risk register live through the first production quarter.

Honest comparison

Layer	Plugsky for banking	Hyperscaler AI platform	In-house model platform
Inference boundary	Region, VPC, on-prem, air-gapped	Cloud regions with dedicated options	Your data centres
Identity	SSO/SCIM and RBAC on Enterprise	Mature enterprise identity	You build and operate
Key custody	BYOK or HSM with revocation	Cloud KMS and HSM	You operate the HSM
Data controls	PII modes and region pinning	Varies by service	You implement everything
Audit	Request logs plus SIEM export	Native cloud audit plus SIEM	Custom pipeline
Certification status	SOC 2 / ISO 27001 readiness in progress	Completed audits in most regions	Your own programme

Frequently asked questions

Where does inference run in this architecture?

Inside the boundary your regulator expects: a pinned region for standard requirements, your VPC when the data must stay in your cloud account, or on-prem and air-gapped when the physical path is in scope for audit.

Can the bank keep its own encryption keys?

Yes. BYOK integrates with AWS KMS, GCP KMS, Azure Key Vault, HashiCorp Vault and HSMs, so rotation and revocation stay under the bank's control.

How is PII protected on the way to the model?

Redact in the gateway before the request leaves, choose the platform PII mode per workload, keep embeddings on no-PII mode for sensitive corpora, and include vectors in deletion scope.

What evidence should be collected for examiners?

Region and workspace configuration exports, audit samples with actor and timestamp, identity and role exports, key custody documentation, DR test results and the subprocessor list.

Is Plugsky certified for financial services?

SOC 2 Type II and ISO 27001 are documented as readiness in progress rather than completed certification. Controls such as SSO/SCIM, RBAC, BYOK, audit export and private deployment can support your review, but certification status must be recorded as pending.

How do we handle model risk governance?

Maintain an approved model list, run evaluation gates before promoting a model or prompt change, log which model answered each request, and rehearse a model-deprecation drill so migration is not a surprise.

What about resilience and DR?

Define failover behaviour explicitly, keep capacity headroom within the permitted region, and test disaster recovery. Plugsky offers multi-region options and documents failover semantics in the deployment guide.

Can we run a pilot without exposing production data?

Yes. Pilot with non-production data, verify region configuration and audit events, test the deprovisioning path, then move to production with a defined cutover and rollback plan.