Key facts
| API surface | OpenAI-compatible /v1/chat/completions; keep your existing SDK and tooling |
| Deployment boundary | Dedicated VPC, on-prem or air-gapped; model serving inside your boundary |
| Data path | No prompts, documents or embeddings sent to public consumer model providers |
| Key custody | Scoped API keys, rotation and BYOK options for enterprise deployments |
| Access control | Usage analytics plus enterprise SSO and RBAC options |
| Auditability | Request and response logging for model, security and compliance review |
| Models | 30+ models behind one API, from free tiers to frontier |
| Free tier | Free plan with plugsky-micro and plugsky-lite; 14-day full-access trial |
TL;DR
- Ground banking answers in approved internal content with RAG instead of model memory.
- Keep customer or transaction content inside VPC, on-prem or air-gapped deployments where policy requires it.
- Scope retrieval per team, client or site so permissions and confidentiality hold at query time.
- Keep a named human owner for every customer or financial decision.
- No prompts, documents or embeddings leave your boundary in a private deployment.
How it works, step by step
- Map where the data lives today: core banking, policy stores, document management and service desks.
- Choose the deployment boundary with security and platform teams: dedicated VPC, on-prem or air-gapped.
- Stand up an OpenAI-compatible endpoint inside that boundary and confirm no traffic egresses to public providers.
- Define key custody and BYOK requirements with security, audit and compliance.
- Pilot on internal policy and procedure content before any customer data is involved.
- Build an evaluation set with risk, compliance and operations reviewers.
- Document the controls, review logs and scale to additional teams.
Original data
Try it yourself
Open the private LLM deployment estimator →
Where banking teams start
Start with internal questions that already have a written answer. High-value first workloads include:
- Policy and procedure search: grounded answers over internal lending and operations policies.
- KYC and AML support: assist analysts with cited procedures inside your environment.
- Customer service assist: draft replies from approved product and policy content.
- Engineering and architecture Q&A: answer questions from internal platform and security documentation.
Each use case augments staff with cited answers; none replaces accountable control owner judgment.
A private RAG architecture for banking knowledge
The stack is consistent across industries: ingest approved internal policies, procedure manuals, product documentation and service-desk knowledge, chunk and embed with a multilingual embedding model, store vectors inside your environment, and call chat completions that answer only from retrieved context. Plugsky embeddings and chat completions are OpenAI-compatible, so teams already using OpenAI SDKs change the base URL and keep their code.
The reference architecture is an OpenAI-compatible endpoint inside your boundary: ingestion, embeddings and a vector store in the same VPC or network segment, with the model served there too. Change the base URL, keep the SDK, and route all traffic through your egress controls.
Access control, confidentiality and audit
The control that matters most is the data path: prompts, documents and embeddings must never leave the chosen boundary or reach a public consumer model endpoint. Enforce egress rules, keep key custody under your policy including BYOK options, and retain audit logs for model, security and compliance review.
The technical controls are consistent: enforce permission-aware retrieval in your own service layer, scope API keys per application, team or tenant, rotate keys, and retain request and response logs on a defined schedule. Regulatory obligations vary by jurisdiction and sector, so map them with counsel rather than assuming one framework covers every deployment; Plugsky supplies the deployment and logging primitives you document.
See private AI endpoint architecture for related deployment and control detail.
Rollout and human oversight
Pilot on internal policy and procedure content before touching customer or transaction content. Build a labelled question set with risk, compliance and platform engineers, then measure retrieval hit rate, citation correctness and answer accuracy before and after every index or model change. Require citations on every answer, refuse out-of-scope questions, and name a human owner for every customer or financial decision.
Review logged interactions weekly at first, correct the index rather than the prompt when retrieval misses, and expand the corpus only when accuracy and access checks pass.
Honest comparison
| Capability | Plugsky | Public AI assistants | Building in-house |
|---|---|---|---|
| Data boundary | Cloud, VPC, on-prem or air-gapped | Vendor cloud only | You control fully |
| Grounding | Embeddings and RAG are live for private knowledge search inside your boundary | Uncontrolled retrieval | You assemble and operate |
| Access control | Scoped keys, usage analytics, enterprise SSO and RBAC options | Account-level only | Custom identity work |
| Auditability | Request and response logging | Limited | You build logging |
| Pricing | Flat monthly self-serve plans; see live pricing | Per-seat or per-token | GPU plus operations cost |
| Time to pilot | Days | Hours, without residency control | Quarters |
Frequently asked questions
Can banks keep data private with Plugsky?
Yes. Choose the deployment boundary that matches the data: Plugsky cloud for public content, or your VPC, on-prem and air-gapped options for restricted material. Access is controlled with scoped API keys and usage analytics.
Do we need to fine-tune on our internal documents?
Not for a first release. Fine-tuning is coming soon and is better for style than facts. RAG keeps answers current, permission-aware and traceable to a source, which matters more for internal knowledge.
Which model should we use?
Start free with plugsky-micro and plugsky-lite to validate retrieval, then evaluate mid-tier and frontier models from the 30+ model catalogue on your own question set.
How do we stop wrong or unsupported answers?
Restrict the assistant to approved indexed content, require citations, refuse out-of-scope questions and keep a human decision-maker for every regulated or client-facing outcome.
How is pricing structured?
Self-serve plans are flat monthly with unlimited fair-use usage, and there are no per-token charges on self-serve plans. See the live pricing page for current plans and the free tier.
Does any data reach a public model?
No. In a private deployment, prompts, documents and embeddings stay in your VPC, on-prem or air-gapped environment, and the model serving endpoint runs inside that boundary. Review your egress rules and deployment diagram to confirm.
Can we keep our own encryption keys?
BYOK options exist for enterprise deployments. Confirm current capabilities and key-handling details with the Plugsky team during architecture review.