Key facts
| Vision | Image and document input in chat completions (live) |
| Embeddings and RAG | Semantic search and built-in retrieval for policy and protocol corpora (live) |
| Audio | Transcription and speech endpoints with whisper-plugsky — coming soon |
| PII handling | No-PII mode auto-redacts, detect-only tags, passthrough for your controls |
| Residency | Region pinning; VPC, on-prem and air-gapped deployment |
| Retention | Request and response bodies are not stored; prompts are not used for training |
| Audit | Model, tokens, latency, status, key, project and region logged per request |
| Boundary | Administrative and documentation support only; clinicians decide care |
TL;DR
- Use vision for intake forms, cards and faxes — administrative, not diagnostic.
- Use RAG so staff answers cite policy and protocol documents.
- Redact PHI before inference and pin the workspace to one region.
- Keep audio transcription on the roadmap until endpoints ship.
- Keep clinicians accountable for anything that touches care.
How it works, step by step
- Pick an administrative workflow — intake extraction, prior-authorisation paperwork or policy search.
- Send document images through vision with strict schemas and validate every field.
- Index policies and protocols for retrieval so answers cite official documents.
- Redact identifiers before inference and pin inference, embeddings and logs to one region.
- Route extractions through staff review before they reach any patient record.
- Measure accuracy per document type and expand only when the error rate is stable.
Try it yourself
Open the embedding API tester →
Administrative multimodal workflows
The safe and valuable applications are administrative. Vision reads intake forms, insurance cards, referral letters and faxes, extracting structured fields for staff to verify. Embeddings and retrieval let front-desk teams ask policy questions and get answers that cite the actual document. Both capabilities are live and work with the same OpenAI-compatible endpoint.
None of this requires the model to reason about a patient's condition. Keeping the scope administrative is not timidity; it is what makes the deployment reviewable, because every output remains a form field or a cited policy rather than a judgment about care.
PHI and clinical boundaries
Redact first, ask questions later. No-PII mode filters and auto-redacts identifiers before inference; detect-only tags them; passthrough leaves controls to your application. Most clinical deployments combine no-PII with a gateway pass of their own, and they pin inference to a region so data stays in-jurisdiction.
- Retention: request and response bodies are not stored and prompts are not used for training.
- Review: staff verify every extracted field before it reaches a record.
- Deployment: VPC, on-prem and air-gapped options where patient data cannot leave the environment.
The audio roadmap and clinical notes
Ambient documentation is the obvious next step for healthcare, and it is exactly where the roadmap matters. Audio transcription via /v1/audio/transcriptions and speech via /v1/audio/speech, with whisper-plugsky planned for transcription, are coming soon. Do not design a note-generation pipeline around unshipped endpoints.
Prepare instead: keep your model calls behind one interface, define the consent and retention model for recorded encounters now, and pilot the workflow on typed input so the process is proven before the modality changes. When audio ships, only the input layer changes.
Honest comparison
| Capability | Plugsky status | Diagnostic use? | Review requirement |
|---|---|---|---|
| Form and card extraction | Live via vision | No — administrative | Staff verify fields |
| Policy search | Live via embeddings and RAG | No — reference | Cite source documents |
| Intake summarisation | Live via chat completions | No — documentation | Clinician reads summary |
| Audio transcription | Coming soon | Not applicable | Planned consent model |
| Speech output | Coming soon | Not applicable | Not recommended for care |
| Clinical decision support | Out of scope | Not claimed | Clinician judgement |
Frequently asked questions
Is multimodal AI diagnostic here?
No. Plugsky's live capabilities are used for administrative and documentation support — extraction, search and summarisation. Clinicians remain accountable for care decisions.
How is PHI protected?
Redact before inference with no-PII mode or your own gateway, pin the workspace to a region, and rely on the platform's no-retention behaviour for request bodies.
Can we transcribe clinical notes yet?
Not yet. Audio transcription is coming soon; prepare consent and retention models now and pilot with typed input meanwhile.
Are uploaded documents stored?
Uploaded RAG documents remain until you delete them; request and response bodies are not stored. Configure retention to match your policy.
Can it run on-prem?
Yes. VPC, on-prem and air-gapped deployments are available for patient data that cannot leave the environment.
How accurate is form extraction?
It depends on scan quality and your schema. Validate every field, keep staff verification, and measure accuracy per document type before expanding.
How should we pilot?
One administrative document type on the free plan with staff review, then scale once the measured error rate is acceptable. See the live pricing page for plan options.