Use Cases + Implementation

How do you build voice assistants with an OpenAI-compatible API?

Send the speech transcript to an OpenAI-compatible chat completions call and stream the reply back into text-to-speech. Use JSON mode for intent and slot extraction, function calling for actions, and the same SDK your text features use. Audio endpoints are coming soon, so pair the live chat API with your own STT and TTS providers and keep the assistant logic entirely in your service.

Key facts

EndpointPOST https://api.plugsky.com/v1/chat/completions with streaming (live)
CompatibilityKeep your OpenAI SDK; change base_url and model names
JSON modeLive for intent, slot and routing output
Function callingLive for bookings, lookups and account actions
Audio endpointsSpeech-to-text and text-to-speech are coming soon; bring your own today
PricingFlat monthly self-serve plans with no per-token charges on self-serve
Free tierplugsky-micro and plugsky-lite on the free plan, no card required
Models30+ models; short turns on cheap tiers, complex requests escalated

TL;DR

  • The assistant is a streamed chat completion driven by a transcript.
  • Use JSON mode to extract intent and slots before generating the reply.
  • Put account actions behind function tools with server-side checks.
  • Keep STT and TTS outside; audio endpoints are not live yet.
  • Cancel model work the moment the caller interrupts.

How it works, step by step

  1. Point your OpenAI-compatible client at Plugsky and verify a streamed chat completion works.
  2. Normalize the incoming transcript and extract intent and slots with a JSON mode call.
  3. Select a model tier per intent: cheap for small talk and status, stronger for troubleshooting.
  4. Generate the spoken reply with streaming and send complete sentences to TTS.
  5. Execute requested actions through function tools with permission checks and idempotency keys.
  6. Update a compact session memory with confirmed facts and pending actions.
  7. Log transcripts, model choices and tool calls for audit and replay.
1Point yourOpenAI-compatibleclient at Plugsky2Normalize theincoming transcriptand extract intent3Select a model tierper intent: cheapfor small talk and4Generate the spokenreply withstreaming and send5Execute requestedactions throughfunction tools with6Update a compactsession memory withconfirmed facts and

Try it yourself

Open the streaming API tester →

From transcript to streamed reply

The core loop is small: normalize the transcript, decide what the caller wants, generate a reply, speak it. Plugsky covers the language side with an OpenAI-compatible chat completions endpoint, so a team already using the OpenAI SDK changes the base URL and model names and keeps streaming, retries and tests. Speech itself stays with your STT and TTS vendors until audio endpoints ship.

Streaming is not optional for voice. Starting synthesis on the first complete sentence cuts perceived latency dramatically, and cancellation matters just as much: when the caller talks over the assistant, the model stream and the audio playback must stop.

JSON mode for intents and slots

Intent handling belongs before generation, not inside it. A small JSON mode call can return the caller's goal, the entities involved and a confidence value; your service then decides which model tier to use, which tools to expose and whether to answer at all. This keeps replies predictable and makes routing a data-driven decision rather than a prompt trick.

  • Schema: {intent, slots{}, confidence, needs_confirmation}.
  • Route low-confidence transcripts to clarification instead of guessing.
  • Read back names, dates and numbers before executing any action.
  • Keep the intent prompt small so it stays fast on cheap tiers.

What is live today and what is coming

Chat completions, streaming, JSON mode, function calling, embeddings, RAG and agents are live. Audio, images, moderation, files, batch, fine-tuning, assistants and responses endpoints are coming soon. That split defines the architecture: Plugsky is the brain and the tool layer, your integration supplies speech in and speech out, and your service owns sessions, permissions and audit.

Test with recorded calls, measuring recognition accuracy, intent accuracy, tool correctness and turn latency. Because compatibility is two-way, you can run Plugsky and your current provider side by side and compare on real traffic before cutting over. Start on the free plan with plugsky-micro and plugsky-lite, then use the 14-day full-access trial for stronger models on difficult calls.

Honest comparison

ConcernVoice assistant on PlugskyVoice platform with built-in modelsSelf-hosted assistant
Language layerOpenAI-compatible chat endpointPlatform-managedYou serve the model
Speech layerYour STT/TTS; Plugsky audio coming soonBuilt inYou host ASR and TTS
Intent handlingJSON mode extraction you controlPlatform NLUCustom classifiers
ActionsFunction calling with your checksPlatform integrationsCustom tool loop
Data controlCloud, VPC, on-prem, air-gappedVendor cloudFull control

Frequently asked questions

Can I use my OpenAI SDK code for a voice assistant?

Yes. The chat completions endpoint is OpenAI-compatible, so you change the base URL and model names and keep your streaming, retry and test code.

Where do speech-to-text and text-to-speech come from?

Your own providers for now. Plugsky audio endpoints are coming soon; the assistant brain, intent extraction and tool calls run on the live chat API.

How do I extract intent from a transcript?

Make a small JSON mode call that returns intent, slots and confidence, then route based on that output. Keep the schema tight so the call stays fast and cheap.

How do I handle interruptions?

Propagate cancellation: when caller speech is detected, abort the model stream, stop TTS playback and discard the partial reply from memory unless it was already confirmed.

Can the assistant perform account actions?

Yes, through function tools with server-side permission checks, confirmation of critical details and idempotency keys so retries cannot duplicate actions.

Which model should run the conversation?

Use plugsky-micro or plugsky-lite for routine turns and escalate troubleshooting or high-value interactions to stronger tiers. All 30+ models are behind one endpoint.

How do I test before launch?

Replay recorded calls through the pipeline, measure intent and tool accuracy and turn latency, then shadow live traffic against your current provider before cutting over.

Is there a free way to build the prototype?

Yes. plugsky-micro and plugsky-lite are on the free plan with no card, and a 14-day full-access trial is available for stronger models.