How do you build chatbots with an OpenAI-compatible API?

Key facts

Endpoint	POST https://api.plugsky.com/v1/chat/completions with streaming enabled for token-by-token replies
Compatibility	Same request shape as OpenAI; change base_url and model name
Tools	Function calling is live for lookups and account actions
Models	30+ models; route simple turns to plugsky-micro or plugsky-lite and complex turns upward
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage
Free tier	Free plan with 2 free AI models, no card required
Governance	Scoped keys, audit logs and region pinning
Roadmap	Moderation and audio endpoints are coming soon

TL;DR

• A chatbot is a streaming chat completions call plus state you control.
• Stream with SSE and cancel upstream work when the user disconnects.
• Trim history by tokens; summaries beat resending every turn.
• Tools turn a talker into a bot that can actually resolve issues.
• OpenAI compatibility keeps your SDK, retries and test harness unchanged.

How it works, step by step

1. Change the base URL to Plugsky and verify an existing chat request returns the familiar response shape.
2. Add a session store keyed by conversation id, with customer id, verified state and a rolling summary.
3. Enable streaming and proxy server-sent events to the client, handling cancellation on disconnect.
4. Trim history by token budget and store a summary of older turns for continuity.
5. Add function tools for account lookups and support actions, with permissions checked server-side.
6. Run a regression suite of recorded conversations before each prompt, tool or alias change.

Original data

Try it yourself

Open the streaming API tester →

Architecture of an API-first chatbot

The API call is one box in a small system:

• Client: widget or app that renders streamed tokens and sends conversation id on every turn.
• Session service: stores messages, verified identity, summary and tool results.
• Model gateway: selects the alias per turn, applies the system prompt and enforces limits.
• Tool layer: read and write tools with server-side permission checks.
• Telemetry: latency to first token, total turn time, tool calls and safety events.

Keeping the gateway in your service means channel changes and model changes never require client releases.

Implementation notes that prevent pain

Streaming and state are where chatbots usually break:

• Parse SSE deltas incrementally and forward them; do not buffer the full answer before rendering.
• Propagate client disconnect upstream so abandoned turns stop consuming capacity.
• Enforce a token budget per turn: system prompt, summary, recent turns and tool output, in that priority order.
• Summarize older turns with a cheap alias rather than dropping them silently — reference loss looks like forgetfulness.
• Keep tool schemas strict and validate arguments before execution; a chatbot with a sloppy refund tool is a liability.
• Set max_tokens per turn and stream a stop reason the client can render as 'continuing'.

Evaluation and operations

Score the chatbot on what users feel and what the business needs:

• Time to first token: the strongest driver of perceived responsiveness in chat.
• Turn success: did the user's request advance, judged on a labeled conversation set?
• Tool accuracy: correct tool, correct arguments, no unauthorized calls.
• Safety: prompt-injection attempts that changed behavior, and unsafe outputs, both tracked to zero tolerance.
• Session health: abandoned conversations, repeated questions and escalation triggers.

Replay recorded conversations after every change. A prompt edit that fixes one intent can silently regress ten others.

Limitations

Compatibility does not remove operational work:

• Moderation and audio endpoints are coming soon, so content filtering and voice remain your responsibility today.
• Conversation state is yours to store, secure and delete; design retention and deletion before launch.
• Context growth is the main long-session failure mode; summaries help but lose detail, so escalate when precision matters.
• Model aliases differ in instruction-following; re-run evaluations when you change aliases or prompts.
• Streaming complicates error handling: a connection can fail mid-answer, so clients need a recovery path.

Honest comparison

Capability	Chatbot on Plugsky API	Vendor-locked assistant platform	Custom model hosting
Integration	OpenAI-compatible SDK call	Platform SDK and hosting	Serve and scale models yourself
Streaming	SSE on chat completions	Platform-managed	Depends on your server
Tools	Function calling on supported models	Platform-defined actions	Build your own loop
Model choice	30+ aliases behind one endpoint	Vendor catalogue	Models you can host
Deployment	Cloud, VPC, on-prem, air-gapped	Vendor cloud	Your infrastructure

Frequently asked questions

Do I need to rewrite my chatbot to use Plugsky?

No. The chat completions endpoint is OpenAI-compatible, so you change the base URL and model names and keep your streaming, retry and test code.

How should I stream replies?

Enable streaming on the chat completion and forward server-sent events to the client, handling partial messages and cancellation.

Where does conversation state live?

In your own store. The model is stateless, so your service owns history, summaries, verified identity and retention rules.

How do I keep long conversations affordable?

Trim by token budget and summarize older turns with a cheap alias. Send tool results compactly instead of full payloads.

Can the chatbot take account actions?

Yes, through function tools with server-side permission checks. Never let the model decide entitlements from free text.

Is there a moderation endpoint?

Not yet — moderation is coming soon. Apply your own filters on input and output in the meantime.

Can I prototype for free?

Yes. plugsky-micro and plugsky-lite are available on the free plan with no card, and a 14-day full-access trial unlocks stronger aliases for evaluation.