Key facts
| Tools shape | OpenAI-compatible tools array on /v1/chat/completions with tool_calls responses (live) |
| Parallel calls | Multiple tool calls can return in one response and execute concurrently |
| Determinism | tool_choice forces, allows or forbids tools; strict:true enforces schemas |
| Structured output | response_format json_object for guaranteed JSON alongside tool calls |
| Identity | Project-scoped keys, RBAC roles and OAuth 2.0 with PKCE for third-party apps |
| Pricing | Flat self-serve plans with unlimited fair-use usage; no per-token charges |
| Residency | Region can be pinned per request or globally for the workspace |
| Roadmap | Assistants and responses endpoints are coming soon; the chat completions tool loop is live |
TL;DR
- Keep your existing OpenAI SDK and framework code — change the base URL.
- Scope keys per service and tenant so no agent can cross account boundaries.
- Use strict schemas and JSON mode where your product parses the output.
- Flat plans keep cost per seat predictable as tool usage grows.
- Run schema tests in CI and watch tool-call traces in your logs.
How it works, step by step
- List the in-product actions the assistant may take, then define each as a typed JSON Schema tool with a narrow scope.
- Issue one scoped key per service and environment, and map tenant identity through your own orchestration layer.
- Implement the loop server-side: send tools, execute tool_calls, append role=tool messages, repeat until completion.
- Stream responses to the UI and render tool activity so users see what the assistant is doing.
- Add schema regression tests to CI using the function calling tester, and set tool_choice for steps that must not improvise.
- Monitor tool-call traces and token telemetry per tenant, then route expensive paths to cheaper tiers.
Try it yourself
Open the function calling tester →
The loop inside your product
An in-product assistant is a tool loop behind your API, not a chatbot bolted on top. Define tools such as lookup_subscription, create_ticket, fetch_usage and trigger_workflow, each with typed fields and descriptions the model can reason about. Send them in the tools array, execute whatever comes back, and return results as role=tool messages until the model answers.
Parallel tool calls matter for latency: a single response can request several lookups, and your service can dispatch them concurrently instead of serialising three round trips. Where your product parses the result directly, add JSON mode or strict schemas so malformed arguments fail in testing rather than in production.
Tenancy, identity and access
Tenant isolation is your responsibility, but Plugsky gives you the primitives: project-scoped keys, scoped permission lists such as chat:write, RBAC roles and rotation without downtime. Keep inference credentials server-side, never in the browser, and tag requests with a user identifier for support forensics.
- Per-request region: pin requests to a region for customers with residency requirements.
- Third-party apps: OAuth 2.0 with PKCE lets your app access a customer's own Plugsky workspace instead of your shared quota.
- Key hygiene: rotate quarterly; previous keys stay valid for 24 hours so deploys roll without a brownout.
Unit economics and reliability
Tool calling multiplies requests, and per-token billing punishes that pattern. Plugsky self-serve plans are flat with unlimited fair-use usage — token counts still return in each response so you can attribute cost internally without a variable invoice. That makes forecasting per-account gross margin straightforward.
For reliability, treat the model as another dependency: set tool timeouts, validate arguments against your schema, and surface tool errors back to the model so it can retry or ask the user. Assistants and responses endpoints are still coming soon, so build on the chat completions loop today and keep the migration path open with the OpenAI-compatible shape.
Honest comparison
| Concern | Plugsky | Vendor agent cloud | Rolling your own gateway |
|---|---|---|---|
| API compatibility | OpenAI-compatible chat completions | Vendor SDK required | Custom client code |
| Cost model | Flat monthly, unlimited fair use | Per-token or per-run | GPU plus platform cost |
| Tenancy primitives | Scoped keys, RBAC, OAuth with PKCE | Vendor account model | You implement all of it |
| Model choice | 30+ models behind one endpoint | Vendor catalogue | You integrate each provider |
| Time to ship | Days to first tool call | Weeks | Months |
Frequently asked questions
Do we need to rewrite our AI code?
No. The endpoint is OpenAI-compatible, so you change the base URL and model name and keep your SDK, framework and tool schemas.
How do we keep tenants separate?
Keep keys server-side, issue one scoped key per service, and map tenant identity through your own orchestration layer before any tool executes.
What are the rate limits?
Fair-use per-minute request limits apply per plan tier, not per model, and self-serve plans carry no per-token charges. See the live pricing page for current tiers.
Can customers bring their own Plugsky account?
Yes. Plugsky supports OAuth 2.0 authorization-code flow with PKCE, so your product can act inside a customer's own workspace.
Does streaming work with tool calls?
Yes. Stream the assistant's progress while tool calls execute server-side, then continue the conversation when results return.
Can we pin inference to one region?
Yes, globally for the workspace or per request for customers whose contracts require specific jurisdictions.
What should we watch in production?
Tool-call error rates, argument validation failures, token telemetry per tenant and latency percentiles. Every request is logged with model, tokens, latency, status, key ID, project ID and region.