Key facts
| Endpoint | POST https://api.plugsky.com/v1/chat/completions — same shape as OpenAI (live) |
| JSON mode | Live for deterministic decision objects per workflow step |
| Function calling | Live for executing workflow actions with strict schemas |
| Compatibility | Keep your SDK; change base_url and model names |
| Models | 30+ models; cheap tiers for classification, stronger for exceptions |
| Pricing | Flat monthly self-serve plans with no per-token charges on self-serve |
| Free tier | plugsky-micro and plugsky-lite on the free plan, no card required |
| Roadmap | The batch endpoint is coming soon for offline workflow steps |
TL;DR
- Insert one chat completions call where the workflow needs judgement.
- Demand JSON decisions and validate them before executing steps.
- Expose actions as function tools with least-privilege credentials.
- Make steps idempotent and log every decision for replay.
- Keep the model out of deterministic steps entirely.
How it works, step by step
- Choose the workflow step that actually needs judgement, usually classification or exception routing.
- Define a JSON schema for the decision: action, category, confidence and required fields.
- Call chat completions with JSON mode and the case context, then validate the response in code.
- Map validated decisions to workflow branches and execute the matching step.
- Expose side-effecting actions as function tools with permission checks and idempotency keys.
- Add retries with backoff for transient errors and a dead-letter queue for repeated failures.
- Log prompt version, model, decision, tool calls and outcome, and replay historical cases after changes.
Try it yourself
A webhook-to-decision step
Most workflows already have an orchestrator: a queue, a rules engine, a scheduler. The LLM step slots in where branching is fuzzy. A webhook arrives, your orchestrator assembles the case context, calls chat completions with JSON mode, validates the returned decision and continues down the matching branch. Nothing else about the workflow changes, which is why OpenAI compatibility matters — the call is the same shape your team already writes.
Keep the context tight: only the fields the decision needs. Sending the whole record wastes tokens, dilutes accuracy and widens the blast radius of any prompt-injection content inside the payload. Treat inbound text as untrusted and strip anything that looks like instructions.
Schemas, validation and retries
Never execute a model decision that has not passed a schema check. Validate types, enums and required fields; reject unknown actions; and treat missing required data as a clarification step rather than a guess. When validation fails, retry once with the validation error appended, then route to a human queue.
- Pin the schema version so workflow code and prompts cannot drift apart.
- Cap
max_tokensto the size of the decision object. - Give every action an idempotency key derived from the case id and step.
- Record the raw response for failed validations; they are your best prompt-improvement signal.
Reliability and observability
Automation earns trust through boring reliability. Retry transient failures with jitter, queue work instead of dropping it, and give each run a state machine so a crash leaves recoverable state rather than a half-finished process. Track decision accuracy, validation failure rate, retry rate and human override rate per step, because aggregate metrics hide which branch is weak.
Roll out by shadowing: run the LLM step in parallel with the existing rule, compare decisions, and only then let it drive execution. Because the endpoint is OpenAI-compatible, you can compare providers on the same prompts before committing. Start on the free plan with plugsky-micro and plugsky-lite, then move to paid tiers or the 14-day full-access trial as volume grows; current plans are on the live pricing page.
Honest comparison
| Concern | Plugsky API step | Vendor automation builder | Custom ML model |
|---|---|---|---|
| Integration | OpenAI SDK call in existing orchestrator | Platform-hosted flows | Train and deploy a model |
| Decision format | JSON mode validated in code | Platform node types | Whatever you build |
| Model choice | 30+ models behind one endpoint | Vendor models | One bespoke model |
| Action safety | Function tools with your permission checks | Platform connectors | Custom |
| Change cost | Prompt and schema updates | Rebuild flows in a console | Retrain and redeploy |
Frequently asked questions
Where does the LLM fit in an existing workflow?
At the step that needs judgement, typically classification, prioritisation or exception routing. The model returns a decision, your orchestrator validates it and executes the rest, so deterministic steps stay in code.
Why use JSON mode?
It makes the decision machine-checkable: types, enums and required fields can be validated before any action runs, which prevents prose parsing from breaking the workflow.
How do I handle invalid model output?
Validate against the schema, retry once with the validation error appended, then route to a human queue. Log every invalid response; they point to prompt or schema problems.
Are function tools required?
Only if the model should trigger actions directly. Many teams prefer the model to return a decision and let the orchestrator execute, which keeps permissions and side effects in one place.
How do I make retries safe?
Use idempotency keys derived from the case id and step, and make each action's server-side handler idempotent. Retries then cannot duplicate payments, emails or records.
Which model should handle which step?
Route routine classification to plugsky-micro or plugsky-lite and escalate complex or high-value cases to stronger tiers. All 30+ models sit behind one OpenAI-compatible endpoint.
Can I migrate existing OpenAI code?
Yes. Change the base URL and model names and keep your prompts, validation, retries and tests. The compatibility also makes reverting a one-line change.
Can I start for free?
Yes. The free plan includes plugsky-micro and plugsky-lite with no card, and a 14-day full-access trial is available for stronger models.