How do you build API orchestration with an OpenAI-compatible API?

Key facts

Endpoint	POST https://api.plugsky.com/v1/chat/completions with tools, tool_choice and JSON mode
Compatibility	The same request shape as OpenAI; keep your SDK and change base_url
Models	30+ models behind one endpoint; mix aliases per orchestration step
Validation	Server-side schema validation and allowlisting before any call executes
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage
Free tier	Free plan with 2 free AI models, no card required
Governance	Scoped keys, audit logs and region pinning for regulated workloads
Roadmap	Assistants and responses endpoints are coming soon

TL;DR

• Your orchestrator is a tool loop on chat completions — no new protocol required.
• Validate and allowlist every tool call before execution.
• Keep durable workflow state in your database, not the transcript.
• Route planning steps to stronger aliases and dispatch steps to cheap ones.
• OpenAI compatibility means your SDK, retries and test harness all carry over.

How it works, step by step

1. Choose the workflows where the call sequence depends on unstructured input.
2. Define tools with JSON Schema, split into read and write classes, and attach permissions per class.
3. Implement the loop against the OpenAI-compatible endpoint: send tools, execute tool_calls, append results.
4. Validate arguments and enforce allowlists before execution; add idempotency keys to writes.
5. Cap iterations, wall-clock time and total work per request; escalate only once on repeated failure.
6. Log the full trace and evaluate tool-call precision, task success and safety on a regression suite.

Original data

Try it yourself

Open the function calling schema generator →

Architecture of an API orchestrator

Three components, all in your process:

• Planner: the model call with tools, a system prompt and the conversation so far.
• Executor: a dispatcher that validates arguments, checks permissions, calls the internal API and returns a structured result.
• State store: durable records for workflow id, steps completed, inputs and outputs, so a restart never loses progress.

The endpoint is only the planner. Everything that touches a real system stays in code you control, which is also what makes the orchestration auditable and testable.

Implementation: the tool loop

Keep the loop explicit and bounded:

1. Send the goal, the tool registry and the policy prompt to plugsky-pro for planning-heavy workflows, or a cheaper alias for short flows.
2. When the response contains tool calls, validate each argument object against its schema and the caller's scopes.
3. Execute independent reads concurrently; serialize writes and attach an idempotency key derived from the workflow id.
4. Append results as tool messages — structured, truncated to what the model needs — then call the model again.
5. Stop on a final answer, a repeated error, or the iteration cap; persist state after every step.

Because the payload is OpenAI-shaped, a retry policy that already works for OpenAI works unchanged here.

Evaluation and safe rollout

Treat orchestration like any other production system with a model dependency:

• Regression suite: recorded goals with expected tool sequences; run it on every prompt, schema or alias change.
• Tool-call precision and recall: unnecessary calls waste time; missed calls break workflows.
• Safety counters: unapproved writes, duplicate writes and out-of-scope calls must be zero.
• Recovery rate: errors handled without human intervention, tracked per tool.
• Latency budget: measure model round trips separately from API time so you know which side to optimize.

Roll out by workflow, not by model: one workflow in production beats a platform-wide migration you cannot score.

Limitations and caveats

Function calling is live on supported models, but the pattern has real edges:

• Model behavior differs across the catalogue; verify tool-call reliability for each alias before promoting it to planner.
• Generated arguments can be plausible and wrong. Schema validation catches shape errors, not semantic ones — add business-rule checks in code.
• Long traces inflate context and cost; summarize completed steps rather than resending every payload.
• Coming-soon endpoints (assistants, responses, batch) are not part of this design; persist state yourself and revisit when they ship.
• Secrets never go into prompts. Tools call your services with server-side credentials.

Honest comparison

Capability	Plugsky tool-loop orchestrator	Agent framework	Hard-coded service calls
Protocol	OpenAI-compatible chat completions	Framework-specific	Native code
Planning	Model chooses the sequence	Model chooses, framework runs	Predetermined
Validation	Server-side before every call	Framework hooks	Wherever you added it
Model mixing	30+ aliases per step	Configured per agent	Not applicable
State	Your database	Framework or provider store	Application database

Frequently asked questions

Is this different from normal function calling?

It is the same mechanism applied to a multi-step goal: the model plans across several API calls, and your runtime executes and validates each one in sequence.

Which model should plan the workflow?

Use a stronger alias such as plugsky-pro for multi-step planning, and cheap aliases such as plugsky-micro or plugsky-lite for single-step dispatch and extraction.

How do I validate tool arguments?

Define JSON Schemas for every tool, validate before execution, and reject calls that reference unknown tools or parameters outside the caller's scope.

Can I run tools in parallel?

Yes for independent reads. Keep writes serialized per record and always idempotent, because a retried turn can resend the same call.

What happens when a tool fails mid-workflow?

Return a structured error to the model once, let it choose a recovery path, then stop and summarize for a human if it repeats.

Do I need the Assistants API?

No. Assistants and responses endpoints are coming soon; this pattern runs on live chat completions and keeps state in your database.

Can I start on the free plan?

Yes. Two free models with no card cover prototyping, and a 14-day full-access trial lets you evaluate stronger aliases for planning.