Key facts
| Agent pattern | Orchestrator with specialised workers and typed handoffs |
| Runtime | Function calling loops on /v1/chat/completions (live) |
| Model routing | plugsky-micro and plugsky-lite for workers; larger models for planning and synthesis |
| Models | 30+ models behind one OpenAI-compatible endpoint |
| Shared state | Held in your orchestration layer, not in model context |
| Guardrails | Turn limits, tool budgets and schema validation per agent |
| Observability | Per-agent traces with model, tools and token metadata |
| Deployment | Region-locked planes, VPC, on-prem and air-gapped options |
TL;DR
- Start with two agents — one planner and one worker — before adding specialists.
- Hand off structured payloads, not transcripts, so context stays small and debuggable.
- Keep shared state in your code; the model should not be the database.
- Budget turns and tool calls per agent to bound cost and runaway loops.
- Trace every agent turn so failures can be attributed to a specific step.
How it works, step by step
- Write the task decomposition by hand first: which steps are independent, which need tools, and where human review belongs.
- Define the agents — typically planner, two or three workers and a synthesiser — with a single responsibility each.
- Specify typed handoff payloads so workers receive exactly the fields they need rather than full conversation history.
- Implement each agent as a function-calling loop with its own tool list, turn cap and timeout.
- Keep shared state in your orchestration layer and pass references into prompts instead of serialising everything.
- Add validation at each handoff and a synthesis step that merges worker outputs with citations or evidence.
- Trace every turn and evaluate the pipeline end to end on a fixed set of tasks before expanding.
Try it yourself
Open the multi-agent workflow designer →
Orchestrator and specialists
Multi-agent systems earn their complexity when a task has genuinely different sub-skills: research, extraction, calculation, writing. A single agent carrying every tool and instruction becomes hard to debug and expensive to run. Splitting responsibilities gives each agent a small tool surface, a clear prompt and measurable output.
The mechanics stay simple on Plugsky. Each agent is a function-calling loop on the live chat completions endpoint: pass a tools array, execute returned tool_calls, append results, repeat. The orchestrator is your code, which is where routing, budgets and state belong.
Handoffs, shared state and guardrails
Most multi-agent failures are integration failures. Passing full conversation history between agents bloats prompts and hides real inputs; passing nothing loses context. Define a typed handoff — a JSON object with the fields the next agent needs — and validate it before use. Keep shared state in your orchestration layer and reference it rather than embedding it.
- Turn and tool budgets: cap each agent so one worker cannot consume the whole run.
- Timeouts and retries: fail a worker cleanly and let the orchestrator decide whether to continue.
- Idempotency: key side-effecting tools so retries do not duplicate actions.
- Validation: reject malformed handoffs instead of forwarding them downstream.
Routing, tracing and evaluation
Cost control comes from routing. Workers doing classification or extraction can run on plugsky-micro or plugsky-lite, while planning and final synthesis use stronger models. With 30+ models behind one endpoint, each agent's model is a configuration value you can tune without changing the architecture.
Observability must be per agent: model, prompt version, tools called, arguments, latency and token usage for every turn. When output is wrong, the trace shows whether the planner decomposed badly, a worker retrieved the wrong data, or synthesis lost a fact. Evaluate the pipeline end to end on fixed tasks, and add each failure as a regression case. Deploy in a region-locked plane, or VPC, on-prem or air-gapped where data policy requires it — the architecture is unchanged.
Honest comparison
| Concern | Plugsky multi-agent | Single mega-agent | Managed agent platform |
|---|---|---|---|
| Structure | Orchestrator plus specialists in your code | One prompt, many tools | Vendor-defined graphs |
| Model choice | 30+ models, route per agent | One model for everything | Vendor catalogue |
| State | Owned by your orchestration layer | Inside model context | Vendor-managed threads |
| Guardrails | Turn budgets and validation per agent | Hard to isolate | Vendor features |
| Observability | Per-agent traces exportable to SIEM | Single log stream | Vendor-controlled |
Frequently asked questions
Do I need multiple agents?
Only when sub-tasks need different tools, prompts or models. If one well-scoped tool loop solves the problem, adding agents increases cost and debugging effort without adding value.
How do agents hand work to each other?
Through typed payloads produced and consumed by your orchestration code. Avoid forwarding raw transcripts; pass the fields the next agent needs and keep full state in your layer.
How do you stop runaway loops?
Cap turns and tool calls per agent, set timeouts, and require the orchestrator to approve additional work. Budgets are enforced in code, not requested in prompts.
Is function calling live?
Yes. OpenAI-style function calling and streaming are live on chat completions, which is the primitive each agent loop is built on.
Which models should each agent use?
Small models for classification and extraction, stronger models for planning and synthesis. 30+ models share one endpoint, so this is a configuration change per agent.
How do we debug a wrong answer?
Per-agent traces with model, tools, arguments and token counts. They show whether the failure came from decomposition, retrieval, a tool call or synthesis.
Can this run on-prem?
Yes. Plugsky supports region-locked cloud planes plus VPC, on-prem and air-gapped deployment, and the orchestration layer stays in your environment either way.