Key facts
| Loop primitive | POST /v1/chat/completions with tools and tool_choice |
| Tool calling | OpenAI-style function calling is live on supported models |
| Streaming | Server-sent events supported on the same endpoint |
| Memory | Message history for short-term context; /v1/embeddings with plugsky-embed for recall |
| Model routing | 30+ models behind one endpoint; mix plugsky-micro for cheap steps with frontier models |
| CLI agent | The Plugsky CLI ships an agent loop with file editing, a shell sandbox, MCP and RAG indexing |
| Endpoint status | Chat, streaming, JSON mode and function calling are live; assistants and responses endpoints are coming soon |
| Free tier | plugsky-micro and plugsky-lite free, no card; 14-day full-access trial available |
TL;DR
- The agent is your loop, not a vendor SDK: tools in, tool_calls out, results appended.
- Use plugsky-micro or plugsky-lite for parsing and routing, frontier models for hard reasoning.
- Store long-term memory as embeddings in your own database so recall stays portable.
- Stream tokens for responsiveness and keep tool execution server-side.
- Build on chat completions today; assistants-style endpoints are still coming soon.
How it works, step by step
- Create a free Plugsky account and generate an API key for local development.
- Define two or three tools with strict JSON schemas — search, database lookup, ticket creation — and mark side-effecting tools as explicit actions.
- Send the conversation plus the tools array and handle the returned tool_calls in a loop with a maximum iteration count.
- Add short-term memory through the message history and long-term recall by embedding documents with plugsky-embed and querying your vector store.
- Stream responses to the client and log every request ID, model alias and token count for debugging.
- Test edge cases: malformed tool arguments, tool timeouts, refusal paths and loops that hit the iteration cap.
- Wire evals into CI so prompt or model changes are compared against a fixed dataset before release.
Original data
Try it yourself
The agent loop, concretely
There is no separate agent runtime to learn. You send a messages array and a tools array to /v1/chat/completions. If the model decides to call a tool it returns tool_calls instead of a final answer; your code executes them, appends a tool message for each result, and calls again. The loop ends when the model returns a normal completion or when you hit a guardrail such as a maximum step count.
Keep the loop in your process. That is where authorisation lives: the model proposes an action, your code decides whether the caller may perform it. Never let a tool call bypass the permission checks of your own service layer.
Memory, retrieval and model routing
Short-term memory is the conversation itself — trim old turns or summarise them once the context grows. For long-term recall, generate embeddings with /v1/embeddings using plugsky-embed, store vectors in your own database, and retrieve the nearest chunks before prompting. Keeping the index on your side means recall survives any provider change.
- Route by step: classification and extraction on plugsky-micro or plugsky-lite; planning and synthesis on plugsky-pro or plugsky-frontier.
- Cache where safe: identical classification calls do not need a frontier model.
- Fail visibly: set timeouts on tools and return a structured error to the loop instead of hanging.
Testing and operating the agent
Agents fail differently from ordinary functions, so test their behaviours: tool argument validity, refusal correctness, recovery after a failed tool, and termination. Build a small dataset of representative tasks with expected tool sequences, and run it whenever you change a prompt or model alias. Plugsky exposes 30+ models behind one endpoint, which makes A/B comparisons cheap — swap the alias, rerun the suite.
In production, log request IDs, model alias, latency and token counts, and stream those events to your observability stack. When something misbehaves, the request ID lets you reconstruct exactly which model and tool sequence produced the outcome.
Honest comparison
| Concern | Plugsky agents | Assistants-style APIs | Framework agents |
|---|---|---|---|
| Build model | Your tool loop on chat completions | Managed threads and runs | Framework abstractions |
| Memory | You own history and vector store | Provider-managed threads | Framework-managed store |
| Model choice | 30+ models, mix per step | Vendor models | Whatever you wire up |
| Control | Prompts, loop and retries fully yours | Less control, faster start | Framework-dependent |
| Portability | OpenAI-compatible; change base URL | Vendor-specific shapes | Varies by framework |
| Status today | Live on chat completions | Varies by vendor | Live with the Plugsky endpoint |
Frequently asked questions
Do I need an agent SDK?
No. The loop is a few dozen lines against an OpenAI-compatible endpoint. Use the CLI or a framework if it speeds you up, but the protocol stays chat completions with tools.
How do I keep long-term memory?
Embed text with /v1/embeddings using plugsky-embed, store vectors in your own database, and retrieve relevant chunks into the prompt. You keep control of the index.
Which model should run the loop?
Use plugsky-micro or plugsky-lite for parsing, routing and extraction, and escalate planning or synthesis to plugsky-pro or plugsky-frontier. The same endpoint serves all of them.
Is streaming compatible with tool calls?
Yes. You can stream the assistant text while executing tools server-side; tool_calls still arrive as structured deltas you can assemble.
How do I stop runaway loops?
Cap iterations, add per-call timeouts, and require explicit confirmation for side-effecting tools. Return tool errors into the loop so the model can recover instead of retrying blindly.
Can I try this without paying?
Yes. The free plan includes plugsky-micro and plugsky-lite with no card, and a 14-day full-access trial exists for heavier testing. Plan details are on the live pricing page.
What about JSON mode?
JSON mode is live and useful for extraction steps, but it does not replace tool calling for actions. Use JSON mode for structured output, tools for execution.