How do you build coding agents with AI agents?

Key facts

Endpoint	POST /v1/chat/completions with tools for file read, patch, shell and test execution
Coding models	plugsky-coder and plugsky-coder-fast sit among 30+ models in the catalogue
Sandbox	Run each task in a disposable workspace with no network and no credentials
Approvals	Write and shell tools follow an explicit policy: auto, allowlist or human approval
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage
Free tier	Free plan with 2 free AI models, no card required
Governance	Scoped keys, audit logs and per-task traces
Roadmap	Fine-tuning and batch endpoints are coming soon

TL;DR

• Scope the agent to one repository and one task at a time.
• Give it tools, not freedom: read, search, patch, test, and nothing else by default.
• Run every task in a disposable sandbox with no network and no credentials.
• Require human approval for writes and shell commands beyond an allowlist.
• Evaluate by task success, checks passed and revert rate — not by activity.

How it works, step by step

1. Define the task contract: repository, target paths, acceptance tests and a time budget.
2. Implement read-only tools first — list, read, search, symbol lookup — and require no approval for them.
3. Add write tools as patches only, with path allowlists and human approval by default.
4. Run the loop in a disposable sandbox: plan, edit, test, read failures, repeat under an iteration cap.
5. Persist task state outside the transcript so restarts and compaction do not lose progress.
6. Review the final diff with check results, and record task outcome, edit distance and revert status.

Original data

Try it yourself

Open the agent workflow designer →

Anatomy of a coding agent

Build it from five parts, each independently testable:

• Context manager: decides what the model sees per step — open files, symbols and recent failures.
• Tools: list files, read file, search code, apply patch, run tests, run lint. Nothing broader by default.
• Planner: the model sequences steps and can be re-plan when evidence contradicts its assumptions.
• Execution sandbox: disposable workspace with pinned dependencies, no network and no secrets.
• Memory: durable task state — plan, files touched, checks run — stored outside the transcript.

Safety model before capability

The default posture should be deny:

• Patches are limited to allowed paths and a maximum diff size; anything wider is rejected, not truncated.
• Shell access is restricted to an allowlist of commands needed for checks, or disabled entirely.
• No outbound network from the sandbox, so generated code cannot exfiltrate data or fetch dependencies at run time.
• Credentials never enter the sandbox, and the model never sees repository secrets.
• Every task is revertible: apply changes on a branch and keep the diff as the artifact.
• Approval policy is explicit per tool class, with an audit trail of who approved what.

Implementation of the loop

Keep the loop boring and bounded:

1. Receive the task and load the acceptance tests; refuse tasks without a verifiable definition of done.
2. Let the model request reads and searches to build a plan.
3. Apply proposed patches in the sandbox, then run the fastest relevant checks first.
4. Feed raw failure output back for one correction pass per failure type.
5. Stop on green checks, repeated failure or budget exhaustion, then produce a diff summary for review.

Use plugsky-coder for edits, but a cheaper alias for summarization and log digestion between steps.

Evaluation and limitations

Measure the agent like a junior engineer with a stopwatch:

• Task success: acceptance tests passing without human intervention.
• Human edit distance: how much of the diff survived review unchanged.
• Revert rate: merged work later undone — the clearest signal of overconfidence.
• Safety incidents: out-of-scope writes or blocked commands, which must be zero.
• Cost per task: iterations and model work divided by successful tasks.

Honest limits: long-horizon refactors exceed what one session can hold, tests remain a weak oracle, and monorepos demand tight task scoping. Assistants and responses endpoints are coming soon, so agent state stays yours today.

Honest comparison

Capability	Coding agent on Plugsky	Chat assistant with code	Manual engineering
Execution	Sandboxed file, shell and test tools	Copy and paste	Full environment
Context	Per-step retrieval from the repo	Conversation window	Human understanding
Guardrails	Path allowlists, budgets, approvals	None	Process and review
Model choice	Coding models among 30+ aliases	One vendor model	Not applicable
Cost shape	Flat monthly self-serve, unlimited fair use	Per-seat	Salaries

Frequently asked questions

What is a coding agent?

A loop that reads repository context, plans a change, edits files and runs checks until the task is done or a budget is exhausted, with a human reviewing the resulting diff.

Do I need a sandbox?

Yes. Generated code is untrusted. Run tasks in a disposable workspace with pinned dependencies, no network and no credentials.

Which model should drive edits?

plugsky-coder for complex changes and plugsky-coder-fast for routine ones. Use cheaper aliases for log digestion and summarization between edit steps.

How do I keep long tasks from drifting?

Persist task state outside the transcript, re-plan when checks contradict assumptions, and cap iterations and wall-clock time.

Should the agent run shell commands?

Only inside an allowlist needed for checks, inside the sandbox. Everything else stays disabled.

What is the best success metric?

Acceptance tests passing plus human edit distance, tracked alongside revert rate. Activity metrics like files touched say nothing about value.

Can I evaluate coding agents on the free plan?

Yes. Two free models with no card cover prototyping, and the 14-day full-access trial lets you compare coding aliases on real repository tasks.