Key facts
| Everyday coding | plugsky-coder — generation, edits and explanations |
| Hard reasoning | plugsky-max and plugsky-frontier for complex refactors |
| Budget tasks | plugsky-micro and plugsky-lite, both on the free plan |
| Large context | plugsky-longctx for long files and repositories |
| API | OpenAI-compatible for every model |
| Model count | 30+ models, one endpoint |
| Evaluation | Run your own evals on your repo — no universal leaderboard |
| Product status | Live |
TL;DR
- Use plugsky-coder as the default coding workhorse.
- Escalate complex refactors to a stronger reasoning model.
- Free models cover boilerplate, tests and autocomplete prototyping.
- Model names belong in configuration so switching is trivial.
- Your repository and test suite are the only benchmark that matters.
How it works, step by step
- Collect a representative task set from your repo: generation, bug fixing, refactoring, review.
- Run two or three candidate models on the same tasks with identical prompts.
- Score on tests passed, diff size, edit precision and review time needed.
- Pick a default and one escalation model, then pin both in configuration.
- Wire fallback so an unavailable model degrades to your alternate.
- Re-run the evaluation when models update or your stack changes.
Try it yourself
Open the best model for coding selector →
Start with a default, escalate deliberately
Most teams get the best economics from a two-tier setup. plugsky-coder handles the bulk of day-to-day work: writing functions, fixing bugs, explaining unfamiliar code and producing tests. When a task involves cross-file reasoning, architecture decisions or a large refactor, escalate to a stronger model such as plugsky-max or plugsky-frontier, where better planning reduces total attempts and review time. For high-volume, low-stakes work — boilerplate scaffolding and autocomplete — the free-tier models plugsky-micro and plugsky-lite keep costs near zero, including on the free plan.
How to evaluate coding models properly
Public benchmarks correlate weakly with your repository, language mix and conventions. Build a small private evaluation instead:
- Task set: 20-50 real tasks spanning generation, bug fixing, refactoring, tests and review.
- Prompts: identical across models, including the same context and constraints.
- Metrics: tests passed, correctness of edits, diff size, number of review comments, latency.
- Blind scoring: reviewers should not know which model produced which output.
Re-run the set when models update. A model that wins today can be replaced next month, so keep the harness cheap enough to repeat. The catalogue lists current options and aliases.
What we do and what we do not do
What we do: expose coding-capable models across tiers behind one OpenAI-compatible API, keep aliases stable so your configuration does not churn, and price self-serve plans flatly so experimentation does not produce surprise bills. What we do not do: publish a definitive 'best coding model' ranking — the honest answer depends on language, framework and task mix, and anyone claiming a universal winner is selling something. Treat model choice as a tunable, not a permanent decision: pin names in config, evaluate, and revisit.
Honest comparison
| Task | Suggested model | Why | Trade-off |
|---|---|---|---|
| Everyday generation and edits | plugsky-coder | Code-tuned and economical | Not the strongest at deep reasoning |
| Complex refactors | plugsky-max or plugsky-frontier | Better planning across files | Higher cost and latency |
| Boilerplate and autocomplete | plugsky-micro or plugsky-lite | Free-tier friendly | Limited nuance |
| Long files and repos | plugsky-longctx | Built for large context | Verify quality on your code |
| Test generation | plugsky-coder | Good at repetitive structure | Review edge cases |
| Architecture discussion | Strong reasoning model | Better trade-off analysis | Slower responses |
Frequently asked questions
Is plugsky-coder the best model for all coding tasks?
No — it is the best default for everyday work. Escalate complex, cross-file reasoning to a stronger model such as plugsky-max or plugsky-frontier.
Can I use the free models for coding?
Yes. plugsky-micro and plugsky-lite are free-plan models suited to boilerplate, simple fixes and prototyping. Use the 14-day full-access trial to compare stronger models on real tasks.
How do I compare models for my codebase?
Build a private evaluation set from real tasks, run candidates with identical prompts, and score tests passed, edit precision and review effort. Public leaderboards are not a substitute.
Should I use one model or several?
Several, selected by task tier. A default plus one escalation model covers most teams, with cheap models for high-volume low-stakes work.
What about long files?
plugsky-longctx targets large context. Test it on your own long files, because context handling and truncation behaviour affect results.
How do I switch models without code changes?
Keep model names in configuration or a routing table, and use aliases where available so updates do not require a deploy.
Does the API differ per model?
No. Every model uses the same OpenAI-compatible chat completions API, including streaming and function calling where the model supports them.