How do you build AI cost optimization with AI agents?

Key facts

Agent endpoint	POST https://api.plugsky.com/v1/chat/completions with tools and tool_choice
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage; no per-token charges on self-serve
Models	30+ models behind one API; plugsky-micro and plugsky-lite are free to start
Cheap path	plugsky-micro and plugsky-lite for classification, extraction and routine replies
Strong path	plugsky-pro and plugsky-frontier reserved for planning and hard reasoning
Semantic cache	POST /v1/embeddings with plugsky-embed plus your own vector store
Controls	Scoped API keys, usage analytics and audit logs per agent fleet
Roadmap	Batch and fine-tuning endpoints are coming soon; do not budget around them yet

TL;DR

• Route before you reason: classify each request and pick the cheapest model that clears your quality bar.
• Cache semantically with plugsky-embed so repeated questions skip the model call entirely.
• Budget iterations, context tokens and retries per task instead of per process.
• Measure cost per resolved task and escalation rate; tokens are an input, not the goal.
• Flat monthly self-serve plans remove per-token billing, so optimization shifts to quota, latency and headroom.

How it works, step by step

1. Define the quality bar per task type and estimate what a wrong answer costs in support time or revenue.
2. Add a cheap classifier step with plugsky-micro that labels each request by complexity and intent.
3. Insert a semantic cache: embed the query with plugsky-embed and return a stored answer above a similarity threshold.
4. Route by label: routine work to plugsky-micro or plugsky-lite, hard work to plugsky-pro or plugsky-frontier.
5. Wrap the loop in a budget guard that caps tool iterations, context tokens and retries per request.
6. Log model, latency, cache hit and task outcome per request, then review weekly and re-tune thresholds.

Original data

Try it yourself

Open the AI agent cost calculator →

Architecture of a cost-aware agent

Put a thin gateway in front of the model and make every request pass through five stages:

• Intake: normalize the request and attach a task type, tenant and budget.
• Router: classify complexity and intent, then choose a model alias from the catalogue.
• Cache: embed the normalized query with plugsky-embed and look for a stored answer above a similarity threshold.
• Executor: run the tool loop on the selected model with trimmed context.
• Controller: enforce iteration, token and retry caps, then emit cost and outcome telemetry.

The gateway is where optimization lives. Agents that embed routing inside prompts are harder to measure and impossible to tune per tenant.

Implementation: router, cache and budget guard

A practical loop: extract a task type with a cheap model, look up a cache entry, then run the agent on the routed model. Keep the classification schema tiny — one enum for complexity and one for intent is enough to start.

• Use plugsky-micro for classification and field extraction; reserve plugsky-pro for planning steps.
• Store cache entries with the prompt hash, embedding, answer, model version and a time-to-live. Invalidate on policy or catalogue changes.
• Trim history by token count, not message count, and summarize older turns with a cheap model before the window fills.
• Cap tool iterations (four to six is typical), retries per tool and total wall-clock time.
• Escalate instead of retrying blindly: a failed cheap attempt should raise the model tier once, not loop.

Evaluating cost optimization

Optimization without evaluation is just degradation. Track four numbers on a fixed task set:

• Cost per resolved task: total model work divided by tasks completed correctly, including escalations.
• Escalation rate: share of requests that needed a stronger model after a cheap attempt.
• Cache hit rate and staleness: how often the cache answers and how often users reject the answer.
• Quality floor: the same rubric you used before optimization, measured on the same prompts.

If quality drops more than your agreed tolerance, loosen routing thresholds before you touch the model list. Re-run the evaluation whenever you change system prompts or catalogue versions.

Limitations and failure modes

Flat monthly plans remove per-token arithmetic, but they do not remove limits: fair-use policies, concurrency and latency still shape what an agent can do, so treat heavy agent traffic as a capacity-planning problem.

• Aggressive caching can serve stale or tenant-crossing answers; scope cache keys per tenant and per data classification.
• Routing classifiers drift when request distributions change; monitor label confidence and sample disagreements.
• Cost-aware cascades add latency on the escalation path, which hurts interactive agents.
• Batch and fine-tuning endpoints are coming soon, so build current pipelines on chat completions and embeddings only.

Honest comparison

Capability	Plugsky agent stack	Fixed single-model agent	Provider-side routing
Model selection	Router picks from 30+ models per step	One model for every request	Vendor default model
Caching	You own a semantic cache built on plugsky-embed	Usually none	Provider cache with limited visibility
Budget control	Iteration, token and retry caps in your code	Ad hoc	Provider limits only
Pricing	Flat monthly self-serve, unlimited fair use	Per-token or per-seat	Per-token
Observability	Per-step logs plus Plugsky usage analytics	Application logs only	Vendor dashboards

Frequently asked questions

What does an AI agent optimize for cost?

It minimizes the model work needed per correctly completed task: cheaper models for routine steps, caching for repeats, and hard caps on iterations and context growth.

Does Plugsky charge per token?

Self-serve plans are flat monthly with unlimited fair-use usage — no per-token charges and no overage fees on self-serve. Canonical prices are on the live pricing page.

Which Plugsky models should handle routine agent steps?

plugsky-micro and plugsky-lite cover classification, extraction and short replies, and both are available on the free plan. Move to plugsky-pro or plugsky-frontier only when the step needs it.

How does a semantic cache work?

You embed the incoming query with plugsky-embed, compare it against stored queries in your own vector store, and return the cached answer when similarity clears a threshold you control.

How do I prove optimization worked?

Compare cost per resolved task, escalation rate and the quality rubric on a frozen evaluation set before and after each change.

Do I need a separate vector database for the cache?

You need somewhere to store vectors and metadata. Any store you already operate works because Plugsky returns embeddings rather than hosting an index for you.

Can I start without paying?

Yes. The free plan includes two free AI models, plugsky-micro and plugsky-lite, with no credit card, and a 14-day full-access trial unlocks the paid catalogue.