How do you build AI cost optimization with an OpenAI-compatible API?

Key facts

Compatibility	OpenAI-compatible POST /v1/chat/completions; change the base URL and model name
Migration	One-line base_url change keeps existing SDK code and tests
Models	30+ models behind one endpoint, from free tiers to frontier reasoning
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage; no per-token charges
Free tier	Free plan with 2 free AI models, no card required
Output control	JSON mode and function calling reduce retries and post-processing tokens
Deployment	Plugsky cloud, VPC, on-prem and air-gapped options
Roadmap	Batch, files and fine-tuning endpoints are coming soon

TL;DR

• Swap base_url and model names; keep the rest of your code unchanged.
• Constrain outputs with JSON mode and max_tokens to stop paying for prose you discard.
• Cache deterministic answers and reuse prompt prefixes instead of resending long context.
• Map each workload to a model alias so routine traffic never reaches frontier tiers.
• Flat monthly self-serve pricing turns optimization into quota and latency management.

How it works, step by step

1. Audit traffic by workload: label each call site with a task type and an acceptable quality bar.
2. Change the base URL to Plugsky and map each call site to a model alias in one configuration file.
3. Add max_tokens per task and switch structured tasks to JSON mode with a schema.
4. Introduce an answer cache keyed by normalized prompt, model alias and schema version.
5. Add per-workload budgets and alerting using your logs plus Plugsky usage analytics.
6. Run the same evaluation set before and after each change, and roll back any setting that breaks the quality floor.

Original data

Try it yourself

Open the LLM cost calculator →

Architecture: a request budget layer

Because Plugsky is OpenAI-compatible, the cheapest architecture change is a wrapper around the SDK rather than a rewrite:

• Config layer: one map from workload name to model alias, timeout and max_tokens.
• Prompt layer: shared system prompts and schemas kept in version control.
• Cache layer: exact-match cache for deterministic requests, with a time-to-live per workload.
• Telemetry layer: log workload, alias, latency, output length and retry count per call.

The wrapper is about ten lines of code per language and pays for itself the first time someone points a bulk job at a frontier model.

Request hygiene that removes waste

Concrete changes, in order of impact:

• Cap outputs: set max_tokens per task. Classification needs tens of tokens, not thousands.
• Force structure: use JSON mode with a schema so downstream code parses once instead of retrying.
• Trim history: send the last relevant turns plus a summary, not the entire transcript.
• Shorten prompts: a 900-token policy block that five tasks share should live in your cache or retrieval, not every request.
• Stop retry storms: one retry with backoff, then a distinct error path. Blind retries multiply work with no quality gain.
• Match tiers: plugsky-micro for extraction, plugsky-lite for routine chat, plugsky-pro for reasoning, plugsky-frontier for the hardest cases.

Evaluation before and after optimization

Keep a frozen set of real requests with expected outputs and run it on every change. Track:

• Quality floor: pass rate on the frozen set, per workload.
• Output length: average tokens returned, which should fall without quality loss.
• Retry rate: failed validations and re-asks per hundred requests.
• Cache hit rate: share of requests answered without a model call.

Optimization is only successful if the quality floor holds. If a cheaper alias drops pass rate beyond tolerance, restore the previous mapping for that workload rather than weakening the test.

Limitations and honest caveats

Flat monthly plans change the economics of self-serve usage, not the laws of capacity. Fair-use limits, concurrency and model availability still apply, and heavy asynchronous workloads should be designed with retries and backoff.

• Cheaper models fail differently: they may produce valid JSON with wrong values, so validate semantics, not just syntax.
• Caching can leak across tenants if keys are not scoped; include tenant and data class in the cache key.
• Some specialist endpoints are not live yet — audio, images, moderation, files, batch, fine-tuning, assistants and responses are coming soon, so keep those workloads on other tooling for now.
• Model aliases can change behaviour between catalogue versions; pin and re-evaluate on upgrade.

Honest comparison

Capability	Plugsky OpenAI-compatible API	Per-token API	Self-hosted open models
Integration effort	Change base_url and model name	Change base_url	Serve, scale and operate GPUs
Pricing	Flat monthly self-serve, unlimited fair use	Per-token	Hardware plus ops time
Model choice	30+ models behind one endpoint	Vendor catalogue	Only models you can host
Output control	JSON mode, function calling, max_tokens	Usually equivalent	Depends on your server stack
Deployment	Cloud, VPC, on-prem, air-gapped	Vendor cloud	Your infrastructure

Frequently asked questions

Do I have to rewrite code to use Plugsky?

No. The chat completions endpoint is OpenAI-compatible, so you change the base URL and model names and keep your existing SDK, retry logic and tests.

How does flat pricing change optimization?

With no per-token charges on self-serve plans, the goal shifts from shaving tokens to protecting quality while staying inside fair-use capacity and latency targets.

What is the quickest waste to remove?

Unbounded max_tokens and unstructured output. Both cause long responses and retries that add work without improving answers.

Should I route everything through one model?

No. Map workloads to aliases: plugsky-micro and plugsky-lite for routine work, plugsky-pro and plugsky-frontier where reasoning quality matters.

Can I cache responses safely?

Yes, if keys include tenant, workload and prompt version, and if the time-to-live matches how often the underlying facts change.

What about batch jobs?

The batch endpoint is coming soon. Today, run asynchronous jobs on chat completions with concurrency limits, backoff and idempotent writes.

How do I start without cost risk?

Use the free plan with plugsky-micro and plugsky-lite and no credit card, then use the 14-day full-access trial to evaluate paid models on your real workload.