How do you build classification with an OpenAI-compatible API?

Key facts

Endpoint	POST https://api.plugsky.com/v1/chat/completions with JSON mode and a label enum
Compatibility	Same shape as OpenAI; change base_url and model name
Models	30+ models; plugsky-micro and plugsky-lite cover high-volume labeling
Validation	Schema-check every response; retry once, then queue for review
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage
Free tier	Free plan with 2 free AI models, no card required
Throughput	Use concurrency limits and idempotent writes; the batch endpoint is coming soon
Governance	Scoped keys and usage analytics per pipeline

TL;DR

• JSON mode plus a strict enum turns a generative call into a classification function.
• Validate on the server; never trust that valid JSON means a valid label.
• Retry once on schema failure, then route to review instead of looping.
• Keep sampling settings and prompts pinned so results are reproducible.
• Track per-class precision and the invalid-output rate as first-class metrics.

How it works, step by step

1. Change the base URL to Plugsky and confirm an existing classification request works unchanged.
2. Define the label set as an enum in the JSON schema and include it in the system prompt.
3. Add validated examples per label, especially boundary cases between similar labels.
4. Call chat completions with JSON mode, parse the response and validate it against the schema.
5. Retry once on schema failure with a corrective instruction, then send to the review queue.
6. Log label, confidence, model version and validation outcome, and monitor per-class metrics weekly.

Original data

Try it yourself

Open the JSON mode tester →

Architecture: a classification service

Five small components, all testable in isolation:

• Input normalizer: trims, strips markup and standardizes language before prompting.
• Prompt builder: system prompt, enum, definitions and a small set of examples.
• Model call: chat completions with JSON mode, a pinned alias and a fixed configuration.
• Validator: schema and enum checks, business-rule checks and a single corrective retry.
• Review queue: items that fail validation or fall below the confidence floor.

Keeping validation outside the model is what makes the service reliable: the model proposes a label, your code decides whether to accept it.

Implementation details

Practical choices for production classification:

• Put the enum in both the JSON schema and the system prompt; redundancy reduces invalid outputs.
• Use a fixed set of examples per label and keep configuration pinned, because sampling settings change output distributions.
• Ask for a short reason field — not for chain-of-thought, but a one-line justification that reviewers can scan quickly.
• Validate business rules after schema checks: some labels may be impossible for certain inputs regardless of model confidence.
• Cap output length so a classification cannot turn into an essay that bloats storage and latency.
• Batch requests with bounded concurrency and idempotency keys derived from the item id.

Evaluation and monitoring

Classification quality is measurable without a human on every item:

• Per-class precision and recall: aggregate accuracy hides the labels that matter most.
• Invalid-output rate: schema failures per thousand requests, which should trend to near zero after prompt tuning.
• Distribution shift: weekly label mix compared with the training sample — sudden shifts usually mean upstream data changed.
• Review queue yield: agreement rate between reviewers and the model on queued items.
• Reproducibility: re-run a frozen golden set on the same pinned configuration before every deploy.

Limitations

JSON mode guarantees structure, not correctness:

• A label can be syntactically valid and semantically wrong; only evaluation against labeled data catches that.
• Boundary labels drift as products and policies change, so schedule taxonomy reviews.
• Long documents may exceed the context window; classify sections and aggregate rather than truncating blindly.
• The batch endpoint is coming soon; large backfills need concurrency limits, retries and idempotent writes today.
• Model aliases differ in instruction-following; pin an alias and re-evaluate before switching.

Honest comparison

Capability	Classification on Plugsky	Hosted classification service	Rules and regex
Output contract	JSON mode with a strict enum	Service-defined schema	Whatever your code returns
Label changes	Update schema and examples	Retrain or reconfigure	Rewrite rules
Model choice	30+ aliases behind one endpoint	Vendor models only	None
Cost shape	Flat monthly self-serve, unlimited fair use	Per-item or per-character	Engineering time
Integration	OpenAI-compatible SDK call	Service SDK	Custom code

Frequently asked questions

How do I force a fixed set of labels?

Use JSON mode with an enum in the schema and repeat the allowed labels in the system prompt. Then validate the returned label against the enum before accepting it.

What if the model returns an invalid label?

Retry once with a corrective instruction appended, then route the item to a review queue if it fails again. Silent remapping to the nearest label hides data problems.

Which model should classify high volumes?

plugsky-micro and plugsky-lite handle high-volume labeling well. Use stronger aliases only for genuinely ambiguous items, and evaluate before assuming they help.

Should I include examples in the prompt?

Yes for boundary cases: a few well-chosen examples per label improve consistency more than long definitions. As the taxonomy grows, consider retrieval instead.

Can I process a large backfill?

The batch endpoint is coming soon. Today, run bounded concurrency with idempotency keys, retries on transient errors and a queue for failures.

How do I prove quality over time?

Maintain a frozen golden set with per-class metrics, monitor invalid-output rate and label distribution, and re-run the set whenever the prompt or alias changes.

Can I start for free?

Yes. The free plan includes plugsky-micro and plugsky-lite with no card, and a 14-day full-access trial lets you benchmark classification quality across models.