How do you build classification with RAG?

Key facts

Retrieval	POST /v1/embeddings with plugsky-embed over label definitions and labeled examples
Generation	POST /v1/chat/completions with retrieved examples and a fixed label enum
Update path	Re-index examples when the taxonomy changes; no prompt redeploy required
Models	30+ models; cheap aliases handle most labels when examples are strong
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage
RAG status	RAG is live; embeddings plus chat completions are the building blocks
Governance	Audit logs can record the evidence behind each label
Roadmap	Batch ingestion is coming soon; schedule index refreshes yourself

TL;DR

• Index label definitions and examples; the prompt becomes a template, not a taxonomy dump.
• Retrieved examples are dynamic few-shot — closer to the input than generic ones.
• Log the evidence behind each label so reviewers can audit decisions.
• Version the example index; taxonomy changes become data changes.
• Watch for retrieval returning a near-duplicate example that anchors the wrong label.

How it works, step by step

1. Write one-line definitions for every label and boundary notes for commonly confused pairs.
2. Collect labeled examples per class, including hard cases and near-misses.
3. Chunk and embed definitions and examples with plugsky-embed; tag each entry with label, version and source.
4. At inference, embed the input, retrieve the most similar definitions and examples, then classify with a fixed enum.
5. Store retrieved evidence ids with the output and validate the label against the enum.
6. Re-index when labels change, and re-run per-class evaluation to confirm the change took effect.

Original data

Try it yourself

Open the RAG architecture builder →

Why retrieve labels and examples

Classic few-shot prompting puts a fixed set of examples in the system prompt. That works until the taxonomy grows or changes often. Retrieval reframes the problem:

• Scope: a hundred labels cannot all fit in a prompt, but the five closest definitions can.
• Freshness: adding a label becomes an index update, not a prompt edit and redeploy.
• Grounding: the model sees the actual policy text or example that justifies the choice, which improves edge-case behavior.
• Auditability: you can log which evidence produced the label, which matters for regulated classification.

Building the classification index

Treat definitions and examples as first-class records:

• Definition chunks: one label per chunk with its boundary rules and a counter-example.
• Example chunks: input text, label, a short rationale and the source case id.
• Metadata: taxonomy version, language, product line and reviewer for each entry.
• Dedup: near-identical examples waste retrieval slots; keep the clearest instance and remove the rest.
• Balance: cap examples per label so high-volume classes do not dominate retrieval.

Index updates should be idempotent: hash each entry, upsert what changed, and remove entries for retired labels.

Implementation and evaluation

The inference path is retrieve, then decide:

1. Embed the input with plugsky-embed — reuse the vector for any duplicate detection.
2. Retrieve definitions plus examples under a context budget, with at least one example per candidate label.
3. Call the model in JSON mode with a fixed enum, asking for label, confidence and evidence ids.
4. Validate the label and confidence, then store the output with the retrieved evidence for review.

Evaluate with per-class precision and recall, plus an evidence-quality check: when the model is wrong, was a misleading example retrieved? That distinction tells you whether to fix the index or the model.

Limitations

Retrieval can help or hinder classification depending on your data:

• A bad example retrieved at the top of the list can anchor the model to the wrong label. Deduplicate and curate aggressively.
• Cold start is real: with no labeled examples, retrieval has nothing useful to return, so begin with prompted definitions and build the index from reviewed outputs.
• Near-duplicate labels create inconsistent retrieval; merge or sharpen them before blaming the model.
• Retrieval adds latency and infrastructure per prediction; for small stable taxonomies a static prompt is simpler.
• Batch ingestion is coming soon, so plan index refreshes with your own scheduler.

Honest comparison

Capability	Retrieval-based classification	Static few-shot prompting	Fine-tuned classifier
Taxonomy size	Retrieves the relevant subset per input	Limited by prompt budget	All labels in weights
Updates	Re-index entries	Rewrite and redeploy prompt	Retrain and redeploy
Evidence	Retrieved definitions and examples are logged	Fixed examples only	None
Cold start	Needs some labeled data to shine	Works with a few examples	Needs substantial data
Infrastructure	Embeddings plus a vector store	One prompt	Training pipeline

Frequently asked questions

What is retrieval-based classification?

Instead of embedding all label definitions in the prompt, you retrieve the most relevant definitions and labeled examples for each input, then classify with that evidence in context.

How many examples should the index hold?

Enough to cover each label's boundaries, including near-misses. Balance matters more than volume: cap examples per label so common classes do not crowd retrieval.

Which embedding model works for this?

plugsky-embed handles general use, and plugsky-embed-multilingual is the choice for Arabic or mixed-language inputs.

Can I audit why a label was chosen?

Yes, if you log evidence ids with the output. Store the retrieved definition and example ids alongside label, confidence and model version.

Does this replace a fine-tuned classifier?

For evolving taxonomies and policy-driven labels, yes with less maintenance. For very high-volume, stable labels, a fine-tuned or smaller dedicated model may still win on cost and latency.

What is the biggest failure mode?

A misleading top-ranked example anchoring the wrong label. Deduplicate the index and check evidence quality when reviewing errors.

Can I build the index on the free plan?

Yes. Embeddings are live, the free plan includes two free models with no card, and a 14-day full-access trial lets you compare aliases on your labeling task.