How do you build AI cost optimization with RAG?

Key facts

Retrieval endpoint	POST /v1/embeddings with plugsky-embed returns vectors you store yourself
Generation endpoint	POST /v1/chat/completions with retrieved chunks inserted into the prompt
Models	30+ models behind one endpoint; plugsky-micro and plugsky-lite are free to start
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage; no per-token billing on self-serve
RAG status	RAG is live; embeddings and chat completions are the documented building blocks
Index control	Chunk size, overlap, metadata and refresh cadence stay in your pipeline
Governance	Scoped keys, usage analytics and audit logs cover the retrieval path
Roadmap	Batch ingestion and fine-tuning endpoints are coming soon

TL;DR

• Most RAG waste is context waste: retrieve fewer, better chunks before you touch the model.
• Chunk on document structure, not fixed character counts alone.
• Cache embeddings for unchanged documents and answers for repeated questions.
• Use cheap models when retrieved context is clean; escalate only on weak evidence.
• Flat monthly self-serve plans mean you tune quality, quota and latency rather than token math.

How it works, step by step

1. Inventory the corpus and define what a correct answer must cite before you chunk anything.
2. Chunk on structure (headings, clauses, table rows) with modest overlap, and keep source metadata on every chunk.
3. Embed once with plugsky-embed, store vectors plus metadata, and re-embed only changed documents.
4. At query time, retrieve a candidate set, then prune to the smallest context that still answers the question.
5. Route the prompt: plugsky-micro or plugsky-lite for extractive answers, plugsky-pro for synthesis across documents.
6. Log retrieval precision, dropped chunks and answer acceptance so tuning has evidence.

Original data

Try it yourself

Open the RAG cost calculator →

Where RAG cost actually accumulates

Three places dominate: oversized chunks pushed into every prompt, redundant embedding jobs over unchanged documents, and strong models used for extractive questions. Fix them in that order.

• Ingestion: embedding the whole corpus on every deploy is pure waste. Hash documents and embed only new or changed ones.
• Retrieval: fetching ten chunks when two contain the answer multiplies prompt size and dilutes the model's attention.
• Generation: a lookup question does not need a frontier model; it needs clean context and a strict instruction to answer only from it.

Implementation: chunk, embed, prune, route

A lean pipeline has five stages:

1. Parse and chunk with structure-aware rules; keep tables and clauses intact where possible.
2. Embed each chunk with plugsky-embed and write the vector, source id, section and updated date to your store.
3. Retrieve a wider candidate set, then apply a budget: maximum chunks, maximum characters, and a minimum similarity cut.
4. Assemble the prompt with compact chunk labels and an instruction to cite the source ids.
5. Route by question type — extraction to plugsky-micro, comparison and synthesis to plugsky-pro or plugsky-frontier.

Keep a fallback: if the top similarity is below your threshold, answer that evidence is missing instead of widening context blindly.

Evaluation that protects quality

Measure retrieval separately from generation so you know which stage to change:

• Retrieval recall: on a labeled question set, is the answering chunk in the top k?
• Context precision: how much of the prompt is actually used in the final answer?
• Groundedness: does the answer stay inside the retrieved text, with citations that resolve?
• Miss handling: when evidence is absent, does the system say so?

Cost tuning should never move groundedness below the pre-optimization baseline. If it does, revert the pruning threshold, not the evaluation.

Limitations and trade-offs

RAG is not a fine-tuning substitute, and this pipeline will not fix a corpus that is duplicated, outdated or contradictory. Additional honest limits:

• Plugsky returns embeddings; you own the index, so search quality depends on your storage and ranking choices.
• Structure-aware chunking is corpus-specific work; generic splitters underperform on contracts, manuals and tables.
• Aggressive pruning increases miss rates on multi-hop questions, which need larger context or an agent loop.
• Reranking models are not part of the documented live stack today; validate alternatives before designing around them.
• Batch ingestion and fine-tuning are coming soon, so schedule them for later rather than assuming availability.

Honest comparison

Capability	Plugsky plus your index	Managed RAG service	Prompt-stuffing without retrieval
Cost driver	Context size and model tier per query	Service unit pricing and storage	Every prompt carries the whole document
Index ownership	Yours; you choose store and ranking	Vendor-managed	None
Model choice	30+ models, routed per query type	Vendor-selected	Whatever model you call
Caching	Embed and answer caches under your control	Limited	None
Freshness	Re-embed only changed documents	Vendor sync	Manual updates

Frequently asked questions

What is the biggest RAG cost lever?

Retrieved context size. Fewer, better chunks shrink every downstream request and improve answer quality at the same time.

Does Plugsky host the vector index?

No. Plugsky provides embeddings through POST /v1/embeddings and chat completions; you store and query vectors in your own database.

Which embedding model should I use?

plugsky-embed is the general-purpose choice, and plugsky-embed-multilingual covers Arabic and mixed-language corpora. Match the embedding model to your corpus language before tuning chunk sizes.

How do I avoid re-embedding everything?

Hash each document and embed only new or changed content. Store the document hash and embedding version alongside each vector.

When should I use a stronger model?

When the answer requires synthesizing several documents, reasoning over conflicting evidence, or producing a structured comparison. Extractive lookups stay on cheap models.

Is there a free way to test the pipeline?

Yes. The free plan includes plugsky-micro and plugsky-lite with no card, and a 14-day full-access trial unlocks the paid catalogue for evaluation.

What is the cheapest thing to build first?

An answer cache for repeated questions plus conservative context pruning. Both reduce work without changing your model choice.