Key facts
| Definition | Retrieving relevant passages and passing them to a model as context for the answer |
| Pipeline | Chunk, embed, index, retrieve, rerank optionally, then generate with citations |
| Why it matters | Answers reflect your data without retraining and can cite sources |
| Plugsky support | Embeddings and chat completions are live; RAG APIs and guidance are documented |
| Chunking | Chunk size and overlap drive retrieval quality more than model choice |
| Evaluation | Measure retrieval recall and answer faithfulness separately |
| Models | 30+ models behind one API; embeddings run on dedicated embedding models |
| Freshness | Reindex on a schedule or on change; stale indexes cause confident wrong answers |
TL;DR
- RAG grounds answers in your data without fine-tuning.
- Chunking and retrieval quality decide most of the outcome.
- Evaluate retrieval and generation separately — they fail differently.
- Citations make answers verifiable; require them where accuracy matters.
- Reindex regularly or answers drift out of date.
How it works, step by step
- Collect and clean source documents, preserving structure and provenance metadata.
- Split content into chunks sized to the embedding model and the question types.
- Generate embeddings with a dedicated embedding model and store vectors with metadata.
- Retrieve top candidates for the query and optionally rerank them.
- Build the prompt with retrieved passages and require citations to passage ids.
- Evaluate retrieval recall and answer faithfulness, then tune chunking and k.
Try it yourself
Open the RAG architecture builder →
How a RAG pipeline works
Documents are split into chunks, each chunk is embedded into a vector and stored in an index with metadata such as source, section and date. At query time the question is embedded, the nearest chunks are retrieved, and those passages are placed in the prompt as context. The model answers from that context. Every stage is replaceable: a different chunker, embedding model or index changes results more than swapping the generation model usually does.
When to use RAG
- Private knowledge: answers must come from internal documents, not model memory.
- Fresh data: content changes faster than any training cycle.
- Attribution: users need to see which source supports an answer.
- Access control: different users may retrieve only permitted documents.
- Scale: the corpus is too large for any context window.
If the model already knows the answer and citations are unnecessary, plain prompting is simpler and cheaper.
Common mistakes
- Chunking arbitrarily and splitting concepts across boundaries.
- Embedding and retrieving without metadata filters, so stale or unauthorized content surfaces.
- Skipping reranking when top-k similarity returns near-duplicates.
- Measuring only answer quality, which hides whether retrieval or generation failed.
- Never reindexing, so the system answers confidently from outdated documents.
RAG on Plugsky
Plugsky provides the live building blocks: embedding endpoints for indexing and query vectors, chat completions for grounded generation, and function calling for agentic retrieval loops. Because the API is OpenAI-compatible, existing RAG frameworks connect with a base URL change. Combine a dedicated embedding model with a strong generation alias, and keep access filtering and provenance in your application layer. For corpora that change constantly, schedule reindexing and monitor retrieval quality as a first-class metric.
Honest comparison
| Approach | RAG | Fine-tuning | Long context prompt |
|---|---|---|---|
| Knowledge source | Your index at query time | Baked into weights | Whatever you paste |
| Freshness | Update by reindexing | Retrain required | Immediate |
| Citations | Passage ids and sources | Weak | Exact positions |
| Setup effort | Chunking plus index | Dataset plus training | Minimal |
| Best for | Changing, private knowledge | Behavior and format | Whole-artifact tasks |
Frequently asked questions
What is RAG?
Retrieval-augmented generation: retrieving relevant passages from your own data and passing them to a model as context so answers are grounded in that data rather than only in model memory.
Does RAG require fine-tuning?
No. RAG and fine-tuning are independent. RAG adds knowledge at query time; fine-tuning changes behavior. Many systems use RAG alone.
What makes RAG fail?
Usually retrieval: bad chunking, missing metadata filters, no reranking or a stale index. When retrieval is wrong, generation cannot recover.
How do I evaluate RAG?
Score retrieval recall separately from answer faithfulness and citation support. Comparing the two tells you which stage to fix.
Can I build RAG on Plugsky?
Yes. Embeddings and chat completions are live, and an OpenAI-compatible API means existing RAG frameworks work after changing the base URL.
Is RAG enough for compliance?
It helps with attribution and access filtering, but you still need clear retention, access control and audit policies around the index and the prompts.