Key facts
| Components | Ingest, embed, store, retrieve, generate |
| Embeddings | plugsky-embed and plugsky-embed-multilingual via /v1/embeddings (live) |
| Generation | Streaming and JSON mode on /v1/chat/completions (live) |
| Models | 30+ models behind one endpoint, choose per task |
| Retrieval | Hybrid vector plus keyword search with metadata filters |
| Citations | Structured answers with source IDs from retrieved chunks |
| Deployment | Region-locked planes, VPC, on-prem and air-gapped options |
| Free tier | plugsky-micro and plugsky-lite on the free plan, no card required |
TL;DR
- Fix the pipeline order first; optimise one stage at a time with measurements.
- Chunk by document structure and keep metadata for filtering and citations.
- Blend keyword and vector retrieval — exact identifiers matter as much as meaning.
- Ground generation in retrieved context and require citations for every claim.
- Evaluate retrieval and generation separately so you fix the right stage.
How it works, step by step
- Pick a corpus and write a golden set of questions with expected source passages before building anything.
- Build ingestion: parse documents preserving structure, then chunk by section with metadata attached.
- Choose an embedding model — plugsky-embed for English, plugsky-embed-multilingual for mixed corpora — and index your chunks.
- Assemble retrieval: vector search plus keyword search, metadata filters and a top-k policy tuned on the golden set.
- Write the generation prompt to answer only from retrieved context, cite chunk IDs and refuse when evidence is missing.
- Add an evaluation harness for retrieval recall, groundedness and citation accuracy, then iterate on one stage at a time.
- Harden for production: incremental re-indexing, permission filters, tracing and a provider-agnostic client.
Original data
Try it yourself
Open the RAG architecture builder →
Architecture: five components
RAG is a pipeline, and each stage has one job. Ingestion turns documents into clean text. Chunking cuts that text into retrievable units. Embedding maps units into vectors. The store keeps vectors with metadata. Retrieval selects context for the model, which then generates a grounded answer. Failures are easier to fix when you know which stage produced them.
Plugsky covers the two model calls in that pipeline — /v1/embeddings and /v1/chat/completions — both live and both OpenAI-compatible. Everything else remains in your code, which is what lets you enforce tenancy, permissions and freshness rules.
Decisions that determine quality
Three choices dominate outcomes: how you chunk, how you retrieve and how you constrain generation. Chunking should follow document structure so each passage stands alone. Retrieval should combine semantic and lexical signals, with metadata filters applied first. Generation should be strictly grounded, with citations required and refusals allowed.
- Chunk size and overlap: tune against recall, not intuition; keep headings in each chunk.
- Embedding model: embed queries and documents with the same model, always.
- Hybrid scoring: combine vector similarity with BM25-style keyword matching.
- Context assembly: deduplicate, order by relevance and stay within the model's context budget.
From prototype to production
A prototype answers questions; a production system stays correct as data changes. Add incremental re-indexing so a document update does not require a full backfill, and store document versions so stale chunks can be filtered or demoted. Apply permission filters before retrieval, and log which chunks each answer used.
Measure retrieval recall, groundedness, citation accuracy and refusal behaviour on every release, and keep a regression set drawn from real failures. Route lightweight tasks — query classification, short answers — to plugsky-micro or plugsky-lite, and reserve stronger models for multi-document synthesis. Deploy in a region that satisfies policy, including on-prem where documents cannot leave your network.
Honest comparison
| Stage | Plugsky role | Your stack | Typical failure |
|---|---|---|---|
| Ingestion | None — runs in your pipeline | Parsing and cleaning | Lost structure and bad text |
| Embedding | plugsky-embed via /v1/embeddings (live) | Batching and retries | Mixed vector spaces |
| Retrieval | None — your service | Hybrid search and filters | Missing the right passage |
| Generation | 30+ models on /v1/chat/completions (live) | Prompt and context assembly | Ungrounded answers |
| Evaluation | None — your harness | Golden sets and scoring | Silent regressions |
Frequently asked questions
What is the minimum viable RAG stack?
A parser, a chunker, an embedding model, a vector store and a chat model. You can run it with a few hundred documents and one golden question set before investing in infrastructure.
Which chunking strategy should I use?
Structure-first: chunk by heading and section, keep titles in each chunk, and add overlap where text is continuous. Tune size against retrieval recall rather than picking a number upfront.
Do I need a vector database?
For a prototype, in-memory scoring works. For production you want metadata filtering, incremental updates and operational tooling — a vector database or an extension such as pgvector provides those.
Should I use keyword search too?
Yes. Embeddings capture meaning but can miss exact identifiers, product codes and clause numbers. Hybrid retrieval combines both, and filters apply before ranking.
How do I stop hallucinations?
Constrain the model to retrieved context, require citations and permit refusal. Then measure groundedness and refusal correctness on every change.
How often should I re-index?
On change, ideally. Incremental indexing by document ID keeps updates cheap; scheduled full rebuilds are a fallback when change detection is unreliable.
Can the model layer run on-prem?
Yes. Plugsky supports region-locked cloud planes plus VPC, on-prem and air-gapped deployment, so the pipeline can run entirely within your environment.