How do you build semantic search with RAG?

Key facts

Pattern	Embed corpus → meaning-based retrieval → grounded answers
Embeddings	plugsky-embed and plugsky-embed-multilingual via /v1/embeddings (live)
Generation	Answers with citations via JSON mode (live)
Metadata	Source, section, date and access labels per chunk
Hybrid option	Combine vector scores with keyword matching for exact terms
Models	30+ models behind one endpoint for synthesis
Deployment	Region-locked planes, VPC, on-prem and air-gapped options
Free tier	plugsky-micro and plugsky-lite on the free plan, no card required

TL;DR

• Embedding-based retrieval matches meaning, so users stop guessing keywords.
• Keep metadata on every chunk for filters, recency and permission scoping.
• Add keyword scoring alongside vectors for codes, names and identifiers.
• Ground answers in retrieved passages and cite them, or return the passage list.
• Evaluate retrieval and answers separately to know what to fix.

How it works, step by step

1. Collect the corpus and decide the retrieval unit — paragraph, section or article — based on how users ask questions.
2. Chunk accordingly, storing source, section, date and access labels on every chunk.
3. Embed chunks with plugsky-embed, or plugsky-embed-multilingual for multi-language corpora.
4. At query time, embed the question, apply filters and retrieve the top passages.
5. Blend vector and keyword scores, then deduplicate near-identical passages.
6. Generate an answer constrained to the retrieved passages, with citations, or return the passage list when no answer is warranted.
7. Evaluate retrieval recall, answer groundedness and citation accuracy on a golden set.

Try it yourself

Open the vector database comparison →

Embeddings plus RAG: search and answers

Semantic search replaces keyword guessing with meaning matching: a query about reducing cloud spend finds documents about cost optimisation even without shared vocabulary. RAG adds a synthesis layer, turning the retrieved passages into a direct answer with citations. The retrieval half is the hard part; generation is comparatively straightforward once the right passages are in the prompt.

On Plugsky both model calls are live: /v1/embeddings for indexing and query encoding, and /v1/chat/completions with JSON mode for cited answers. Corpus preparation and ranking remain yours, which is where quality is won.

Building the retrieval layer

Chunking and metadata determine what semantic search can do. Chunks should be self-contained units that a reader would recognise as an answer, and metadata should carry everything you might filter on: source, section, date, document type and access labels.

• Hybrid scoring: blend vector similarity with keyword matching so identifiers and names still hit.
• Filters first: apply access and freshness constraints before ranking, not after.
• Deduplication: collapse overlapping chunks from the same document.
• Context budget: rank and trim to fit the model's window rather than truncating arbitrarily.

Answer grounding and evaluation

Two output modes matter: a grounded answer with citations, or a ranked passage list when synthesis adds nothing. Let the model refuse when retrieval is weak, and always return the sources so users can verify. This is what separates a trustworthy search product from a plausible-sounding one.

Evaluate retrieval recall on questions with known source passages, then groundedness and citation accuracy on the answers. When recall drops, check chunking and metadata before changing models. When answers drift despite good retrieval, tighten the prompt and context assembly. Re-index as content changes, and deploy in a region that satisfies policy, including on-prem for restricted corpora.

Honest comparison

Capability	Semantic search plus RAG	Keyword search	Vector search only
Matching	Meaning plus optional keywords	Literal terms	Meaning only
Output	Grounded answers with citations	Ranked links	Ranked passages
Filters	Metadata before ranking	Limited	Metadata support varies
Refusals	Explicit when evidence is weak	No results page	N/A
Evaluation	Recall plus groundedness	Click metrics	Recall only

Frequently asked questions

What is the difference between semantic search and RAG?

Semantic search retrieves by meaning; RAG adds grounded generation on top of those results. You can ship semantic search alone, but the answer layer is what most users actually want.

Should I keep keyword search?

Yes, in most corpora. Hybrid scoring catches exact identifiers, product codes and proper names that embeddings can miss, and it costs little to combine both signals.

How do I choose chunk sizes?

Match the retrieval unit to how users ask questions — usually a section or a few paragraphs. Tune against recall rather than picking a size by intuition.

Do I need a vector database?

For production, yes: metadata filtering, incremental updates and operational tooling matter. For a prototype, in-memory scoring over a small corpus is fine.

How do I stop irrelevant answers?

Allow refusal, require citations, and return the passage list when synthesis is not warranted. Measure groundedness so the behaviour is enforced, not assumed.

Which embedding model should I use?

plugsky-embed for English-heavy corpora, plugsky-embed-multilingual when documents and queries span languages. Always embed queries with the same model as documents.

Can it run on-prem?

Yes. Plugsky supports region-locked cloud planes plus VPC, on-prem and air-gapped deployment, and your corpus and index remain under your control.