How do you build chatbots with RAG?

Key facts

Retrieval	POST /v1/embeddings with plugsky-embed over help-center and policy content
Generation	POST /v1/chat/completions with retrieved passages, citation rules and a refusal path
Models	30+ models; cheap aliases for short answers and stronger ones for multi-article synthesis
Freshness	Re-embed changed articles on publish and keep an updated timestamp per chunk
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage
Free tier	Free plan with 2 free AI models, no card required
RAG status	RAG is live; embeddings plus chat completions are the building blocks
Roadmap	Rerankers are not in the documented live stack; batch ingestion is coming soon

TL;DR

• Answer only from retrieved passages and cite them; otherwise hand off.
• Chunk help content by section and keep the article title on every chunk.
• Re-embed on publish so answers track current policy.
• Replace weak retrieval with a refusal or handoff, never with model memory.
• Track unanswered questions as content gaps, not model failures.

How it works, step by step

1. Export the knowledge base and tag each article with audience, product and last-reviewed date.
2. Chunk by heading and keep the title, section and a short breadcrumb in every chunk.
3. Embed chunks with plugsky-embed and store vectors plus metadata and source URLs.
4. On each turn, retrieve candidates, drop those below a similarity floor, and fit the rest into a context budget.
5. Generate with an instruction to cite source ids and to say when the answer is not covered.
6. Review fallback conversations weekly and update content where retrieval genuinely lacks coverage.

Original data

Try it yourself

Open the RAG sandbox →

Architecture of a grounded chatbot

Separate the knowledge path from the conversation path:

• Ingestion: publish webhook or scheduled job that parses, chunks and embeds changed articles only.
• Retrieval: query embedding, top-k search, similarity floor and a context budget.
• Answer: chat completion with retrieved passages and a citation-first system prompt.
• Fallback: below the floor, the bot explains what it does not know and offers handoff or a ticket.
• Feedback: unresolved questions feed content owners, closing the loop that keeps retrieval useful.

Implementation details

Small decisions that strongly affect answer quality:

• Include the article title and section heading inside each chunk; retrieval often needs that context more than the body text.
• Duplicate-safe ingestion: hash each article, re-embed only new or changed content, and delete vectors for removed articles.
• Keep a per-article updated date and prefer recent chunks when scores tie.
• Cap the number of chunks and characters per turn; too much context dilutes the instruction to stay grounded.
• Cite using stable source ids and render a link, not a raw URL, in the final response.
• Handle conversational follow-ups by including the previous answer's sources as a retrieval hint, not by resending the whole transcript.

Evaluating a RAG chatbot

Score retrieval, answers and business outcomes separately:

• Retrieval recall: on a labeled question set, does the correct article appear in the top k?
• Groundedness: does every claim trace to a retrieved passage?
• Deflection: conversations resolved without an agent, measured only where content should support it.
• Escalation precision: unnecessary handoffs and missed handoffs are both expensive.
• Content gaps: clustered unanswered questions by topic, driving a content backlog instead of prompt tweaks.

Reward refusals when evidence is missing — a correct 'I do not know' is better than a plausible wrong policy answer.

Limitations

RAG does not fix a broken knowledge base, and it adds infrastructure:

• Contradictory or outdated articles produce contradictory answers; retrieval surfaces the problem but cannot resolve it.
• You own the index: Plugsky returns embeddings, so search quality depends on your storage and ranking.
• Multi-turn conversations can drift from the retrieval query; re-derive search terms from the latest user intent each turn.
• Rerankers are not part of the documented live stack, and batch ingestion is coming soon — plan re-indexing with your own jobs.
• Highly personalized answers need account data, which means tools, not retrieval — combine both when the question spans policy and account state.

Honest comparison

Capability	RAG chatbot on Plugsky	Fine-tuned chatbot	Prompt-stuffed static bot
Knowledge updates	Re-embed changed articles	Retrain and redeploy	Edit the prompt
Citations	Source ids from retrieved chunks	None, facts are in weights	Manual
Coverage changes	Content backlog from fallbacks	Training data dependency	Prompt size limit
Model choice	30+ aliases, routed per turn	One tuned model	One model
Refusal path	Similarity floor with handoff	Often overconfident	Hard-coded

Frequently asked questions

Why use RAG instead of fine-tuning for a chatbot?

Support content changes constantly. RAG lets you update answers by re-indexing an article, keeps citations possible, and avoids retraining when policy changes.

How do I chunk help-center articles?

By heading, keeping the article title and section in each chunk. That gives retrieval the context it needs and produces cleaner citations.

What should the bot do when it has no good match?

Explain that it lacks an answer, offer a human handoff or ticket, and log the question as a content gap. Never let the model improvise policy.

Which model should answer?

Extractive answers work on cheap aliases such as plugsky-micro or plugsky-lite; use plugsky-pro when a question requires combining several articles.

How often should I re-index?

On every article publish or update, plus a periodic full reconciliation. Hash-based ingestion keeps this cheap by embedding only what changed.

Do I need a vector database?

Yes, or any store that supports vector similarity. Plugsky provides embeddings; indexing and search remain in your stack.

Can I try a RAG chatbot without paying?

Yes. Embeddings and chat completions are live, the free plan includes two free models with no card, and the 14-day full-access trial opens the paid catalogue.