How do you build Arabic AI with RAG?

Key facts

Embeddings	plugsky-embed-multilingual is the multilingual embedding model in the catalogue
Retrieval	POST /v1/embeddings plus your own vector store; run lexical search alongside vectors
Normalization	Unify alef and ya forms, remove diacritics and tatweel, normalize digits and whitespace
Generation	POST /v1/chat/completions with retrieved Arabic passages and citation instructions
Models	30+ models behind one endpoint; verify Arabic answer quality on your own set
RAG status	RAG is live; embeddings plus chat completions are the building blocks
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage
Roadmap	Batch ingestion is coming soon; schedule re-indexing yourself

TL;DR

• Normalize Arabic before embedding and before querying — both sides must use the same rules.
• Use plugsky-embed-multilingual instead of English-only embeddings.
• Pair vector search with lexical search for names, codes and numbers.
• Keep document structure in chunks: Arabic contracts and policies break badly on fixed splits.
• Evaluate recall and groundedness per dialect, not in aggregate.

How it works, step by step

1. Define a normalization function once and apply it identically at ingestion and query time.
2. Chunk documents on structure — headings, clauses, table rows — and keep source metadata on every chunk.
3. Embed chunks with plugsky-embed-multilingual and store vectors, normalized text and original text.
4. At query time, retrieve with both vector similarity and lexical matching, then merge results under a context budget.
5. Generate with instructions to answer only from retrieved passages and cite source ids.
6. Evaluate Arabic recall, dialect handling and number accuracy, and re-index when documents change.

Original data

Try it yourself

Open the embedding model comparison →

Why Arabic retrieval needs normalization

Arabic text has many ways to write the same word: hamza and alef variants, final ya versus alef maqsura, optional diacritics and decorative tatweel. If ingestion and query normalization differ, identical words produce different embeddings and lexical misses.

• Unify alef forms to a single character and ya forms to a single character.
• Remove diacritics and tatweel for matching, but keep a diacritic-bearing copy for display if the domain needs it.
• Standardize Arabic-Indic and Western digits to one internal form; format for display separately.
• Collapse whitespace and unify punctuation variants.

Embed the normalized text with plugsky-embed-multilingual and store both forms so answers can quote the original.

Building the pipeline

A workable Arabic RAG pipeline:

1. Parse: extract text, headings and tables; Arabic PDFs vary in quality, so check extraction before indexing.
2. Normalize: apply the shared normalization function to the extracted text.
3. Chunk: split on structure with overlap, keeping clause numbers and headings with their text.
4. Embed: call plugsky-embed-multilingual in batches and store vectors plus metadata.
5. Retrieve: merge vector and lexical results, deduplicate near-identical chunks, then fit the context budget.
6. Generate: use a multilingual model with a citation-first instruction, and refuse when evidence is weak.

Keep the raw and normalized text side by side so cited answers remain faithful to the source.

Evaluating Arabic RAG

English evaluation sets will not tell you whether Arabic retrieval works:

• Retrieval recall: does the answering passage appear in the top k for Arabic and mixed-script questions?
• Dialect spread: report recall by dialect, because coverage is uneven.
• Entity accuracy: names, dates and amounts, where normalization errors are most visible.
• Number formatting: Arabic-Indic versus Western digits, matched to the user's expectation.
• Groundedness: does the answer stay within retrieved passages, with resolvable citations?

Sample real user questions rather than translating English ones; translation removes the exact phenomena you are testing.

Limitations

Arabic RAG has genuine constraints worth planning around:

• Dialect coverage varies by model and by corpus; a pipeline that works for Modern Standard Arabic may underperform on Gulf or Maghrebi queries.
• Arabic PDF extraction quality is inconsistent, especially for scanned documents, so validate parsing before tuning retrieval.
• Stripping diacritics improves matching but loses pronunciation information; keep the original text for domains where that matters.
• Lexical search for transliterated names needs its own handling, since Arabic and Latin spellings do not map cleanly.
• Rerankers are not part of the documented live stack, and batch ingestion is coming soon — plan re-indexing yourself.

Honest comparison

Capability	Arabic RAG on Plugsky	English embeddings on translated text	Keyword-only Arabic search
Embeddings	plugsky-embed-multilingual	Translation hop before embedding	None
Normalization	Shared function at ingest and query time	Rarely addressed	Manual rules
Names and codes	Vector plus lexical search	Translation distorts entities	Strong on exact matches
Dialect handling	Model-dependent, measured per dialect	Flattened by translation	Spelling-dependent
Answer grounding	Citations with refusal path	Depends on translation quality	Passage list only

Frequently asked questions

Which Plugsky embedding model should Arabic RAG use?

plugsky-embed-multilingual. It is the multilingual entry in the catalogue and the right starting point for Arabic and mixed-language corpora.

How should I normalize Arabic text?

Unify alef and ya variants, remove diacritics and tatweel for matching, normalize digits and punctuation, and apply identical rules at ingestion and query time.

Why combine vector and lexical search?

Embeddings capture meaning but blur exact strings. Lexical search recovers names, clause numbers, codes and transliterations that users type literally.

How do I chunk Arabic documents?

Use structure-aware chunking: headings, clauses and table rows. Fixed character splits break Arabic sentences and separate definitions from the terms they explain.

Does dialect matter for retrieval?

Yes. Coverage differs by dialect, so include Gulf, Egyptian, Levantine or Maghrebi queries in your evaluation if you serve those users.

Can Arabic RAG run on the free plan?

Embeddings and chat completions are live; the free plan includes two free models with no card, and a 14-day full-access trial opens stronger models for evaluation.

What about scanned Arabic PDFs?

Validate extraction quality first. Normalization and retrieval cannot recover text that OCR never captured correctly.