Key facts
| Common local models | BGE-M3, multilingual-e5, Nomic Embed, GTE and Qwen embedding families |
| Typical dimensions | 384, 768, 1024 and higher; larger is not automatically better |
| Context length | Models commonly embed 512 to 8192 tokens per chunk, depending on family |
| Multilingual | BGE-M3 and multilingual-e5 cover many languages in one index |
| Storage math | Dimensionality times vector count times bytes per value sets index size |
| Reranking | A cross-encoder reranker improves precision on top-k retrieval |
| Cloud option | Plugsky serves embeddings live over an OpenAI-compatible endpoint |
TL;DR
- Choose by language coverage and context length before chasing leaderboard scores.
- Run one blind evaluation on your own documents; generic rankings rarely predict your results.
- Higher dimensions improve recall but increase storage and query cost.
- Add a reranker before replacing the embedding model.
- Keep embeddings local for privacy; move them to a hosted API when volume outgrows one machine.
How it works, step by step
- List the languages and document types your index must cover.
- Shortlist two or three embedding models, including one multilingual candidate.
- Chunk a sample of real documents with consistent size and overlap settings.
- Build a small labelled query set of 20-50 questions with known answers.
- Measure recall at k and answer grounding for each model on that set.
- Add a reranker and re-measure to see whether retrieval or generation is the bottleneck.
- Estimate index size and query latency, then choose the model you can sustain in production.
Original data
Try it yourself
Open the embedding model comparison →
How to choose a local embedding model
Start with language coverage. If your documents mix languages, a multilingual model such as BGE-M3 or multilingual-e5 keeps one index instead of several. If everything is English and domain-specific, a smaller English-focused model may match quality at lower cost.
Then weigh dimension and context. Dimensions determine index size and query latency: 384 and 768 are cheap, 1024 and above improve recall on large corpora but consume more memory and disk. Context length decides how much text one vector represents, which shapes your chunking strategy. A 512-token limit forces smaller chunks; an 8k limit lets a vector represent a whole section, at some loss of precision.
Finally, check the licence and how the model is distributed. Local embedding weights are usually downloadable, but commercial terms differ between families and versions.
Retrieval quality comes from the pipeline
The embedding model is one variable among several. Chunk size, overlap, metadata filters and hybrid search often move metrics more than a model swap. Test combinations rather than single changes, and keep a fixed evaluation set so results compare.
- Chunking: 256-512 tokens with 10-20% overlap is a solid default for prose.
- Hybrid search: combine vector similarity with keyword scoring for names, codes and rare terms.
- Reranking: retrieve a wide candidate set, then rerank to the passages you actually pass to the model.
- Grounding: require citations and verify that answers quote real chunks.
If answers improve when you feed gold passages directly, retrieval is the problem. If they still fail, the generator or prompt is the bottleneck.
Local versus hosted embeddings
Local embeddings keep document text and vectors on your hardware, which matters for regulated content. The trade-off is operational: you own the model version, serving process and scaling. A single GPU or even a CPU can serve embeddings for moderate corpora because embedding calls are short and parallelisable, but index rebuilds on large corpora take real time.
Hosted embeddings remove that operations work and scale on demand. Plugsky serves embeddings through an OpenAI-compatible API, alongside chat, streaming, JSON mode, function calling, RAG and agents, all live with 30+ models. Audio, image, moderation, batch and fine-tuning endpoints are coming soon. A practical pattern is local embeddings for private corpora and hosted embeddings for public or high-volume collections. Plan limits are on the live pricing page.
Honest comparison
| Option | Language fit | Runs on | Operations | Privacy |
|---|---|---|---|---|
| BGE-M3 | Multilingual | CPU or GPU | You serve and version it | Fully local |
| Multilingual-e5 | Multilingual | CPU or GPU | You serve and version it | Fully local |
| Nomic Embed / GTE | Mostly English, some multilingual | CPU or GPU | You serve and version it | Fully local |
| Plugsky embeddings | Multilingual via hosted API | Hosted | Managed | Provider-processed |
Frequently asked questions
Do I need a GPU to run local embeddings?
No. Embedding models are small compared with chat models and run acceptably on CPU for moderate volumes. A GPU helps during large index builds and high query rates.
Which embedding model is best for Arabic or Indonesian?
Prefer a multilingual family such as BGE-M3 or multilingual-e5, then evaluate on your own documents in those languages before committing.
How many dimensions should I use?
Start at 768 or 1024 for general RAG. Higher dimensions help large corpora but raise index size and latency; test whether the recall gain justifies the cost.
Should I use the same model for indexing and querying?
Yes. Always embed queries with the exact model and version used at index time, or similarity scores become meaningless.
Do I need a reranker if my embeddings are good?
A reranker usually still helps. Retrieve a generous candidate set, then rerank to the few passages the generator sees.
Can I mix local and hosted embeddings?
You can, but keep separate indexes per model. Vectors from different models are not comparable, so never mix them in one collection.
How do I evaluate a local embedding model?
Build 20-50 questions with known answers, measure recall at k on your real documents, and compare rivals on the same chunks and queries.