Key facts
| Pipeline | OCR → chunk with page anchors → embed → cited answers |
| Embeddings | plugsky-embed and plugsky-embed-multilingual via /v1/embeddings (live) |
| Generation | Cited answers with JSON mode on /v1/chat/completions (live) |
| Citations | Document, page and region anchors stored per chunk |
| OCR | Dedicated OCR engine in your pipeline; image endpoints are coming soon |
| Models | 30+ models behind one endpoint for retrieval answers |
| Deployment | Region-locked planes, VPC, on-prem and air-gapped options |
| Free tier | plugsky-micro and plugsky-lite on the free plan, no card required |
TL;DR
- OCR quality sets the ceiling — clean text first, retrieval second.
- Keep page and region anchors so citations point into the scanned original.
- Store OCR confidence per chunk and demote low-confidence passages.
- Handle tables and forms as structured text, not as prose.
- Re-run OCR when better engines arrive, then re-embed the affected documents.
How it works, step by step
- Run OCR over the archive and store text alongside page numbers, bounding regions and confidence scores.
- Clean the output: join hyphenated line breaks, normalise whitespace and reconstruct table structure.
- Chunk by page and section, attaching document ID, page number and confidence to every chunk.
- Embed chunks with plugsky-embed, or plugsky-embed-multilingual for mixed-language archives.
- Retrieve passages for a question, filtering or demoting chunks whose OCR confidence is low.
- Generate an answer with citations to document and page, and expose the source scan at that page in the UI.
- Evaluate retrieval recall and citation accuracy against questions with known scanned sources.
Original data
Try it yourself
Open the vector database comparison →
From scan to searchable text
OCR is the foundation of the whole pipeline, so its errors propagate into every answer. Choose an engine that reports per-word confidence, preserve layout information, and clean the text before indexing: joined line breaks, split words and broken tables sabotage retrieval more than vocabulary gaps do.
Image-generation endpoints are coming soon, so OCR remains a preprocessing step you operate. The model layer downstream is live on Plugsky: /v1/embeddings for indexing and /v1/chat/completions for cited answers. Keeping OCR and retrieval separate also lets you swap engines without touching the index schema.
Chunking OCR output with page anchors
Scanned documents are navigated by page, so page-level anchors are the citation unit users trust. Chunk within pages where headings allow, and always record the page number, the document ID and, where available, a bounding region for the passage.
- Tables: serialise with headers repeated and keep them as single chunks so numbers stay aligned.
- Confidence: store the minimum OCR confidence per chunk and use it to demote weak passages at ranking.
- Headers and footers: strip repeated page furniture so it does not dominate embeddings.
- Metadata: capture document type, date and language to support filters.
Quality control for OCR-fed retrieval
Evaluate the pipeline with questions whose answers are known to be on specific scanned pages. Measure recall of the correct page, citation accuracy and refusal behaviour when a scan is too degraded to read. If retrieval misses, check OCR quality before blaming chunking — the fault is often upstream.
Build a reprocessing strategy: when a better OCR engine or a cleaner scan becomes available, re-run OCR for the affected documents, re-embed the new text and retire the old chunks. Keep both versions until the new one passes evaluation. Route answers to small models for simple lookups and stronger models for cross-document synthesis, and deploy in a region that satisfies document policy, including on-prem when scans cannot leave the archive.
Honest comparison
| Capability | OCR plus RAG | Filename search | Full manual archive |
|---|---|---|---|
| Search unit | Text passages with page anchors | File names and folders | Human memory |
| Citations | Document and page level | None | Manual |
| Scanned content | Searchable after OCR | Invisible | Depends on person |
| Confidence | OCR scores stored per chunk | None | None |
| Residency | Region-locked planes, VPC, on-prem, air-gapped | Local systems | Physical |
Frequently asked questions
Does Plugsky perform OCR?
No. Use a dedicated OCR engine and treat its output as your corpus. Plugsky provides the live embedding and chat endpoints; image generation endpoints are coming soon.
How do we cite a scanned page?
Store document ID, page number and, where available, a bounding region per chunk, then include them in the answer's sources array so the UI can open the scan at that page.
What if OCR quality is poor?
Flag it. Store confidence per chunk, demote low-confidence passages, and let the answer state that the source text is degraded. Re-run OCR when better scans or engines are available.
How should tables be handled?
Serialise tables with repeated headers into single chunks so rows stay aligned. Splitting a table across chunks is one of the most common causes of wrong answers over scanned records.
Which embedding model fits scanned archives?
plugsky-embed for single-language archives; plugsky-embed-multilingual when the archive mixes languages such as Arabic and English.
How do we evaluate the pipeline?
Use questions with known answer pages, then measure page recall, citation accuracy and refusal behaviour on unreadable scans. Check OCR quality first when recall drops.
Can this run on-prem?
Yes. Plugsky supports region-locked cloud planes plus VPC, on-prem and air-gapped deployment, so scanned archives can remain inside your network.