Key facts
| Pipeline | OCR engine → agent validation → filing with human review |
| Agent runtime | Function calling loop on /v1/chat/completions (live) |
| Structured output | JSON mode for extracted fields, confidence and exceptions |
| Models | 30+ models behind one endpoint, route extraction and reasoning separately |
| Tools | Vendor matching, duplicate checks, ERP filing and exception queues |
| Vision status | Image generation endpoints are coming soon; keep OCR in a dedicated preprocessing step |
| Audit | Per-document traces with model, tool and decision metadata |
| Free tier | plugsky-micro and plugsky-lite on the free plan, no card required |
TL;DR
- Treat OCR as preprocessing; the agent's job is validation, routing and filing.
- Ask for structured fields with source references so reviewers can verify quickly.
- Check totals, dates and identifiers with deterministic tools, not model judgement.
- Route low-confidence pages to humans and measure that queue like a first-class metric.
- Keep per-document traces so any filing decision can be reconstructed.
How it works, step by step
- Map the document types, their key fields and the systems each record must reach.
- Run a dedicated OCR engine over incoming scans and store text plus per-word confidence.
- Ask the agent for structured extraction through JSON mode: field values, source region and confidence.
- Validate with tools: arithmetic on totals, date sanity, vendor master matching and duplicate detection.
- Route records into three paths — auto-file, review queue and reject — based on validation and confidence.
- Log every decision with the extracted fields, tool results and model metadata for audit.
- Measure straight-through rate, exception rate and correction rate, and widen auto-filing as accuracy holds.
Try it yourself
Open the function calling tester →
OCR first, then agent
Keep the two jobs separate. OCR turns pixels into text and reports confidence per region; the agent reasons over that text and decides what to do with it. Mixing them invites a system that silently guesses at illegible text instead of flagging it. Note that image-generation endpoints are coming soon, so OCR belongs in a dedicated preprocessing step in your pipeline today.
Once text exists, Plugsky's live function calling provides the rest: pass a tools array on /v1/chat/completions, receive tool_calls, execute validation and filing steps, append results and continue. JSON mode gives you field-level records your systems can consume directly.
Tools for validation, matching and filing
Deterministic checks catch most document errors before a human sees them. Recompute totals, verify tax identifiers against checksums, match vendor names against the master list, and detect duplicates by invoice number and amount. The model extracts and explains; tools decide what is arithmetically and logically consistent.
- Totals and tax: recompute line items and tax rather than trusting printed sums.
- Matching: fuzzy-match vendor names but require a threshold and record the match score.
- Duplicates: check invoice number plus amount plus date across systems before filing.
- Filing: write records through idempotent tools keyed on a document hash.
Confidence, review and evaluation
Auto-filing thresholds should be earned. Start with conservative rules: file only high-confidence extractions that pass every validation, and send the rest to a review queue with the extracted fields and source image side by side. Track the correction rate in that queue — it is the direct measure of extraction quality.
Evaluate on a labelled set of real documents, scoring field-level accuracy, straight-through rate and exception routing. Watch for drift when document layouts or suppliers change, and treat new templates as a reason to re-evaluate rather than trust prior numbers. Route extraction to small models and complex reconciliation to stronger ones, keep per-document traces for audit, and deploy in a region that satisfies your document policy — including on-prem when scans cannot leave the building.
Honest comparison
| Concern | Agent-based workflow | OCR plus rules | Manual data entry |
|---|---|---|---|
| Extraction | OCR then structured model output | OCR plus templates | Human typing |
| Validation | Deterministic tools plus model checks | Rules only | Human review |
| Exceptions | Routed with context attached | Hard failures | Everything is review |
| Adaptability | Handles new layouts with prompt changes | Brittle templates | Slow |
| Audit | Per-document traces | Limited logs | Paper trail |
Frequently asked questions
Does Plugsky provide OCR?
Not as a live endpoint. Run a dedicated OCR engine in your pipeline, then use Plugsky's live chat completions for extraction, validation and routing. Image generation endpoints are coming soon.
How do we handle low-confidence pages?
Route them to a review queue with the extracted fields and the source image. Track correction rate there and only lower the confidence threshold when that rate is acceptable.
Should the model compute totals?
No. Recompute totals and tax with tools and compare against the printed values. Model arithmetic is not auditable, and invoice totals are exactly where errors matter.
Is JSON mode enough for records?
It provides the structure, but you still validate fields against your schema and business rules. Reject invalid records to the exception queue rather than repairing them silently.
How do we stop duplicate filings?
Key every filing tool on a document hash and check invoice number, amount and date across systems before writing.
What should we measure?
Field-level accuracy, straight-through rate, exception rate, correction rate and time-to-file. These show where the pipeline actually needs work.
Can this run on-prem?
Yes. Plugsky supports region-locked cloud planes plus VPC, on-prem and air-gapped deployment, so scanned documents can stay inside your network.