Key facts
| Pattern | Expand query → vector search → inspect → refine → return |
| Runtime | Function calling loop on /v1/chat/completions (live) |
| Embeddings | plugsky-embed and plugsky-embed-multilingual via /v1/embeddings (live) |
| Models | 30+ models behind one endpoint, route per step |
| Guardrails | Search and refinement budgets enforced in code |
| Observability | Per-attempt traces of queries, filters and hit scores |
| Deployment | Region-locked planes, VPC, on-prem and air-gapped options |
| Free tier | plugsky-micro and plugsky-lite on the free plan, no card required |
TL;DR
- Let the agent expand the query — users rarely phrase searches the way content is written.
- Expose search as a tool with filters for date, source and type.
- Bound refinements so latency stays predictable.
- Return passages with scores and metadata, not just a ranked list of links.
- Evaluate first-attempt hit rate and refinement success separately.
How it works, step by step
- Index the corpus with plugsky-embed, storing metadata for the filters your users need.
- Expose a search tool that accepts query text, filters, top-k and returns passages with IDs and scores.
- Have the agent expand the query into a few variants that cover synonyms and likely terminology.
- Search in parallel across variants, merge results and deduplicate by passage ID.
- Inspect the merged set for relevance and refine with tighter filters or new terms if needed.
- Return the best passages with metadata and a short explanation of why each matches.
- Trace every attempt and evaluate first-pass hit rate, refinement success and latency.
Original data
Try it yourself
Open the embedding model comparison →
Search as an agent loop
Vector search alone struggles with vocabulary mismatch: users describe needs in their own words while documents use domain terminology. An agent closes that gap by generating query variants, searching each, and inspecting what comes back before deciding whether to refine. The retrieval primitive is unchanged; the loop around it adds resilience.
On Plugsky, retrieval runs on /v1/embeddings and the loop on live function calling over /v1/chat/completions. With 30+ models behind one endpoint, query expansion and relevance judging can run on small, fast models while harder synthesis uses stronger ones.
Query expansion and filters
Expansion should be modest and targeted: a handful of variants that add synonyms, expand acronyms and include likely product or technical terms. More variants mean more latency and diminishing returns. Filters matter just as much — date ranges, source types and document classes narrow the space before scoring.
- Synonyms and acronyms: map user language to domain terminology explicitly.
- Filters as tools: expose them as parameters so the agent can tighten or relax them.
- Hybrid scoring: combine keyword and vector scores so exact identifiers still match.
- Deduplication: collapse near-identical passages before presenting results.
Quality, latency and evaluation
Measure the loop in stages: first-attempt hit rate (did the original query find something useful), refinement success (did extra attempts improve results), and latency per request. If first-attempt hit rate is high, refinements add cost for little gain; if it is low, invest in query expansion and metadata before increasing attempts.
Cache embeddings for repeated queries and expansion templates, and run variant searches in parallel to keep latency flat. Return passages with scores and metadata so downstream interfaces can show why each result matched. Trace every attempt with query text, filters and hit scores — that trace is what turns search tuning from guesswork into measurement. For sensitive corpora, deploy in a region-locked plane or on-prem with Plugsky.
Honest comparison
| Concern | Agentic semantic search | Vector search only | Keyword search |
|---|---|---|---|
| Query handling | Expanded and refined by the agent | Single embedding | Literal terms |
| Vocabulary mismatch | Handled with variants and synonyms | Partial | Poor |
| Filters | Tool parameters the agent can adjust | Fixed | Supported |
| Latency | Bounded by refinement budget | Single round trip | Fast |
| Diagnostics | Per-attempt traces | Single log line | Query logs |
Frequently asked questions
How is this different from plain vector search?
Plain vector search embeds the query once. The agent expands it into variants, inspects results and refines, which handles vocabulary mismatch and vague queries far better.
How many query variants should the agent try?
A small number — typically two or three — searched in parallel. More variants add latency without proportional gains once synonyms and acronyms are covered.
Should I keep keyword search?
Yes. Hybrid retrieval catches exact identifiers, codes and names that embeddings can miss. Merge both score sets before presenting results.
How do we control latency?
Bound refinements, run variant searches in parallel, cache expansion templates and embeddings, and use small models for expansion and judging.
What should the tool return?
Passages with stable IDs, scores and metadata — not just links. Downstream interfaces and citation logic need the text and provenance.
How do we evaluate it?
Track first-attempt hit rate, refinement success and per-request latency, and review traces where the loop failed to converge on relevant passages.
Can it run on-prem?
Yes. Plugsky supports region-locked cloud planes plus VPC, on-prem and air-gapped deployment, and the search index remains under your control.