Use Cases + Implementation

How do you build semantic search with AI agents?

Semantic search agents add reasoning to vector retrieval: the model expands the query, calls the search tool with different phrasings and filters, inspects the results, and refines until it finds passages that actually answer the question. On Plugsky the loop uses live function calling, embeddings for the index, and 30+ models to route expansion and judging cheaply.

Key facts

PatternExpand query → vector search → inspect → refine → return
RuntimeFunction calling loop on /v1/chat/completions (live)
Embeddingsplugsky-embed and plugsky-embed-multilingual via /v1/embeddings (live)
Models30+ models behind one endpoint, route per step
GuardrailsSearch and refinement budgets enforced in code
ObservabilityPer-attempt traces of queries, filters and hit scores
DeploymentRegion-locked planes, VPC, on-prem and air-gapped options
Free tierplugsky-micro and plugsky-lite on the free plan, no card required

TL;DR

  • Let the agent expand the query — users rarely phrase searches the way content is written.
  • Expose search as a tool with filters for date, source and type.
  • Bound refinements so latency stays predictable.
  • Return passages with scores and metadata, not just a ranked list of links.
  • Evaluate first-attempt hit rate and refinement success separately.

How it works, step by step

  1. Index the corpus with plugsky-embed, storing metadata for the filters your users need.
  2. Expose a search tool that accepts query text, filters, top-k and returns passages with IDs and scores.
  3. Have the agent expand the query into a few variants that cover synonyms and likely terminology.
  4. Search in parallel across variants, merge results and deduplicate by passage ID.
  5. Inspect the merged set for relevance and refine with tighter filters or new terms if needed.
  6. Return the best passages with metadata and a short explanation of why each matches.
  7. Trace every attempt and evaluate first-pass hit rate, refinement success and latency.
1Index the corpuswith plugsky-embed,storing metadata2Expose a searchtool that acceptsquery text,3Have the agentexpand the queryinto a few variants4Search in parallelacross variants,merge results and5Inspect the mergedset for relevanceand refine with6Return the bestpassages withmetadata and a

Original data

Function calliRuntimeplugsky-embed Embeddings30+ models behModelsSource: Plugsky facts table · updated 2026-09-26

Try it yourself

Open the embedding model comparison →

Search as an agent loop

Vector search alone struggles with vocabulary mismatch: users describe needs in their own words while documents use domain terminology. An agent closes that gap by generating query variants, searching each, and inspecting what comes back before deciding whether to refine. The retrieval primitive is unchanged; the loop around it adds resilience.

On Plugsky, retrieval runs on /v1/embeddings and the loop on live function calling over /v1/chat/completions. With 30+ models behind one endpoint, query expansion and relevance judging can run on small, fast models while harder synthesis uses stronger ones.

Query expansion and filters

Expansion should be modest and targeted: a handful of variants that add synonyms, expand acronyms and include likely product or technical terms. More variants mean more latency and diminishing returns. Filters matter just as much — date ranges, source types and document classes narrow the space before scoring.

  • Synonyms and acronyms: map user language to domain terminology explicitly.
  • Filters as tools: expose them as parameters so the agent can tighten or relax them.
  • Hybrid scoring: combine keyword and vector scores so exact identifiers still match.
  • Deduplication: collapse near-identical passages before presenting results.

Quality, latency and evaluation

Measure the loop in stages: first-attempt hit rate (did the original query find something useful), refinement success (did extra attempts improve results), and latency per request. If first-attempt hit rate is high, refinements add cost for little gain; if it is low, invest in query expansion and metadata before increasing attempts.

Cache embeddings for repeated queries and expansion templates, and run variant searches in parallel to keep latency flat. Return passages with scores and metadata so downstream interfaces can show why each result matched. Trace every attempt with query text, filters and hit scores — that trace is what turns search tuning from guesswork into measurement. For sensitive corpora, deploy in a region-locked plane or on-prem with Plugsky.

Honest comparison

ConcernAgentic semantic searchVector search onlyKeyword search
Query handlingExpanded and refined by the agentSingle embeddingLiteral terms
Vocabulary mismatchHandled with variants and synonymsPartialPoor
FiltersTool parameters the agent can adjustFixedSupported
LatencyBounded by refinement budgetSingle round tripFast
DiagnosticsPer-attempt tracesSingle log lineQuery logs

Frequently asked questions

How is this different from plain vector search?

Plain vector search embeds the query once. The agent expands it into variants, inspects results and refines, which handles vocabulary mismatch and vague queries far better.

How many query variants should the agent try?

A small number — typically two or three — searched in parallel. More variants add latency without proportional gains once synonyms and acronyms are covered.

Should I keep keyword search?

Yes. Hybrid retrieval catches exact identifiers, codes and names that embeddings can miss. Merge both score sets before presenting results.

How do we control latency?

Bound refinements, run variant searches in parallel, cache expansion templates and embeddings, and use small models for expansion and judging.

What should the tool return?

Passages with stable IDs, scores and metadata — not just links. Downstream interfaces and citation logic need the text and provenance.

How do we evaluate it?

Track first-attempt hit rate, refinement success and per-request latency, and review traces where the loop failed to converge on relevant passages.

Can it run on-prem?

Yes. Plugsky supports region-locked cloud planes plus VPC, on-prem and air-gapped deployment, and the search index remains under your control.