Key facts
| Pattern | Retrieve candidates → enrich with tools → rank → explain |
| Runtime | Function calling loop on /v1/chat/completions (live) |
| Candidate source | Embedding search over your catalogue via /v1/embeddings (live) |
| Models | 30+ models behind one endpoint, route ranking and explanation separately |
| Business rules | Inventory, pricing and eligibility checked in tools, not prompts |
| Output | JSON mode for ranked items with reasons and constraints |
| Observability | Per-turn traces with retrieved items and rule outcomes |
| Free tier | plugsky-micro and plugsky-lite on the free plan, no card required |
TL;DR
- Retrieval proposes, rules dispose — agents should not invent eligibility.
- Enrich candidates with live inventory and pricing before ranking.
- Ask for a specific reason per item; generic explanations erode trust.
- Keep a deterministic fallback ranking for when the agent fails.
- Measure acceptance and return rates, not just click-through.
How it works, step by step
- Define the recommendation surface — bundle, upsell, content or replacement — and the constraints that apply.
- Expose catalogue search, inventory, pricing and eligibility as tools with typed schemas.
- Retrieve a candidate set with embeddings from the user's context, then enrich each candidate through tools.
- Rank with explicit rules first, and let the model re-order within the compliant set.
- Generate a short reason per item grounded in the enriched facts, using JSON mode for structure.
- Log retrieved candidates, rule outcomes and final ranking for every request.
- Compare against a deterministic baseline on acceptance, return rate and revenue per session.
Original data
Try it yourself
Open the embedding model comparison →
Retrieve, enrich, rank, explain
The useful split is between proposal and authority. Embeddings propose candidates that match the user's context; tools supply the facts that decide whether each candidate is actually offerable; a ranker orders them; the model writes the reason. This keeps business-critical constraints in code while letting language do the part it is good at.
Plugsky's live function calling supports the loop directly: a tools array on /v1/chat/completions, tool_calls returned and executed, results appended, iterate until the ranking is ready. JSON mode gives you structured output your merchandising systems can consume.
Tools for rules and real-time data
Recommendations fail when they suggest something unavailable, ineligible or mispriced. Those checks belong in tools that query the systems of record, not in prompt instructions. Cached catalogue data is fine for retrieval; authoritative checks happen at ranking time.
- Availability: inventory and delivery windows per region.
- Eligibility: contract terms, age or licence restrictions, regional rules.
- Pricing: current price and promotion, computed by your pricing service.
- History: what the user already owns or recently declined, to avoid repeats.
Feedback loops and evaluation
Recommendation quality is measured by outcomes, not relevance scores: acceptance rate, return rate, revenue per session and how often users dismiss items. Capture explicit feedback — accepted, saved, dismissed — alongside implicit signals, and use both to tune retrieval and ranking weights.
Guard against feedback loops that collapse diversity: inject exploration candidates, cap how often the same item can appear, and evaluate on held-out traffic. Use small models for retrieval expansion and candidate filtering, and stronger models for explanation and complex trade-offs. Every turn should be traceable — candidates retrieved, rules evaluated, final order — so a bad recommendation can be explained to the business. Deploy in a region that matches your customer data policy.
Honest comparison
| Concern | Agent-based recommender | Rules-only engine | Black-box ML model |
|---|---|---|---|
| Candidate generation | Embedding retrieval over catalogue | Segments and tags | Learned embeddings |
| Constraint checks | Live tools for stock, price, eligibility | Static rules | Usually post-filtered |
| Explanations | Specific reasons grounded in facts | Template text | Often unavailable |
| Control | Rules decide, model re-orders | Full control, limited nuance | Limited interpretability |
| Observability | Per-turn traces of candidates and rules | Rule logs | Model internals |
Frequently asked questions
Where does the model add value over rules?
In understanding loose intent, blending many weak signals and writing specific explanations. Hard constraints — stock, eligibility, price — still belong in tools and rules.
How do we prevent bad recommendations?
Enrich candidates through live tools before ranking, and keep a deterministic fallback that always returns compliant items if the agent fails or times out.
Is function calling live?
Yes. OpenAI-style function calling and streaming are live on chat completions, which is all the recommendation loop needs.
How do we avoid showing the same items repeatedly?
Track recent impressions and dismissals per user, pass them as a tool, and cap repetition in the ranker rather than asking the model to remember.
What should we measure?
Acceptance rate, return rate, revenue per session and diversity of shown items, compared against your existing ranker on held-out traffic.
How do we keep latency acceptable?
Cache retrieval and enrichment, run tools in parallel, and reserve stronger models for explanation while small models handle filtering.
Can it run on-prem?
Yes. Plugsky supports region-locked cloud planes plus VPC, on-prem and air-gapped deployment, so customer data can stay where policy requires.