How do you build API orchestration with RAG?

Key facts

Retrieval	POST /v1/embeddings with plugsky-embed over API specs, docs and error examples
Execution	POST /v1/chat/completions with retrieved schemas in the prompt and optional function calling
Models	30+ models; plugsky-micro for operation selection, plugsky-pro for multi-step composition
Grounding rule	Reject any call whose parameters are absent from the retrieved schema
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage
RAG status	RAG is live; embeddings plus chat completions are the documented building blocks
Governance	Scoped keys, audit logs and usage analytics across the orchestration path
Roadmap	Batch ingestion is coming soon; index APIs on a schedule you control

TL;DR

• Retrieve operations, not whole specs: chunk by endpoint and operation.
• Store a version tag on every chunk so stale specs cannot answer.
• Reject calls whose parameters do not appear in the retrieved schema.
• Dry-run writes when the API supports it; approval-gate the rest.
• Evaluate operation selection and parameter accuracy separately.

How it works, step by step

1. Collect every API source: OpenAPI documents, auth notes, error catalogues and real request examples.
2. Chunk by operation, keeping the path, method, parameters, response shape and one example together.
3. Embed chunks with plugsky-embed and store them with API name, version and last-updated metadata.
4. At request time, retrieve candidate operations and pass only the top few schemas into the prompt.
5. Compose the call, validate parameters against the retrieved schema, then execute or dry-run.
6. Log chosen operation, version and outcome; re-index when any spec changes.

Original data

Try it yourself

Open the MCP server config generator →

Why retrieve API knowledge instead of prompting it

Large API surfaces break prompt-only orchestration in three ways:

• Volume: a full spec for a hundred endpoints does not fit comfortably in context, and quality degrades as it grows.
• Drift: endpoints, parameters and error codes change; prompts that embed them go stale silently.
• Versioning: clients may run against several versions at once, and each version needs its own grounded schemas.

Retrieval fixes all three by keeping schemas in an index you can refresh independently of application releases, and by sending the model only the operations that match the current request.

Building the API knowledge index

Chunking choices determine whether retrieval returns the right operation:

• One chunk per operation, not per document or per line. Include path, method, auth requirements, parameters, response fields and one realistic example.
• Keep error entries as their own chunks — models plan recovery better when error semantics are retrievable.
• Tag every chunk with API name, version, deprecation status and last-updated date.
• Exclude secrets and internal hostnames. Index shapes and examples, not credentials.
• Re-embed only changed operations by comparing document hashes at ingestion time.

Implementation: retrieve, compose, validate

The runtime path has four stages:

1. Retrieve: embed the user goal, search the API index and select the top operations under a schema budget.
2. Compose: ask the model for a call plan in JSON, referencing only retrieved operations and parameters.
3. Validate: check each call against the retrieved schema, permissions and read/write class before execution.
4. Execute: run reads directly; dry-run or queue writes with idempotency keys.

Store the retrieved operation ids in the trace so a wrong call is traceable to a retrieval miss rather than blamed on the model.

Evaluation and failure modes

Score retrieval and composition separately:

• Operation selection accuracy: did the correct endpoint appear in the retrieved candidates?
• Parameter accuracy: are composed arguments valid against the schema, including required fields and enums?
• End-to-end success: did the orchestrated sequence achieve the goal, counting recoverable errors correctly?
• Staleness: share of answers grounded in a deprecated or outdated version.

Common failures: near-duplicate operations across versions, missing auth context, and chunking that separates a path from its parameters. All are index problems, not model problems, and are fixed by re-chunking rather than by upgrading the alias.

Limitations

RAG grounds orchestration but cannot guarantee correctness, and it adds moving parts:

• Retrieval quality depends on your spec hygiene; undocumented endpoints cannot be retrieved.
• Specs describe shapes, not business rules. Keep policy checks in code, never in the index.
• Multi-step transactions still need compensation logic when a later call fails.
• Batch ingestion is coming soon, so schedule re-indexing with your own job runner.

Honest comparison

Capability	RAG API orchestration	Full spec in the prompt	Hand-built wrappers per endpoint
API surface	Retrieves the right operations per request	Limited to what fits context	One wrapper per endpoint
Freshness	Re-index on spec change	Prompt edits and redeploys	Code deploys
Versioning	Version-tagged chunks	One spec at a time	Branches per version
Validation	Against retrieved schema	Manual	Typed code
Failure tracing	Retrieval ids logged per call	Opaque	Stack traces only

Frequently asked questions

What does RAG add to API orchestration?

It narrows a large API surface to the operations relevant to the current goal, so the model composes calls from grounded schemas instead of recalling parameter names from training data.

How should I chunk OpenAPI documents?

One chunk per operation, including path, method, parameters, response fields and a realistic example. Keep error definitions as separate chunks.

Can the model execute my APIs directly?

No. The model composes calls; your runtime validates arguments, checks permissions and executes. Never let generated code reach an unchecked endpoint.

How do I handle multiple API versions?

Tag every chunk with a version, retrieve within the version allowed for the caller, and alert when deprecated operations are still being selected.

What if my APIs are undocumented?

Generate specification entries from traffic captures or code annotations first. Retrieval cannot compensate for missing schemas.

Which model should compose the calls?

plugsky-micro handles operation selection, while plugsky-pro is a better fit for multi-step compositions and conflict resolution across operations.

Can I index specs with the free plan?

Yes. Embeddings are part of the live API, and the free plan includes two free models with no card to start building the index and testing retrieval.