How do you call Plugsky from Rust?

Key facts

Chat endpoint	POST https://api.plugsky.com/v1/chat/completions
Rust client	OpenAI-compatible Rust SDK per the docs; async-openai is the common community crate
Rust version	1.74+ per the Plugsky SDK reference
Async runtimes	tokio and async-std supported; sync wrappers available
Base URL	with_api_base("https://api.plugsky.com/v1")
Auth	API key from PLUGSKY_API_KEY, sent as Authorization: Bearer
Streaming and tools	create_stream with StreamExt; tools and response_format via typed request builders
Product status	Live; audio, images, batch and fine-tuning are coming soon

TL;DR

• One config builder reaches 30+ models with the Plugsky base URL and key.
• Requests stay typed: model, messages and tools are structs.
• Streaming is a futures Stream you poll until done.
• serde handles tool schemas and JSON mode output.
• Bound your HTTP timeouts — an unbounded Rust client will wait forever.

How it works, step by step

1. Create a Cargo project and add an OpenAI-compatible client plus tokio and futures.
2. Build an OpenAIConfig with your API key and api.plugsky.com base URL.
3. Send a ChatCompletion request with model plugsky-pro and read the first choice.
4. Stream with create_stream and poll the returned stream to completion.
5. Define tools with serde_json schemas and handle tool call responses.
6. Configure reqwest timeouts, connection reuse and retry policy.
7. Log token usage and status codes for every call.

Original data

Try it yourself

Open the OpenAI-compatible API tester →

Add an OpenAI-compatible client

The Plugsky SDK reference lists Rust 1.74+ with tokio, async-std and sync styles. The widely used community crate for the OpenAI-compatible endpoint is async-openai. Add it with tokio and futures:

cargo add async-openai tokio --features tokio/full
cargo add futures

Configuration is a small builder; the only Plugsky-specific values are the API base and your key:

use async_openai::{Client, config::OpenAIConfig};

let config = OpenAIConfig::new()
    .with_api_key(std::env::var("PLUGSKY_API_KEY")?)
    .with_api_base("https://api.plugsky.com/v1");

let client = Client::with_config(config);

Reuse the Client across requests. It wraps an HTTP connection pool, and rebuilding it per call throws away keep-alive connections under load.

Your first completion

Requests are strongly typed. The message list, model id and optional parameters are all builder fields, so a typo in temperature is a compile error rather than a 400 at runtime:

use async_openai::types::{
    ChatCompletionRequestUserMessageArgs, CreateChatCompletionRequestArgs,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let request = CreateChatCompletionRequestArgs::default()
        .model("plugsky-pro")
        .messages([ChatCompletionRequestUserMessageArgs::default()
            .content("Explain this borrow-checker error in one paragraph.")
            .build()?
            .into()])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

Streaming with futures

For interactive output, request a stream and drive it with StreamExt:

use futures::StreamExt;

let request = CreateChatCompletionRequestArgs::default()
    .model("plugsky-lite")
    .messages([/* ... */])
    .build()?;

let mut stream = client.chat().create_stream(request).await?;

while let Some(result) = stream.next().await {
    let response = result?;
    for choice in response.choices {
        if let Some(content) = choice.delta.content {
            print!("{content}");
        }
    }
}

Because Plugsky returns server-sent events in the OpenAI format, the crate decodes frames for you. Wire the stream behind a bounded buffer if you forward tokens to slow consumers — back-pressure is cheaper than unbounded memory growth.

Tools, JSON mode and serde

Tool calling uses the typed tool builders: define each function name, description and a JSON-schema parameter object, pass them on the request, then branch when the response contains tool calls, execute the function, append the tool output as a new message, and call the model again. The schemas are plain json! values, close to the wire format and easy to test.

For JSON mode, set the response format on the request and deserialize into your own struct with serde_json::from_str. Keep a fallback path for parse failures: retry once with the error included, then surface a human-review path rather than writing malformed data downstream.

Timeouts, retries and operations

Rust will await a hung connection forever. Configure explicit connect and read timeouts, and cap total retries. Retry 429 and 5xx responses with backoff and honor Retry-After. For POSTs that create resources, send an Idempotency-Key header so a retried request returns the cached result instead of duplicating work. Track status codes and token usage; errors follow the OpenAI schema. Batch, audio, images and fine-tuning endpoints are coming soon, so check the roadmap before designing around them.

Honest comparison

Capability	Plugsky + async-openai	reqwest directly	Self-hosted inference engine
Setup	Configure base URL and keep typed builders	Define structs, headers and SSE parsing	Run and operate a model server
Streaming	create_stream plus StreamExt	Manual SSE decoding	Engine-dependent
Tools and JSON	Typed tool structs and serde	json! macros and manual validation	Varies by runtime
Async runtimes	tokio or async-std	Whatever you already use	Your infrastructure
Error handling	Typed errors you match on	Status codes and JSON bodies	Varies
Ops	Managed API	Managed API	GPUs, scaling and upgrades

Frequently asked questions

Which Rust client works with Plugsky?

Any OpenAI-compatible client that supports a custom API base works. The Plugsky SDK reference lists Rust 1.74+ support; async-openai is the common community crate used in the examples above.

Do I need tokio?

tokio is the most common runtime, and the docs also list async-std and sync styles. Pick whichever runtime your service already uses.

How do I stream responses?

Call client.chat().create_stream(request) and poll the returned stream with futures::StreamExt until it ends, printing delta content.

Is function calling supported?

Yes. Build typed tool definitions from JSON schemas, handle tool-call responses, execute the functions and append results before the next request.

How should I handle rate limits?

Retry 429 and 5xx with bounded exponential backoff and honor Retry-After. Use Idempotency-Key on POSTs that could be replayed.

Can I deserialize JSON mode output?

Yes. Request JSON mode and deserialize with serde_json into your own structs, keeping a retry path for parse failures.

What about embedded or no-async use?

The docs list sync styles, so you can block on a small runtime or use a blocking client. Keep the timeout explicit either way.

What is coming soon?

Audio, images, moderation, files, batch, fine-tuning, assistants and the responses endpoint are roadmap items; chat, streaming, tools, embeddings and vision are live.