Integration · Spring Boot

Spring Boot with an OpenAI-compatible API

Spring AI gives you a typed chat client, streaming and tool calling. Pointing it at an OpenAI-compatible provider is mostly configuration, plus a few retry and test decisions that decide whether production is boring.

Short answer. Add the Spring AI OpenAI starter and BOM, set spring.ai.openai.base-url to https://api.plugsky.com/v1, choose a model ID, and inject the auto-configured ChatClient.Builder. Then tune retries, stream with Flux, and test with a mocked ChatModel.

Dependencies

Spring AI wraps the provider in a Spring Boot auto-configuration. You need the BOM for consistent versions and the OpenAI starter for the chat client. The starter is spring-ai-starter-model-openai; older tutorials use a different artifact name, so check the current reference if it does not resolve.

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.ai</groupId>
      <artifactId>spring-ai-bom</artifactId>
      <version>2.1.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>

Use Spring Boot 3.x and Java 17 or newer. Keep the Spring AI BOM version in one property so chat, embedding and test modules move together. If you also use embeddings, add that starter explicitly.

Configuration

The base URL is what changes when you leave OpenAI. Spring AI appends the chat-completions path, so give it the versioned root, not the full endpoint.

SettingValueWhy
spring.ai.openai.base-urlhttps://api.plugsky.com/v1Versioned API root
spring.ai.openai.api-key${PLUGSKY_API_KEY}Secret from the environment
spring.ai.openai.chat.modelplugsky-proExact provider model ID
spring.ai.retry.max-attempts4Bounded transient retry
spring:
  ai:
    openai:
      base-url: https://api.plugsky.com/v1
      api-key: ${PLUGSKY_API_KEY}
      chat:
        model: plugsky-pro
    retry:
      max-attempts: 4
      backoff:
        initial-interval: 1s
        multiplier: 2
        max-interval: 20s
      on-client-errors: false

Keep the key in an environment variable or secret manager. For separate chat and embedding endpoints, use overrides such as spring.ai.openai.chat.base-url; they take precedence over the common property. Set temperature and output length in code with ChatOptions when a request differs from the default; not every compatible model supports every parameter.

First call

Inject ChatClient.Builder and build one client for the application. Keep the controller thin; put prompt construction in a service so it can be tested without HTTP.

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.*;

@RestController
class SupportController {
    private final ChatClient chat;

    SupportController(ChatClient.Builder builder) {
        this.chat = builder.defaultSystem("Answer in two sentences. If unsure, say so.").build();
    }

    @PostMapping("/api/support")
    Answer ask(@RequestBody Question question) {
        String text = chat.prompt().user(question.question()).call().content();
        return new Answer(text);
    }

    record Question(String question) {}
    record Answer(String answer) {}
}

call() is blocking. Use it for short internal requests where a servlet thread can wait. For a user-facing stream, use the reactive path instead of wrapping a blocking call in a Flux.

Streaming

stream() returns a Flux of content deltas. A text-event-stream endpoint is the smallest useful integration; EventSource or a fetch stream can consume it.

import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.*;
import reactor.core.publisher.Flux;

@RestController
class StreamController {
    private final ChatClient chat;
    StreamController(ChatClient.Builder builder) { this.chat = builder.build(); }

    @GetMapping(value = "/api/support/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    Flux<String> stream(@RequestParam String question) {
        return chat.prompt().user(question).stream().content();
    }
}

Streaming changes the failure model. A 200 can arrive before the first token, and a provider error can occur mid-stream. Send a completion marker, handle disconnects, and do not retry a partially delivered answer without telling the user. If you need usage for billing, check whether the provider returns it in the stream.

Tool calling

Tools turn a chat request into a small agent loop: the model proposes a call, Spring AI invokes the method, and the result goes back. Keep the surface narrow and validate arguments. A tool that refunds an order is a production action, not a prompt trick.

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.stereotype.Component;

@Component
class OrderTools {
    private final OrderService orderService;
    OrderTools(OrderService orderService) { this.orderService = orderService; }

    @Tool(description = "Look up the status of an order by its ID")
    String orderStatus(String orderId) {
        if (!orderId.matches("[A-Z0-9-]{6,20}")) {
            throw new IllegalArgumentException("invalid order id");
        }
        return orderService.status(orderId);
    }
}

// Per request:
String answer = chat.prompt().user("Where is order A1B2C3?")
    .tools(orderTools).call().content();

Register only the tools a request needs. Log the tool name, argument and result, and redact sensitive values. Give each tool a distinct description and typed parameter; ambiguous descriptions produce wrong calls. For destructive actions, require confirmation outside the model loop.

Retries and observability

There are two retry layers: the Spring AI retry template and the OpenAI client's max-retries. Stacking them multiplies attempts. Pick one layer for transient errors and set the other low or off. Do not retry 400, 401 or context-length errors.

  • Timeouts: set connect and read timeouts for the slowest expected completion. Streaming needs a longer read timeout than classification.
  • Backoff: exponential with jitter. A fixed one-second retry against a rate limit makes it worse.
  • Circuit breaking: fail fast with a degraded answer instead of holding every request thread.
  • Metrics: request count, latency, token usage, finish reason and error class by model. Do not put prompt text in a tag.
  • Tracing: Spring AI publishes observations through Micrometer. Add Actuator and a tracing bridge; propagate a request ID.

Read response metadata for token usage and finish reason. A length finish reason means truncation, not completion. A spike in input tokens usually means history or retrieved context is growing; cap both.

Tests

Do not call the live provider in a unit test. Mock the ChatModel and let the auto-configured ChatClient use it. This checks the HTTP contract and answer mapping without a network call.

import org.junit.jupiter.api.Test;
import org.springframework.ai.chat.messages.AssistantMessage;
import org.springframework.ai.chat.model.*;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.MediaType;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc;

import java.util.List;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;

@SpringBootTest
@AutoConfigureMockMvc
class SupportControllerTest {
    @Autowired MockMvc mvc;
    @MockitoBean ChatModel chatModel;

    @Test
    void returnsAnswer() throws Exception {
        when(chatModel.call(any(Prompt.class))).thenReturn(
            new ChatResponse(List.of(new Generation(new AssistantMessage("Refunds take five days."))))
        );
        mvc.perform(post("/api/support").contentType(MediaType.APPLICATION_JSON)
                .content("{\"question\":\"How long do refunds take?\"}"))
            .andExpect(status().isOk())
            .andExpect(jsonPath("$.answer").value("Refunds take five days."));
    }
}

@MockitoBean is the current annotation; older releases use @MockBean. Keep one integration test against a staging key to catch provider changes, marked so it does not run on every local build. Test a 429, a timeout and a truncated answer; each should produce a defined response.

For a plain Java or Kotlin client, see Calling Plugsky from Java and Kotlin. For the raw HTTP shape, see the quickstart.

FAQ

Can Spring AI use an OpenAI-compatible API?

Yes. Set spring.ai.openai.base-url to the provider's versioned root and use a model ID the provider supports. The OpenAI starter speaks the chat-completions shape, so most of the Spring AI client code stays the same.

What base URL should I use for Plugsky in Spring Boot?

Use https://api.plugsky.com/v1. Spring AI appends the chat-completions path, so do not include /chat/completions in the base URL. A doubled path returns 404.

How do I stream responses from Spring AI?

Call stream() instead of call() and return the Flux of content deltas from a text-event-stream endpoint. Handle mid-stream errors and client disconnects; a 200 response does not guarantee the stream completes.

How do I test a Spring AI controller without calling the provider?

Mock the ChatModel bean and stub call(Prompt.class) to return a ChatResponse. The auto-configured ChatClient uses the mock, so the test covers your controller and prompt wiring without a network call.

Should I use Spring AI retries or the OpenAI client retries?

Pick one layer for transient errors. Stacking spring.ai.retry and the client's max-retries multiplies attempts. Do not retry 400, 401 or context-length errors, and add jitter to any retry.

Sources: the Spring AI reference for OpenAI chat (configuration, ChatClient, streaming and tools) and the Spring AI observability reference. Both checked 20 November 2026. Reviewed 20 November 2026. Spring AI and Spring Boot versions change; confirm the property names and artifact IDs for your release.