Building AI-Powered Spring Boot Apps with Spring AI#

If you’re working with Spring Boot and feeling left out of the AI wave, don’t. Spring AI brings AI integration to the Spring ecosystem without forcing you to rewrite everything in Python.

I’ll walk through building an AI powered REST API using real code. We’ll create an article summarizer that actually works in production.

Why Bother?#

Your stack is already Spring Boot. Your team knows Spring. Your infrastructure runs Spring. Why add Python microservices or learn new frameworks when Spring AI works with what you have?

You get familiar patterns (ChatClient feels like RestTemplate), dependency injection, auto configuration, and support for multiple AI providers like OpenAI, Azure, Anthropic, or local models via Ollama.

The Project#

We’re building a REST API that takes articles and returns AI-generated summaries. You’ll be able to specify summary length and get actual production-ready code.

Setup#

Here are the Maven dependencies:

1
<project>
2
    <parent>
3
        <groupId>org.springframework.boot</groupId>
4
        <artifactId>spring-boot-starter-parent</artifactId>
5
        <version>3.2.0</version>
6
    </parent>
7

8
    <dependencies>
9
        <!-- Spring Boot Web for REST API -->
10
        <dependency>
11
            <groupId>org.springframework.boot</groupId>
12
            <artifactId>spring-boot-starter-web</artifactId>
13
        </dependency>
14

15
        <!-- Spring AI for OpenAI integration -->
16
        <dependency>
17
            <groupId>org.springframework.ai</groupId>
18
            <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
19
            <version>1.0.0-M1</version>
20
        </dependency>
21

22
        <!-- For API documentation (optional but recommended) -->
23
        <dependency>
24
            <groupId>org.springdoc</groupId>
25
            <artifactId>springdoc-openapi-starter-webmvc-ui</artifactId>
26
            <version>2.3.0</version>
27
        </dependency>
28
    </dependencies>
29
</project>

Configuration in application.yml:

1
spring:
2
  ai:
3
    openai:
4
      api-key: ${OPENAI_API_KEY}
5
      chat:
6
        options:
7
          model: gpt-4
8
          temperature: 0.7
9
          max-tokens: 500
10

11
# Application-specific settings
12
app:
13
  ai:
14
    rate-limit:
15
      requests-per-minute: 10
16
    timeout-seconds: 30

Set your API key as an environment variable:

1
export OPENAI_API_KEY=sk-your-key-here

Core Concepts#

ChatClient#

The main interface. Send a string, get a string back:

1
@Autowired
2
private ChatClient chatClient;
3

4
public String chat(String message) {
5
    return chatClient.call(message);
6
}

ChatOptions#

Control AI behavior with options:

1
ChatOptions options = OpenAiChatOptions.builder()
2
    .withModel("gpt-4")
3
    .withTemperature(0.7)  // 0 = deterministic, 1 = creative
4
    .withMaxTokens(500)
5
    .build();

Temperature controls randomness (0 = deterministic, 1 = creative). Max tokens limits response length. Set these globally or per-request.

PromptTemplate#

Don’t hardcode prompts. Use templates:

1
String template = """
2
    Summarize the following article in {length} sentences.
3

4
    Title: {title}
5
    Content: {content}
6

7
    Summary:
8
    """;
9

10
PromptTemplate promptTemplate = new PromptTemplate(template);
11
Prompt prompt = promptTemplate.create(Map.of(
12
    "length", "3",
13
    "title", article.getTitle(),
14
    "content", article.getContent()
15
));
16

17
String summary = chatClient.call(prompt).getResult();

Building the Summarizer#

Domain Model#

1
package com.example.aisummary.model;
2

3
public record Article(String title, String content) {
4
    public Article {
5
        if (title == null || title.isBlank()) {
6
            throw new IllegalArgumentException("Title cannot be blank");
7
        }
8
        if (content == null || content.isBlank()) {
9
            throw new IllegalArgumentException("Content cannot be blank");
10
        }
11
    }
12
}
13

14
public record SummaryRequest(Article article, SummaryLength length) {}
15

16
public enum SummaryLength {
17
    SHORT(3),
18
    MEDIUM(5),
19
    LONG(10);
20

21
    private final int sentences;
22

23
    SummaryLength(int sentences) {
24
        this.sentences = sentences;
25
    }
26

27
    public int getSentences() {
28
        return sentences;
29
    }
30
}
31

32
public record SummaryResponse(String summary, int tokensUsed) {}

The Service#

1
package com.example.aisummary.service;
2

3
import org.springframework.ai.chat.ChatClient;
4
import org.springframework.ai.chat.ChatResponse;
5
import org.springframework.ai.chat.messages.UserMessage;
6
import org.springframework.ai.chat.prompt.Prompt;
7
import org.springframework.ai.chat.prompt.PromptTemplate;
8
import org.springframework.ai.openai.OpenAiChatOptions;
9
import org.springframework.stereotype.Service;
10

11
import java.util.Map;
12

13
@Service
14
public class ArticleSummarizerService {
15

16
    private static final String SUMMARY_TEMPLATE = """
17
        You are a professional editor summarizing articles for busy readers.
18

19
        Summarize the following article in exactly {length} concise sentences.
20
        Focus on the main points and key takeaways.
21

22
        Article Title: {title}
23

24
        Article Content:
25
        {content}
26

27
        Summary (exactly {length} sentences):
28
        """;
29

30
    private final ChatClient chatClient;
31
    private final PromptTemplate promptTemplate;
32

33
    public ArticleSummarizerService(ChatClient chatClient) {
34
        this.chatClient = chatClient;
35
        this.promptTemplate = new PromptTemplate(SUMMARY_TEMPLATE);
36
    }
37

38
    public SummaryResponse summarize(SummaryRequest request) {
39
        // Build the prompt with variables
40
        Prompt prompt = promptTemplate.create(Map.of(
41
            "title", request.article().title(),
42
            "content", truncateContent(request.article().content()),
43
            "length", String.valueOf(request.length().getSentences())
44
        ));
45

46
        // Configure for this specific request
47
        OpenAiChatOptions options = OpenAiChatOptions.builder()
48
            .withTemperature(0.5)  // Lower for consistent summaries
49
            .withMaxTokens(calculateMaxTokens(request.length()))
50
            .build();
51

52
        // Make the AI call
53
        ChatResponse response = chatClient.call(
54
            new Prompt(prompt.getContents(), options)
55
        );
56

57
        // Extract the result
58
        String summary = response.getResult()
59
            .getOutput()
60
            .getContent();
61

62
        int tokensUsed = response.getMetadata()
63
            .getUsage()
64
            .getTotalTokens();
65

66
        return new SummaryResponse(summary, tokensUsed);
67
    }
68

69
    private String truncateContent(String content) {
70
        // Limit to ~4000 tokens worth of text (~3000 words)
71
        // GPT-4 has 8k context window, leave room for prompt + response
72
        int maxChars = 15000;
73
        return content.length() > maxChars
74
            ? content.substring(0, maxChars) + "..."
75
            : content;
76
    }
77

78
    private int calculateMaxTokens(SummaryLength length) {
79
        // Rough estimate: each sentence needs ~20-30 tokens
80
        return length.getSentences() * 30 + 50; // +50 for buffer
81
    }
82
}

Key points:

Truncate content: GPT-4 has an 8k token limit (~6k words). We truncate long articles.
Dynamic token limits: Different summary lengths need different token budgets.
Track metadata: Response includes token usage for cost monitoring.

Error Handling#

1
package com.example.aisummary.exception;
2

3
public class AiServiceException extends RuntimeException {
4
    public AiServiceException(String message, Throwable cause) {
5
        super(message, cause);
6
    }
7
}
8

9
@ControllerAdvice
10
public class GlobalExceptionHandler {
11

12
    private static final Logger log = LoggerFactory.getLogger(GlobalExceptionHandler.class);
13

14
    @ExceptionHandler(AiServiceException.class)
15
    public ResponseEntity<ErrorResponse> handleAiServiceException(AiServiceException ex) {
16
        log.error("AI service error", ex);
17
        return ResponseEntity
18
            .status(HttpStatus.SERVICE_UNAVAILABLE)
19
            .body(new ErrorResponse(
20
                "AI service temporarily unavailable",
21
                "Please try again later"
22
            ));
23
    }
24

25
    @ExceptionHandler(IllegalArgumentException.class)
26
    public ResponseEntity<ErrorResponse> handleValidationException(IllegalArgumentException ex) {
27
        return ResponseEntity
28
            .status(HttpStatus.BAD_REQUEST)
29
            .body(new ErrorResponse(
30
                "Invalid request",
31
                ex.getMessage()
32
            ));
33
    }
34

35
    record ErrorResponse(String error, String message) {}
36
}

And update the service to use it:

1
public SummaryResponse summarize(SummaryRequest request) {
2
    try {
3
        // ... existing code ...
4
    } catch (Exception e) {
5
        throw new AiServiceException(
6
            "Failed to generate summary for: " + request.article().title(),
7
            e
8
        );
9
    }
10
}

REST Controller#

1
package com.example.aisummary.controller;
2

3
import com.example.aisummary.model.*;
4
import com.example.aisummary.service.ArticleSummarizerService;
5
import io.swagger.v3.oas.annotations.Operation;
6
import io.swagger.v3.oas.annotations.tags.Tag;
7
import jakarta.validation.Valid;
8
import org.springframework.http.ResponseEntity;
9
import org.springframework.web.bind.annotation.*;
10

11
@RestController
12
@RequestMapping("/api/v1/summaries")
13
@Tag(name = "Article Summarization", description = "AI-powered article summarization")
14
public class SummaryController {
15

16
    private final ArticleSummarizerService summarizerService;
17

18
    public SummaryController(ArticleSummarizerService summarizerService) {
19
        this.summarizerService = summarizerService;
20
    }
21

22
    @PostMapping
23
    @Operation(
24
        summary = "Summarize an article",
25
        description = "Generate an AI-powered summary of the provided article"
26
    )
27
    public ResponseEntity<SummaryResponse> summarizeArticle(
28
            @Valid @RequestBody SummaryRequest request) {
29

30
        SummaryResponse response = summarizerService.summarize(request);
31
        return ResponseEntity.ok(response);
32
    }
33
}

Testing#

Start your application:

1
mvn spring-boot:run

Then make a request:

1
curl -X POST http://localhost:8080/api/v1/summaries \
2
  -H "Content-Type: application/json" \
3
  -d '{
4
    "article": {
5
      "title": "The Future of Java",
6
      "content": "Java has been around for nearly 30 years, and it is not going anywhere. With modern features like records, pattern matching, and virtual threads, Java is faster and more expressive than ever. The JVM ecosystem continues to innovate, with frameworks like Spring, Quarkus, and Micronaut pushing boundaries..."
7
    },
8
    "length": "SHORT"
9
  }'

Response:

1
{
2
  "summary": "Java remains relevant after 30 years with modern features like records and virtual threads. The JVM ecosystem continues to thrive with innovative frameworks. Java's combination of performance and expressiveness ensures its future.",
3
  "tokensUsed": 127
4
}

You get a clean three-sentence summary as requested.

Production Patterns#

Streaming for Long Responses#

1
@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
2
public Flux<String> streamSummary(@Valid @RequestBody SummaryRequest request) {
3

4
    Prompt prompt = buildPrompt(request);
5

6
    // Stream tokens as they arrive
7
    return chatClient.stream(prompt)
8
        .map(response -> response.getResult().getOutput().getContent());
9
}

Users see progress as the AI generates text.

Rate Limiting#

1
@Component
2
public class RateLimiter {
3

4
    private final Bucket bucket;
5

6
    public RateLimiter(@Value("${app.ai.rate-limit.requests-per-minute}") int requestsPerMinute) {
7
        Bandwidth limit = Bandwidth.classic(
8
            requestsPerMinute,
9
            Refill.intervally(requestsPerMinute, Duration.ofMinutes(1))
10
        );
11
        this.bucket = Bucket.builder().addLimit(limit).build();
12
    }
13

14
    public void checkLimit() {
15
        if (!bucket.tryConsume(1)) {
16
            throw new RateLimitExceededException("Too many requests");
17
        }
18
    }
19
}
20

21
// In service
22
public SummaryResponse summarize(SummaryRequest request) {
23
    rateLimiter.checkLimit();
24
    // ... rest of implementation ...
25
}

Monitoring and Caching#

Track usage with Micrometer metrics and cache identical requests:

1
@Component
2
public class AiMetrics {
3

4
    private final MeterRegistry registry;
5
    private final Counter requestCounter;
6
    private final Timer responseTimer;
7
    private final Counter tokensCounter;
8

9
    public AiMetrics(MeterRegistry registry) {
10
        this.registry = registry;
11
        this.requestCounter = Counter.builder("ai.requests")
12
            .description("Total AI requests")
13
            .register(registry);
14
        this.responseTimer = Timer.builder("ai.response.time")
15
            .description("AI response time")
16
            .register(registry);
17
        this.tokensCounter = Counter.builder("ai.tokens.used")
18
            .description("Total tokens consumed")
19
            .register(registry);
20
    }
21

22
    public void recordRequest() {
23
        requestCounter.increment();
24
    }
25

26
    public void recordResponse(long milliseconds, int tokens) {
27
        responseTimer.record(milliseconds, TimeUnit.MILLISECONDS);
28
        tokensCounter.increment(tokens);
29
    }
30
}

1
@Service
2
public class CachedSummarizerService {
3

4
    @Cacheable(value = "summaries", key = "#request")
5
    public SummaryResponse summarize(SummaryRequest request) {
6
        return summarizerService.summarize(request);
7
    }
8
}

Use Redis for distributed caching in production.

Switching AI Providers#

Spring AI supports multiple providers. Switching is easy:

1
# OpenAI
2
spring:
3
  ai:
4
    openai:
5
      api-key: ${OPENAI_API_KEY}
6

7
# Or Azure OpenAI
8
spring:
9
  ai:
10
    azure:
11
      openai:
12
        api-key: ${AZURE_OPENAI_KEY}
13
        endpoint: ${AZURE_OPENAI_ENDPOINT}
14

15
# Or Anthropic (Claude)
16
spring:
17
  ai:
18
    anthropic:
19
      api-key: ${ANTHROPIC_API_KEY}
20

21
# Or local models with Ollama
22
spring:
23
  ai:
24
    ollama:
25
      base-url: http://localhost:11434

The ChatClient interface stays the same regardless of provider. Great for testing with cheaper models or avoiding vendor lock-in.

Common Mistakes#

Ignoring token limits: Know your model’s context window. Truncate or chunk content.
Wrong temperature: Use 0.0-0.3 for facts, 0.7-1.0 for creative content.
No retry logic: Add @Retryable with exponential backoff.
Exposing raw responses: Validate AI output before returning to users.
Hardcoded prompts: Store prompts in config so you can update without redeploying.

Where This Makes Sense#

AI works well for content generation (summaries, descriptions, emails), analysis (sentiment, classification), conversational interfaces (support bots, Q&A), and code assistance (reviews, documentation).

The key is augmentation, not replacement. AI assists; it doesn’t replace humans.

Performance Tips#

Batch requests when possible
Use GPT-3.5 for simple tasks (10x cheaper, 2x faster)
Process AI calls asynchronously with @Async
Use local models (Ollama) for development

Wrapping Up#

Spring AI integrates cleanly with the Spring ecosystem. The code we built is production-ready—add authentication, persistence, and deploy it.

If you’re wondering how to add AI without rewriting everything in Python, this is your path. Spring AI meets you where you are.

Resources: Spring AI Docs • OpenAI API Reference • Prompt Engineering Guide