Spring - Concurrency Model

Table of Contents

Reading material
TODO
Overview
Spring Boot Concurrency Basics
How Does the Spring Singleton Bean Serve Concurrent Requests?
Book recommendations
Tags

Reading material

TODO

Overview

When building services with Spring Boot we have to deal with concurrency.
In Spring Boot- Controllers and Services are by default Singletons. That introduces possible concurrency problems if you are not careful. You are also usually dealing with a limited thread-pool. Familiarise yourself with these concepts.
There is this misconception that because of using Servlets and getting a new Thread allocated per request there is no need to think about concurrency.
Concurrency and Multi-Threading in Spring are big and important topics.
If you want to be successful when building high-demand, high-quality services, you need to make conscious decisions and trade-offs around this topics.

Spring Boot Concurrency Basics

The key areas worth considering when thinking about concurrency in Spring Boot applications are:

Maximum number of threads – This is the maximum number of threads that are allocated for dealing with requests to the application
Shared external resources – Calls to external shared resources such as databases
Asynchronous method calls – These are method calls that release the thread back to the thread-pool when waiting for a response
Shared internal resources – Calls to internal shared resources- such as caches and potentially shared application state

Maximum number of threads in Spring Boot Application

The first thing to be aware is that you are dealing with a limited number of threads.

If you are using Tomcat as your embedded server (default), then you can use the property server.tomcat.max-threads to control how many threads you want to allow. This is set to 0 by default which means- use the Tomcat default which is 200.

It is important to know this, as you may need to scale this number to work effectively with the resources that the service is given. It can also become problematic when dealing with external resources…

The problem with shared external resources

Calling databases and other REST endpoints can take significant time.

The limited number of threads that you are dealing with means that you really want to avoid long-running, slow, synchronous requests. If you are waiting for some slow process to complete and holding the thread, you are potentially under-utilizing your server.

If you have many long-running threads that are waiting for responses, you may essentially end up with a situation where really fast, simple requests are waiting for long, “forever-waiting” requests to terminate.

How can this be improved?

Asynchronous method calls to the rescue

It often helps to request for multiple things at once. Ideally, if you need to call three services: Service A, Service B, and Service C; you don’t want to do that:

Call Service A
Wait for a response from Service A
Call Service B
Wait for a response from Service B
Call Service C
Wait for a response from Service C
Compose responses from A, B and C and finish the processing

If each service takes 3 seconds to respond, the whole process would take 9 seconds. It is much better to do the following:

Call Service A
Call Service B
Call Service C
Wait for responses from Service A, B, and C
Compose responses from A, B and C and finish the processing

In this case, you make al three calls without waiting for completion and assuming that services A, B, and C, are not dependent on one another, it takes 3 seconds to respond.

See

Asynchronous calls in Spring applications

Shared internal resources

While the previous sections deal with things we often have no control over- external resources, we are in full control of the internal resources of the system.

Knowing that we control the internal resources, the best advice in avoiding issues related to sharing them, is not to share them!

Spring Services and Controllers are Singletons by default. It is important to be aware of that and be very careful. The moment there is a mutable state in your Service, you need to deal with it as you would in any standard application.

Other potential sources of the shared state are caches and custom, server-wide components (often monitoring, security etc.).

If you absolutely need to share some state, here is my advice:

Deal with immutable objects. You avoid many concurrency related issues if your objects are immutable. If you need to change something- just create a new object.
Know your Collections. Not all collections are Thread-Safe. A common pitfall is using HashMap assuming that it is Thread-Safe (It is not. If you need concurrent access use ConcurrentHashMap, HashTable or another thread-safe solution.).
Do not assume third-party libraries are thread-safe. Most code is not, and access to the shared state has to be controlled.
If you are going to rely on it - learn proper concurrency.

How Does the Spring Singleton Bean Serve Concurrent Requests?

https://www.baeldung.com/spring-singleton-concurrent-requests

How do Spring beans that are created with the singleton scope work behind the scenes to serve multiple concurrent requests?
How does Java store the bean instances in memory and how it handles concurrent access to them?

Spring Beans and Java Heap Memory

The Java heap, as we know, is a globally shared memory accessible to all the running threads within an application. When the Spring container creates a bean with the singleton scope, the bean is stored in the heap. This way, all the concurrent threads are able to point to the same bean instance.

Next, let’s understand what the stack memory of a thread is and how it helps to serve concurrent requests.

How Are Concurrent Requests Served?

As an example, let’s take a Spring application that has a singleton bean called ProductService:

@Service
public class ProductService {
    private final static List<Product> productRepository = asList(
      new Product(1, "Product 1", new Stock(100)),
      new Product(2, "Product 2", new Stock(50))
    );

    public Optional<Product> getProductById(int id) {
        Optional<Product> product = productRepository.stream()
          .filter(p -> p.getId() == id)
          .findFirst();
        String productName = product.map(Product::getName)
          .orElse(null);

        System.out.printf("Thread: %s; bean instance: %s; product id: %s has the name: %s%n", currentThread().getName(), this, id, productName);

        return product;
    }
}

This bean has a method getProductById() which returns product data to its callers. Further, the data returned by this bean is exposed to the clients on the endpoint /product/{id}

Next, let’s explore what happens at runtime when simultaneous calls hit the endpoint /product/{id}. Specifically, the first thread will call the endpoint /product/1 and the second thread will call /product/2.

Spring creates a different thread for each request. As we see in the console output below, both threads use the same ProductService instance to return the product data:

Thread: pool-2-thread-1; bean instance: com.baeldung.concurrentrequest.ProductService@18333b93; product id: 1 has the name: Product 1
Thread: pool-2-thread-2; bean instance: com.baeldung.concurrentrequest.ProductService@18333b93; product id: 2 has the name: Product 2

It’s possible for Spring to use the same bean instance in multiple threads, firstly because for each thread, Java creates a private stack memory (https://www.baeldung.com/java-stack-heap#stack-memory-in-java)

The stack memory is responsible for storing the states of the local variables used inside methods during thread execution. This way, Java makes sure that threads executing in parallel do not overwrite each other’s variables.

Secondly, because ProductService bean sets no restrictions or locks at the heap level, the program counter (https://www.baeldung.com/cs/process-control-block#2-program-counter) of each thread is able to point to the same reference of the bean instance in the heap memory. Therefore, both threads can execute getProdcutById() method simultaneously.

Next, we’ll understand why it’s crucial for singleton beans to be stateless.

Stateless Singleton Beans vs. Stateful Singleton Beans

To understand why stateless singleton beans are important, let’s see what the side effects of using stateful singleton beans are.

Suppose we moved the productName variable to the class level:

@Service
public class ProductService {
    private String productName = null;

    // ...

    public Optional getProductById(int id) {
        // ...

        productName = product.map(Product::getName).orElse(null);

       // ...
    }
}

Now, let’s run the service again and look at the output

Thread: pool-2-thread-2; bean instance: com.baeldung.concurrentrequest.ProductService@7352a12e; product id: 2 has the name: Product 2
Thread: pool-2-thread-1; bean instance: com.baeldung.concurrentrequest.ProductService@7352a12e; product id: 1 has the name: Product 2

As we can see, the call for productId 1 shows the productName “Product 2” instead of “Product 1”. This happens because the ProductService is stateful and shares the same productName variable with all running threads.

To avoid undesired side effects like this, it’s crucial to keep our singleton beans stateless.

Book recommendations

I really recommend getting a copy of Java Concurrency in Practice. Written in 2006, but still very relevant to this day.