Executor framework

Task Execution and Scheduling

https://docs.spring.io/spring-framework/reference/integration/scheduling.html#scheduling-task-executor

The Spring TaskExecutor Abstraction

Executors are the JDK name for the concept of thread pools.

The “executor” naming is due to the fact that there is no guarantee that the underlying implementation is actually a pool. An executor may be single-threaded or even synchronous. Spring’s abstraction hides implementation details between the Java SE and Jakarta EE environments.

Spring’s TaskExecutor interface is identical to the java.util.concurrent.Executor interface. In fact, originally, its primary reason for existence was to abstract away the need for Java 5 when using thread pools. The interface has a single method (execute(Runnable task)) that accepts a task for execution based on the semantics and configuration of the thread pool.

TaskExecutor Types

Spring includes a number of pre-built implementations of TaskExecutor. In all likelihood, you should never need to implement your own. The variants that Spring provides are as follows:

SyncTaskExecutor: This implementation does not run invocations asynchronously. Instead, each invocation takes place in the calling thread. It is primarily used in situations where multi-threading is not necessary, such as in simple test cases.
SimpleAsyncTaskExecutor: This implementation does not reuse any threads. Rather, it starts up a new thread for each invocation. However, it does support a concurrency limit that blocks any invocations that are over the limit until a slot has been freed up. If you are looking for true pooling, see ThreadPoolTaskExecutor, later in this list. This will use JDK 21’s Virtual Threads, when the “virtualThreads” option is enabled. This implementation also supports graceful shutdown through Spring’s lifecycle management.
ConcurrentTaskExecutor: This implementation is an adapter for a java.util.concurrent.Executor instance. There is an alternative (ThreadPoolTaskExecutor) that exposes the Executor configuration parameters as bean properties. There is rarely a need to use ConcurrentTaskExecutor directly. However, if the ThreadPoolTaskExecutor is not flexible enough for your needs, ConcurrentTaskExecutor is an alternative.
ThreadPoolTaskExecutor: This implementation is most commonly used. It exposes bean properties for configuring a java.util.concurrent.ThreadPoolExecutor and wraps it in a TaskExecutor. If you need to adapt to a different kind of java.util.concurrent.Executor, we recommend that you use a ConcurrentTaskExecutor instead. It also provides a pause/resume capability and graceful shutdown through Spring’s lifecycle management.
DefaultManagedTaskExecutor: This implementation uses a JNDI-obtained ManagedExecutorService in a JSR-236 compatible runtime environment (such as a Jakarta EE application server), replacing a CommonJ WorkManager for that purpose.

To configure the rules that the TaskExecutor uses, we expose simple bean properties:

@Bean
ThreadPoolTaskExecutor taskExecutor() {
        ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
        taskExecutor.setCorePoolSize(5);
        taskExecutor.setMaxPoolSize(10);
        taskExecutor.setQueueCapacity(25);
        return taskExecutor;
}

@Bean
TaskExecutorExample taskExecutorExample(ThreadPoolTaskExecutor taskExecutor) {
        return new TaskExecutorExample(taskExecutor);
}

Another example

@Configuration
public class SpringAsyncConfig {

    @Bean(name = "threadPoolTaskExecutor")
    public TaskExecutor threadPoolTaskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(5);  // Minimum number of threads in the pool
        executor.setMaxPoolSize(10);  // Maximum number of threads in the pool
        executor.setQueueCapacity(25);  // Queue capacity for pending tasks
        executor.setThreadNamePrefix("AsyncExecutor-");  // Prefix for thread names
        executor.setWaitForTasksToCompleteOnShutdown(true);  // Ensures tasks complete on shutdown
        executor.setAwaitTerminationSeconds(60);  // Timeout for waiting for tasks to complete
        executor.initialize();  // Initializes the thread pool
        return executor;
    }
}

Explanation:

Use ThreadPoolTaskExecutor: This is the Spring abstraction for creating thread pool executors. It wraps ThreadPoolExecutor and provides additional Spring-specific functionality.
Set Core and Maximum Pool Size: The CorePoolSize is the number of threads that are always kept alive, while MaxPoolSize is the maximum number of threads that can be created if the pool is fully utilized.
Set Queue Capacity: Determines how many tasks can be queued for execution if all threads are busy.
Set Thread Name Prefix: Useful for identifying threads in logs or monitoring tools.
Ensure Proper Shutdown: It’s good practice to ensure that tasks are completed when the application shuts down.

What is the purpose of Executor framework in java?

The Java Executor Service is a higher-level replacement for manually creating and managing threads. It provides a pool of pre-created threads to execute submitted tasks, which can be either Runnable or Callable objects. This separation of task submission from thread execution simplifies concurrent programming, reduces the overhead of creating new threads for each task, and helps manage the number of active threads to prevent resource exhaustion.

The Java Executor framework simplifies the execution of asynchronous tasks by managing a pool of worker threads. Its primary purpose is to decouple task submission from task execution, which helps to improve the performance, responsiveness, and stability of an application. Instead of creating a new thread for every task, the framework reuses existing threads, which reduces the overhead associated with thread creation and destruction.

Key Components and Benefits

The Executor framework is built on three main interfaces:

Executor: This is the core interface with a single method, execute(Runnable command). It accepts a Runnable task and executes it at some point in the future.
ExecutorService: This is a more comprehensive interface that extends Executor. It provides methods for managing the lifecycle of the executor, such as shutdown() and submit(). The submit() method returns a Future object, which represents the result of an asynchronous computation.
ScheduledExecutorService: This sub-interface of ExecutorService allows you to schedule tasks to run after a delay or to execute repeatedly at fixed intervals.

Benefits

Using the Executor framework provides several benefits:

Improved Performance: Reusing threads from a pool significantly reduces the overhead of creating and destroying threads, especially in applications with a high volume of short-lived tasks.
Enhanced Responsiveness: By offloading long-running tasks to a separate thread pool, the main application thread remains free to handle user interface events or other critical operations, preventing the application from freezing.
Resource Management: The framework provides a way to control the number of threads running concurrently, preventing the system from being overwhelmed by a large number of threads.
Simplified Concurrency: It simplifies concurrent programming by abstracting away the complexities of thread creation, management, and synchronization. Developers can focus on the logic of their tasks rather than the low-level details of thread handling.

Common Use Cases

The Executor framework is highly versatile and is used in a wide range of applications, including:

Web Servers: Handling incoming requests by submitting each request to a thread pool.
Database Operations: Performing long-running database queries in the background without blocking the main application flow.
Batch Processing: Processing large datasets by dividing the work into smaller tasks and distributing them among a pool of threads.
GUI Applications: Running tasks that might block the UI thread, such as file I/O or network requests, to keep the user interface responsive.

An example showing Java executor service in action

Here’s an example of a simple Java program that uses an ExecutorService to execute a series of tasks. This program simulates downloading a number of files concurrently. We’ll use a fixed-size thread pool and Runnable tasks.

This example shows how the ExecutorService abstracts away the complexities of manual thread management, allowing you to focus on the tasks themselves.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class ExecutorServiceExample {

    public static void main(String[] args) {

        // Create a fixed-size thread pool with 3 threads
        ExecutorService executorService = Executors.newFixedThreadPool(3);

        // Submit 10 tasks to the executor service
        for (int i = 0; i < 10; i++) {
            final int taskId = i;
            // The executor will pick up these tasks and execute them using the available threads
            executorService.submit(() -> {
                System.out.println("Executing Task " + taskId + " on thread: " + Thread.currentThread().getName());
                try {
                    // Simulate work being done
                    TimeUnit.SECONDS.sleep(1);
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                }
            });
        }

        // Initiate a shutdown, preventing new tasks from being accepted
        executorService.shutdown();

        // Wait for all tasks to complete before exiting the program
        try {
            // Wait for up to 60 seconds for all tasks to complete
            if (!executorService.awaitTermination(60, TimeUnit.SECONDS)) {
                System.err.println("Tasks did not terminate in the specified time.");
            }
        } catch (InterruptedException e) {
            System.err.println("Termination was interrupted.");
        }

        System.out.println("All tasks completed.");
    }
}

executors.newFixedThreadPool(3)

This line creates an ExecutorService that maintains a pool of exactly three worker threads. These threads are created once and are reused to execute submitted tasks.
executorService.submit(() -> { ... })

The submit() method adds a new Runnable task to a queue. The ExecutorService then assigns these tasks to the available threads in the pool. Since we have 10 tasks and only 3 threads, the threads will process the tasks in batches of three. As soon as a thread finishes its task, it becomes available to pick up the next one from the queue.
executorService.shutdown()

This is a crucial step. It tells the ExecutorService to stop accepting new tasks. The service will, however, continue to process all tasks that have already been submitted and are in the queue.
executorService.awaitTermination(...)

After calling shutdown(), we use awaitTermination() to block the main thread until all submitted tasks have completed execution. This ensures that our program doesn’t exit prematurely while tasks are still running in the background. If the tasks don’t finish within the specified time, the method returns false, which can be used for error handling.

Executor framework

Started since java 1.5

See Callable and Runnable

There are a few different ways to delegate tasks for execution to an ExecutorService.

execute(Runnable task):void crates new thread but does not block the main thread or caller thread as this method returns void.
submit(Callable<?>):Future<?>, submit(Runnable):Future<?> crates new thread and blocks main thread when you are using future.get()

java.util.concurrent package

The java.util.concurrent package defines three executor interfaces:

Executor, a simple interface that supports launching new tasks.
ExecutorService, a subinterface of Executor, which adds features that help manage the life cycle, both of the individual tasks and of the executor itself.
ScheduledExecutorService, a subinterface of ExecutorService, supports future and/or periodic execution of tasks.

The Executor Interface

The Executor interface provides a single method, execute, designed to be a drop-in replacement for a common thread-creation idiom. If r is a Runnable object, and e is an Executor object you can replace

(new Thread(r)).start();

with

e.execute(r);

Drawbacks

The Executor implementation, execute may do the same thing, but is more likely to use an existing worker thread to run r, or to place r in a queue to wait for a worker thread to become available.

The ExecutorService

ExecutorService can execute Runnable and Callable tasks.

How to instantiate?

https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/Executors.html

https://github.com/explorer436/programming-playground/tree/main/java-playground/java-multithreading-and-concurrency-demo

Assigning tasks to ExecutorService

We can assign tasks to the ExecutorService using several methods including execute(), which is inherited from the Executor interface, and also submit(), invokeAny() and invokeAll().

The execute() method is void and doesn’t give any possibility to get the result of a task’s execution or to check the task’s status (is it running)

executorService.execute(runnableTask);

submit() submits a Callable or a Runnable task to an ExecutorService and returns a result of type Future:

See LaunchingCallableUsingExecutorService.java

Future<String> future = executorService.submit(callableTask);

invokeAny() assigns a collection of tasks to an ExecutorService, causing each to run, and returns the result of a successful execution of one task (if there was a successful execution):

String result = executorService.invokeAny(callableTasks);

invokeAll() assigns a collection of tasks to an ExecutorService, causing each to run, and returns the result of all task executions in the form of a list of objects of type Future:

List<Future<String>> futures = executorService.invokeAll(callableTasks);

Common pitfalls

Despite the relative simplicity of ExecutorService, there are a few common pitfalls.

Keeping an unused ExecutorService alive: How do we shut down an ExecutorService?
Wrong thread-pool capacity while using fixed length thread pool: It is very important to determine how many threads the application will need to run tasks efficiently. A too-large thread pool will cause unnecessary overhead just to create threads that will mostly be in the waiting mode. Too few can make an application seem unresponsive because of long waiting periods for tasks in the queue.
Calling a Future‘s get() method after task cancellation: Attempting to get the result of an already canceled task triggers a CancellationException.
Unexpectedly long blocking with Future‘s get() method: We should use timeouts to avoid unexpected waits.

References

TODO

KNOWLEDGE GAP - LEARN MORE, IMPLEMENT THIS

ExecutorService - 10 tips and tricks https://nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html
ExecutorCompletionService in practice https://nurkiewicz.com/2013/02/executorcompletionservice-in-practice.html

Links to this note

Multi threading and Concurrency

Task Execution and Scheduling

The Spring TaskExecutor Abstraction

TaskExecutor Types

To configure the rules that the TaskExecutor uses, we expose simple bean properties:

What is the purpose of Executor framework in java?

Key Components and Benefits

Benefits

Common Use Cases

An example showing Java executor service in action

Executor framework

java.util.concurrent package

The Executor Interface

Drawbacks

The ExecutorService

How to instantiate?

Assigning tasks to ExecutorService

Common pitfalls

References

Tags

TODO

Links to this note