Last Updated on September 12, 2022

The ThreadPoolExecutor implements the Executor abstract class and provides a thread pool in Python.

In this tutorial, you will discover the ThreadPoolExecutor class.

Let’s get started.

What Are Python Threads

A thread refers to a thread of execution by a computer program.

Every Python program is a process with one thread called the main thread used to execute your program instructions. Each process is, in fact, one instance of the Python interpreter that executes Python instructions (Python byte-code), which is a slightly lower level than the code you type into your Python program.

Sometimes we may need to create additional threads within our Python process to execute tasks concurrently.

Python provides real native (system-level) threads via the threading.Thread.

A task can be run in a new thread by creating an instance of the Thread class and specifying the function to run in the new thread via the target argument.

Once the thread is created, it must be started by calling the start() function.

We can then wait around for the task to complete by joining the thread; for example:

We can demonstrate this with a complete example with a task that sleeps for a moment and prints a message.

The complete example of executing a target task function in a separate thread is listed below.

Running the example creates the thread object to run the task() function.

The thread is started and the task() function is executed in another thread. The task sleeps for a moment; meanwhile, in the main thread, a message is printed that we are waiting around and the main thread joins the new thread.

Finally, the new thread finishes sleeping, prints a message, and closes. The main thread then carries on and also closes as there are no more instructions to execute.

This is useful for running one-off ad hoc tasks in a separate thread, although it becomes cumbersome when you have many tasks to run.

Each thread that is created requires the application of resources (e.g. memory for the thread’s stack space). The computational costs for setting up threads can become expensive if we are creating and destroying many threads over and over for ad hoc tasks.

Instead, we would prefer to keep worker threads around for reuse if we expect to run many ad hoc tasks throughout our program.

This can be achieved using a thread pool.

What Are Thread Pools

A thread pool is a programming pattern for automatically managing a pool of worker threads.

The pool is responsible for a fixed number of threads.

• It controls when the threads are created, such as just-in-time when they are needed.
• It also controls what threads should do when they are not being used, such as making them wait without consuming computational resources.

Each thread in the pool is called a worker or a worker thread. Each worker is agnostic to the type of tasks that are executed, along with the user of the thread pool to execute a suite of similar tasks (homogeneous tasks) or dissimilar tasks (heterogeneous tasks) in terms of the function called, function arguments, task duration, and more.

Worker threads are designed to be re-used once the task is completed and provide protection against the unexpected failure of the task, such as raising an exception, without impacting the worker thread itself.

This is unlike a single thread that is configured for the single execution of one specific task.

The pool may provide some facility to configure the worker threads, such as running an initialization function and naming each worker thread using a specific naming convention.

Thread pools can provide a generic interface for executing ad hoc tasks with a variable number of arguments but do not require that we choose a thread to run the task, start the thread, or wait for the task to complete.

It can be significantly more efficient to use a thread pool instead of manually starting, managing, and closing threads, especially with a large number of tasks.

Python provides a thread pool via the ThreadPoolExecutor class.

What Is the ThreadPoolExecutor

The ThreadPoolExecutor extends the Executor and makes use of Future objects.

Let’s take a closer look at these elements, starting with Executor objects.

Python Executors

The concurrent.futures.Executor class is an abstract class, meaning that it cannot be instantiated.

It defines a generic interface for a thread pool and provides two implementations: the concurrent.futures.ThreadPoolExecutor class that manages a pool of threads and the concurrent.futures.ProcessPoolExecutor class that manages a pool of processes.

You can learn more about the Executor Python API here:

• concurrent.futures — Launching parallel tasks

The Executor class defines three methods used to control our thread pool; they are: submit(), map(), and shutdown().

• submit(): Dispatch a function to be executed and return a future object.
• map(): Apply a function to an iterable of elements.
• shutdown(): Shut down the executor.

The Executor is started when the class is created and must be shut down explicitly by calling shutdown(), which will release any resources held by the Executor. We can also shut down automatically, but we will look at that a little later.

The submit() and map() functions are used to submit tasks to the Executor for asynchronous execution.

The map() function operates just like the built-in map() function and is used to apply a function to each element in an iterable object, like a list. Unlike the built-in map() function, each application of the function to an element will happen asynchronously instead of sequentially.

The submit() function takes a function as well as any arguments and will execute it asynchronously, although the call returns immediately and provides a Future object.

Next, let’s take a closer look at Future objects.

Python Futures

A future is a programming pattern that represents a delayed result for an asynchronous task.

It is also sometimes called a promise or a delay. It provides a context for the result of a task that may or may not be executing and a way of getting a result once it is available.

In Python, the concurrent.futures.Future object is returned from an Executor, such as a ThreadPoolExecutor when calling the submit() function to dispatch a task to be executed asynchronously.

In general, we do not create Future objects; we only receive them, and we may need to call functions on them.

There is always one Future object for each task sent into the ThreadPoolExecutor via a call to submit().

The Future object provides a number of helpful functions for inspecting the status of the task, such as: cancelled(), running(), and done() to determine if the task was cancelled, is currently running, or has finished execution.

• cancelled(): Returns True if the task was cancelled before being executed.
• running(): Returns True if the task is currently running.
• done(): Returns True if the task has completed or was cancelled.

A running task cannot be cancelled and a done task could have been cancelled.

A Future object also provides access to the result of the task via the result() function. If an exception was raised while executing the task, it will be re-raised when calling the result() function, or can be accessed via the exception() function.

• result(): Access the result from running the task.
• exception(): Access any exception raised while running the task.

Both the result() and exception() functions allow a timeout to be specified as an argument, which is the number of seconds to wait for a return value if the task is not yet complete. If the timeout expires, then a TimeoutError will be thrown.

Finally, we may want to have the thread pool automatically call a function once the task is completed.

This can be achieved by attaching a callback to the Future object for the task via the add_done_callback() function.

• add_done_callback(): Add a callback function to the task to be executed by the thread pool once the task is completed.

We can add more than one callback to each task and they will be executed in the order they were added. If the task has already completed before we add the callback, then the callback is executed immediately.

Any exceptions raised in the callback function will not impact the task or thread pool.

Now that we are familiar with the Executor and Future classes, let’s take a closer look at the ThreadPoolExecutor.

Python ThreadPoolExecutor

The ThreadPoolExecutor provides a pool of generic worker threads in Python.

The ThreadPoolExecutor was designed to be easy and straightforward to use.

If multithreading was like the transmission for changing gears in a car, then using threading.Thread is a manual transmission (e.g. hard to learn and and use) whereas concurrent.futures.ThreadPoolExecutor is an automatic transmission (e.g. easy to learn and use).

• threading.Thread: Manual threading in Python.
• concurrent.futures.ThreadPoolExecutor: Automatic or “just work” mode for threading in Python.

Executing functions using worker threads in the ThreadPoolExecutor involves first creating the thread pool, then submitting the task into the pool. If the task returns a value, we can then retrieve it when the task is completed.

We can demonstrate this with a small example.

First, let’s define a task that blocks for a moment, then reports a message and returns a value.

Next, we can create the thread pool by calling the constructor of the ThreadPoolExecutor class that will create a number of worker threads for us to use.

We can submit a task into the thread pool by calling the submit() function and specifying the name of the function to run in a worker thread. The task will begin executing as soon as a worker thread is available, in this case, nearly immediately.

This returns a Future object that provides a handle on the task that is executing concurrently.

We can then report a message that we are waiting for the task to complete.

We can then retrieve the return value from this function by calling the result() function on the Future object.

Finally, we can shut down the thread pool and release all of the worker threads.

Tying this together, a complete example of using the Python thread pool class ThreadPoolExecutor is listed below.

Running the example creates the thread pool, submits the task, and waits for the result in the main thread.

The thread pool creates the default number of worker threads and waits for a task. The task is submitted and is consumed by a worker thread in the pool and begins executing.

The task blocks for a moment, reports a message, and returns a value.

Finally, the main thread retrieves the result of the task and closes the thread pool.

We can see that the thread pool is indeed very easy to use once created. We could keep calling submit and issuing tasks to be executed concurrently until we have no further tasks to execute.

Further Reading

This section provides additional resources that you may find helpful.

Books

• ThreadPoolExecutor Jump-Start, Jason Brownlee, (my book!)
• Concurrent Futures API Interview Questions
• ThreadPoolExecutor Class API Cheat Sheet

I also recommend specific chapters from the following books:

• Effective Python, Brett Slatkin, 2019.
  • See Chapter 7: Concurrency and Parallelism
• Python in a Nutshell, Alex Martelli, et al., 2017.
  • See: Chapter: 14: Threads and Processes

Guides

• Python ThreadPoolExecutor: The Complete Guide
• Python ProcessPoolExecutor: The Complete Guide
• Python Threading: The Complete Guide
• Python ThreadPool: The Complete Guide

APIs

• concurrent.futures - Launching parallel tasks
• threading — Thread-based parallelism

References

• Thread (computing), Wikipedia.
• Futures and promises, Wikipedia.
• Thread pool, Wikipedia.

Takeaways

You now know about the ThreadPoolExecutor.

Do you have any questions about the ThreadPoolExecutor?
Ask your questions in the comments below and I will do my best to answer.

Photo by Beau Runsten on Unsplash