Last Updated on November 24, 2023

The ThreadPoolExecutor will create threads on demand as tasks are issued to the pool.

Worker threads are kept idle and ready in the ThreadPoolExecutor and reused if new tasks are issued to the pool. If the pool is left for an extended period, idle worker threads in the ThreadPoolExecutor are not released or terminated and instead remain ready.

In this tutorial, you will discover how the ThreadPoolExecutor manages idle worker threads.

Let’s get started.

How Does ThreadPoolExecutor Handle Idle Threads

The ThreadPoolExecutor provides a pool of reusable worker threads using the executor design pattern.

Tasks executed in new threads are executed concurrently in Python, making the ThreadPoolExecutor appropriate for I/O-bound tasks.

A ThreadPoolExecutor can be created directly or via the context manager interface and tasks can be issued one-by-one via the submit() method or in batch via the map() method.

For example:

You can learn more about the ThreadPoolExecutor in the tutorial:

• ThreadPoolExecutor in Python: The Complete Guide

The way a thread pool manages its workers can have an impact on performance.

For example, if worker threads are created early, long before tasks are issued to the pool, then the ThreadPoolExecutor may use a lot of memory.

Similarly, if idle worker threads are kept around long after all tasks have been finished, then the memory required by all of the workers will be kept until the ThreadPoolExecutor is closed.

How does the ThreadPoolExecutor manage idle threads?

Are idle threads in the ThreadPoolExecutor released when they are no longer needed?

Idle Threads In The ThreadPoolExecutor

We may consider the way that worker threads are managed in the ThreadPoolExecutor in three ways:

1. How are worker threads managed before any tasks are issued?
2. How are worker threads managed after tasks have been issued but before new tasks are issued?
3. How are worker threads managed long after tasks have been completed?

Let’s take a closer look at each of these issues in turn.

Worker Threads Before Tasks Are Issued

The ThreadPoolExecutor does not create worker threads when the pool is created.

Instead, worker threads are created as needed.

ThreadPoolExecutor now reuses idle worker threads before starting max_workers worker threads too.

— concurrent.futures — Launching parallel tasks

This means that when the ThreadPoolExecutor is created, no worker threads are created. Then as tasks are issued to the pool, new worker threads are created to service those tasks.

You can learn more about when worker threads are started in the tutorial:

• ThreadPoolExecutor When Are Workers Started

Worker Threads After Tasks Are Issued, Before New Tasks

After some tasks have been issued to the ThreadPoolExecutor and then complete, we may need to issue new tasks again to the pool.

At this point, the worker threads created in the pool to service the first batch of tasks will still be present and will be reused to service the second batch of tasks.

New worker threads are only created in the ThreadPoolExecutor when tasks are issued and there are no free worker threads at the time that tasks are issued.

A maximum number of worker threads is created, limited to the “max_workers” argument to the ThreadPoolExecutor.

This defaults to the number of CPUs in your system plus four.

You can learn more about the maximum number of worker threads in the tutorial:

• Configure Max Workers for the ThreadPoolExecutor

Worker Threads After All Tasks Are Completed

After all tasks in the ThreadPoolExecutor have been completed, the worker threads will sit idle.

The ThreadPoolExecutor will not close the idle threads, regardless of how long they have sat idle.

The threads are only closed after the ThreadPoolExecutor is shut down, either explicitly via the shutdown() method or implicitly by exiting the Python interpreter.

This means that if a large number of worker threads are used for a task, e.g. hundreds or thousands, and all tasks are finished, it is important to shut down the ThreadPoolExecutor in order to free up those resources.

You can learn more about shutting down the ThreadPoolExecutor in the tutorial:

• How to Shutdown the ThreadPoolExecutor in Python

Now that we know how ThreadPoolExecutor manages idle worker threads, let’s look at some worked examples.

Example of No Idle Threads Before Tasks Are Issued

We can explore the case where no worker threads are created in the ThreadPoolExecutor when the pool is created, e.g. not until tasks are issued.

In this example, we will create a ThreadPoolExecutor, wait a moment for it to start up completely, then report all running threads in the process. This should show that no worker threads have been created.

The complete example is listed below.

Running the example first creates the ThreadPoolExecutor.

The main thread then blocks for a moment to allow the ThreadPoolExecutor to fully start up.

The main thread resumes and reports all active threads in the process. We can see that there is only a single active thread, the main threads, and no worker threads.

Finally, the ThreadPoolExecutor is shut down.

This highlights that worker threads in the ThreadPoolExecutor are not created when the thread pool is created. Instead, we know that workers are created in demand in the ThreadPoolExecutor as tasks are issued to the pool.

You can learn more about worker threads being created on demand in the tutorial:

• ThreadPoolExecutor When Are Workers Started

Example of Idle Threads Being Reused For New Tasks

We can explore the case of the ThreadPoolExecutor reusing worker threads for new tasks issued to the pool.

In this example, we will create a ThreadPoolExecutor with a fixed number of workers, then issue enough tasks to occupy half of all possible workers, ensuring some workers are started and some capacity for new workers remains in the pool. We will then wait for all tasks to complete and report all active threads, showing the worker threads are running. We will then issue more tasks, wait for them to complete, and show that no additional worker threads have been created, instead that worker threads were reused for the subsequent batch of tasks.

Firstly, we can define a simple task to run in the ThreadPoolExecutor. The task blocks for a moment.

The task() function below implements this.

Next, we will create a ThreadPoolExecutor with four worker threads using the context manager interface.

You can learn more about the context manager interface in the tutorial:

• How to Use the ThreadPoolExecutor Context Manager

We will then issue 2 tasks to the pool and wait for them to complete, then report all active threads in the process.

Finally, we will issue one more task to the pool, wait for it to complete, and report all active threads again.

Tying this together, the complete example is listed below.

Running the example first creates the ThreadPoolExecutor with four worker threads.

Two tasks are then issued to the thread pool, and return Future objects. The main thread then blocks until the tasks are completed.

Two worker threads are created in the ThreadPoolExecutor, one for each task. The tasks execute, block, resume, and complete.

The two worker threads remain idle in the ThreadPoolExecutor.

The main thread resumes and reports all active threads, showing the main thread and two worker threads in the ThreadPoolExecutor, as we expect.

Next, one additional task is issued to the ThreadPoolExecutor, and the main thread blocks until it is complete.

An existing worker thread in the ThreadPoolExecutor executes the task. The task runs, blocks for a moment then terminates.

The main thread resumes again and reports all active threads. As before, we can see that there are only the main thread and two worker threads running in the process.

This highlights that an additional worker thread was not created for the third task, instead an idle thread in the ThreadPoolExecutor was reused to execute the new task.

Example of Idle Threads Not Released After Tasks Are Done

We can explore the case where idle threads in the ThreadPoolExecutor are not released.

In this example, we will create a ThreadPoolExecutor and issue tasks, then wait for them to complete. We will then wait some period of time and then report all active threads in the process. This will show that even after an extended period, the idle worker threads are not released.

We can update the above example, and reuse the simple task that blocks.

We will create a thread pool with four workers, then issue 4 tasks to ensure all worker threads are created. The main thread then blocks until all tasks are completed.

Next, the main thread will block for an extended period, 10 minutes (600 seconds) in this case.

Finally, the main thread resumes and reports all active threads.

Tying this together, the complete example is listed below.

Running the example first creates a ThreadPoolExecutor with four worker threads.

Four tasks are issued to the thread pool, and the main thread blocks until the tasks are completed.

The ThreadPoolExecutor creates four worker threads, one for each task issued. The tasks execute, block, then terminate.

The main thread resumes and then blocks for an extended period of ten minutes.

The main thread eventually resumes and reports all active threads. This shows the main thread and the four worker threads.

This example highlights that worker threads in the ThreadPoolExecutor that sit idle for an extended period are not closed or terminated.

Worker threads in the ThreadPoolExecutor are not terminated until the thread pool is shut down.

Further Reading

This section provides additional resources that you may find helpful.

Books

• ThreadPoolExecutor Jump-Start, Jason Brownlee, (my book!)
• Concurrent Futures API Interview Questions
• ThreadPoolExecutor Class API Cheat Sheet

I also recommend specific chapters from the following books:

• Effective Python, Brett Slatkin, 2019.
  • See Chapter 7: Concurrency and Parallelism
• Python in a Nutshell, Alex Martelli, et al., 2017.
  • See: Chapter: 14: Threads and Processes

Guides

• Python ThreadPoolExecutor: The Complete Guide
• Python ProcessPoolExecutor: The Complete Guide
• Python Threading: The Complete Guide
• Python ThreadPool: The Complete Guide

APIs

• concurrent.futures - Launching parallel tasks
• threading — Thread-based parallelism

References

• Thread (computing), Wikipedia.
• Futures and promises, Wikipedia.
• Thread pool, Wikipedia.

Takeaways

You now know how the ThreadPoolExecutor manages idle worker threads.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by Luca Zanon on Unsplash