ThreadPoolExecutor Idle Threads
The ThreadPoolExecutor will create threads on demand as tasks are issued to the pool.
Worker threads are kept idle and ready in the ThreadPoolExecutor and reused if new tasks are issued to the pool. If the pool is left for an extended period, idle worker threads in the ThreadPoolExecutor are not released or terminated and instead remain ready.
In this tutorial, you will discover how the ThreadPoolExecutor manages idle worker threads.
Let's get started.
How Does ThreadPoolExecutor Handle Idle Threads
The ThreadPoolExecutor provides a pool of reusable worker threads using the executor design pattern.
Tasks executed in new threads are executed concurrently in Python, making the ThreadPoolExecutor appropriate for I/O-bound tasks.
A ThreadPoolExecutor can be created directly or via the context manager interface and tasks can be issued one-by-one via the submit() method or in batch via the map() method.
For example:
...
# create a thread pool
with ThreadPoolExecutor() as tpe:
# issue a task
future = tpe.submit(task)
# the task result once the task is done
result = future.result()
You can learn more about the ThreadPoolExecutor in the tutorial:
The way a thread pool manages its workers can have an impact on performance.
For example, if worker threads are created early, long before tasks are issued to the pool, then the ThreadPoolExecutor may use a lot of memory.
Similarly, if idle worker threads are kept around long after all tasks have been finished, then the memory required by all of the workers will be kept until the ThreadPoolExecutor is closed.
How does the ThreadPoolExecutor manage idle threads?
Are idle threads in the ThreadPoolExecutor released when they are no longer needed?
Idle Threads In The ThreadPoolExecutor
We may consider the way that worker threads are managed in the ThreadPoolExecutor in three ways:
- How are worker threads managed before any tasks are issued?
- How are worker threads managed after tasks have been issued but before new tasks are issued?
- How are worker threads managed long after tasks have been completed?
Let's take a closer look at each of these issues in turn.
Worker Threads Before Tasks Are Issued
The ThreadPoolExecutor does not create worker threads when the pool is created.
Instead, worker threads are created as needed.
ThreadPoolExecutor now reuses idle worker threads before starting max_workers worker threads too.
-- concurrent.futures — Launching parallel tasks
This means that when the ThreadPoolExecutor is created, no worker threads are created. Then as tasks are issued to the pool, new worker threads are created to service those tasks.
You can learn more about when worker threads are started in the tutorial:
Worker Threads After Tasks Are Issued, Before New Tasks
After some tasks have been issued to the ThreadPoolExecutor and then complete, we may need to issue new tasks again to the pool.
At this point, the worker threads created in the pool to service the first batch of tasks will still be present and will be reused to service the second batch of tasks.
New worker threads are only created in the ThreadPoolExecutor when tasks are issued and there are no free worker threads at the time that tasks are issued.
A maximum number of worker threads is created, limited to the "max_workers" argument to the ThreadPoolExecutor.
This defaults to the number of CPUs in your system plus four.
You can learn more about the maximum number of worker threads in the tutorial:
Worker Threads After All Tasks Are Completed
After all tasks in the ThreadPoolExecutor have been completed, the worker threads will sit idle.
The ThreadPoolExecutor will not close the idle threads, regardless of how long they have sat idle.
The threads are only closed after the ThreadPoolExecutor is shut down, either explicitly via the shutdown() method or implicitly by exiting the Python interpreter.
This means that if a large number of worker threads are used for a task, e.g. hundreds or thousands, and all tasks are finished, it is important to shut down the ThreadPoolExecutor in order to free up those resources.
You can learn more about shutting down the ThreadPoolExecutor in the tutorial:
Now that we know how ThreadPoolExecutor manages idle worker threads, let's look at some worked examples.
Example of No Idle Threads Before Tasks Are Issued
We can explore the case where no worker threads are created in the ThreadPoolExecutor when the pool is created, e.g. not until tasks are issued.
In this example, we will create a ThreadPoolExecutor, wait a moment for it to start up completely, then report all running threads in the process. This should show that no worker threads have been created.
The complete example is listed below.
# SuperFastPython.com
# example of no idle threads after threadpoolexecutor is created
from concurrent.futures import ThreadPoolExecutor
from time import sleep
import threading
# create a thread pool
tpe = ThreadPoolExecutor()
# wait a moment
sleep(1)
# report all running threads
for thread in threading.enumerate():
print(thread)
# shutdown the thread pool
tpe.shutdown()
Running the example first creates the ThreadPoolExecutor.
The main thread then blocks for a moment to allow the ThreadPoolExecutor to fully start up.
The main thread resumes and reports all active threads in the process. We can see that there is only a single active thread, the main threads, and no worker threads.
Finally, the ThreadPoolExecutor is shut down.
<_MainThread(MainThread, started 4689919488)>This highlights that worker threads in the ThreadPoolExecutor are not created when the thread pool is created. Instead, we know that workers are created in demand in the ThreadPoolExecutor as tasks are issued to the pool.
You can learn more about worker threads being created on demand in the tutorial:
Example of Idle Threads Being Reused For New Tasks
We can explore the case of the ThreadPoolExecutor reusing worker threads for new tasks issued to the pool.
In this example, we will create a ThreadPoolExecutor with a fixed number of workers, then issue enough tasks to occupy half of all possible workers, ensuring some workers are started and some capacity for new workers remains in the pool. We will then wait for all tasks to complete and report all active threads, showing the worker threads are running. We will then issue more tasks, wait for them to complete, and show that no additional worker threads have been created, instead that worker threads were reused for the subsequent batch of tasks.
Firstly, we can define a simple task to run in the ThreadPoolExecutor. The task blocks for a moment.
The task() function below implements this.
# task executed in the thread pool
def task():
# block for a moment
sleep(1)
Next, we will create a ThreadPoolExecutor with four worker threads using the context manager interface.
...
# create the thread pool
with ThreadPoolExecutor(4) as tpe:
# ...
You can learn more about the context manager interface in the tutorial:
We will then issue 2 tasks to the pool and wait for them to complete, then report all active threads in the process.
...
# issue two tasks
futures = [tpe.submit(task) for _ in range(2)]
# wait for the tasks to complete
_ = wait(futures)
# report all running threads
print('Running threads:')
for thread in threading.enumerate():
print(f'\t{thread}')
Finally, we will issue one more task to the pool, wait for it to complete, and report all active threads again.
...
# issue one task
futures = [tpe.submit(task)]
# wait for the task to complete
_ = wait(futures)
# report all running threads
print('Running threads:')
for thread in threading.enumerate():
print(f'\t{thread}')
Tying this together, the complete example is listed below.
# SuperFastPython.com
# example of idle threads being reused for new tasks
from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import wait
from time import sleep
import threading
# task executed in the thread pool
def task():
# block for a moment
sleep(1)
# create the thread pool
with ThreadPoolExecutor(4) as tpe:
# issue two tasks
futures = [tpe.submit(task) for _ in range(2)]
# wait for the tasks to complete
_ = wait(futures)
# report all running threads
print('Running threads:')
for thread in threading.enumerate():
print(f'\t{thread}')
# issue one task
futures = [tpe.submit(task)]
# wait for the task to complete
_ = wait(futures)
# report all running threads
print('Running threads:')
for thread in threading.enumerate():
print(f'\t{thread}')
Running the example first creates the ThreadPoolExecutor with four worker threads.
Two tasks are then issued to the thread pool, and return Future objects. The main thread then blocks until the tasks are completed.
Two worker threads are created in the ThreadPoolExecutor, one for each task. The tasks execute, block, resume, and complete.
The two worker threads remain idle in the ThreadPoolExecutor.
The main thread resumes and reports all active threads, showing the main thread and two worker threads in the ThreadPoolExecutor, as we expect.
Next, one additional task is issued to the ThreadPoolExecutor, and the main thread blocks until it is complete.
An existing worker thread in the ThreadPoolExecutor executes the task. The task runs, blocks for a moment then terminates.
The main thread resumes again and reports all active threads. As before, we can see that there are only the main thread and two worker threads running in the process.
This highlights that an additional worker thread was not created for the third task, instead an idle thread in the ThreadPoolExecutor was reused to execute the new task.
Running threads:
<_MainThread(MainThread, started 4718198272)>
<Thread(ThreadPoolExecutor-0_0, started 123145437466624)>
<Thread(ThreadPoolExecutor-0_1, started 123145454256128)>
Running threads:
<_MainThread(MainThread, started 4718198272)>
<Thread(ThreadPoolExecutor-0_0, started 123145437466624)>
<Thread(ThreadPoolExecutor-0_1, started 123145454256128)>
Example of Idle Threads Not Released After Tasks Are Done
We can explore the case where idle threads in the ThreadPoolExecutor are not released.
In this example, we will create a ThreadPoolExecutor and issue tasks, then wait for them to complete. We will then wait some period of time and then report all active threads in the process. This will show that even after an extended period, the idle worker threads are not released.
We can update the above example, and reuse the simple task that blocks.
We will create a thread pool with four workers, then issue 4 tasks to ensure all worker threads are created. The main thread then blocks until all tasks are completed.
...
# create the thread pool
with ThreadPoolExecutor(4) as tpe:
# issue four tasks
futures = [tpe.submit(task) for _ in range(4)]
# wait for the tasks to complete
_ = wait(futures)
Next, the main thread will block for an extended period, 10 minutes (600 seconds) in this case.
...
print('Main thread waiting...')
sleep(600)Finally, the main thread resumes and reports all active threads.
...
# report all running threads
print('Running threads:')
for thread in threading.enumerate():
print(f'\t{thread}')
Tying this together, the complete example is listed below.
# SuperFastPython.com
# example of idle threads not being released after tasks are done
from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import wait
from time import sleep
import threading
# task executed in the thread pool
def task():
# block for a moment
sleep(1)
# create the thread pool
with ThreadPoolExecutor(4) as tpe:
# issue four tasks
futures = [tpe.submit(task) for _ in range(4)]
# wait for the tasks to complete
_ = wait(futures)
# wait a while for any idle threads to be released
print('Main thread waiting...')
sleep(600)
# report all running threads
print('Running threads:')
for thread in threading.enumerate():
print(f'\t{thread}')
Running the example first creates a ThreadPoolExecutor with four worker threads.
Four tasks are issued to the thread pool, and the main thread blocks until the tasks are completed.
The ThreadPoolExecutor creates four worker threads, one for each task issued. The tasks execute, block, then terminate.
The main thread resumes and then blocks for an extended period of ten minutes.
The main thread eventually resumes and reports all active threads. This shows the main thread and the four worker threads.
This example highlights that worker threads in the ThreadPoolExecutor that sit idle for an extended period are not closed or terminated.
Worker threads in the ThreadPoolExecutor are not terminated until the thread pool is shut down.
Main thread waiting...
Running threads:
<_MainThread(MainThread, started 4687715840)>
<Thread(ThreadPoolExecutor-0_0, started 123145478320128)>
<Thread(ThreadPoolExecutor-0_1, started 123145495109632)>
<Thread(ThreadPoolExecutor-0_2, started 123145511899136)>
<Thread(ThreadPoolExecutor-0_3, started 123145528688640)>
Takeaways
You now know how the ThreadPoolExecutor manages idle worker threads.
If you enjoyed this tutorial, you will love my book: Python ThreadPoolExecutor Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.