Last Updated on October 29, 2022

You can get the details of the thread used to initialize worker threads and to execute tasks in the ThreadPool.

Additionally, the ThreadPool uses helper threads internally to help with the management of the ThreadPool. We can get the details of these threads in the main thread that creates the ThreadPool.

In this tutorial, you will discover how to get the details of threads used by the ThreadPool in Python.

Let’s get started.

Need Details of Threads in the ThreadPool

The multiprocessing.pool.ThreadPool in Python provides a pool of reusable threads for executing ad hoc tasks.

A thread pool object which controls a pool of worker threads to which jobs can be submitted.

— multiprocessing — Process-based parallelism

The ThreadPool class extends the Pool class. The Pool class provides a pool of worker processes for process-based concurrency.

Although the ThreadPool class is in the multiprocessing module it offers thread-based concurrency and is best suited to IO-bound tasks, such as reading or writing from sockets or files.

A ThreadPool can be configured when it is created, which will prepare the new threads.

We can issue one-off tasks to the ThreadPool using methods such as apply() or we can apply the same function to an iterable of items using methods such as map().

Results for issued tasks can then be retrieved synchronously, or we can retrieve the result of tasks later by using asynchronous versions of the methods such as apply_async() and map_async().

When using the ThreadPool, we may need the names of the thread used to initialize and execute tasks in worker threads.

This may be for many reasons, although it is likely to aid in designing the thread-safe handling of data or program state in worker threads or to help with debugging.

How can we get the worker thread’s names in Python?

How to Get Threads Used in ThreadPool

We can get the current thread via the threading.current_thread() function.

This will return a threading.Thread instance that can be used to get the details of the current thread in situations such as:

1. A worker threads in the ThreadPool executing a task.
2. Initializing a worker thread in the ThreadPool.

For example, we may have questions such as:

• Are worker threads initialized using the main thread or in a worker thread?

You can learn more about utility functions for getting threads in the tutorial:

• How to Use Thread Utility Functions in Python

Additionally, the ThreadPool itself uses worker threads. We can get the details of these threads, to help differentiate them from any other threads we may have in our program.

Let’s take a closer look at each one of these aspects in turn.

Thread Used to Execute Task

We can get the details of the current thread executing a task in the ThreadPool.

This can be achieved by calling the threading.current_thread() function to get the threading.Thread instance for the thread executing the task in the worker thread.

For example, we can call this function in a custom function executed by the ThreadPool:

Thread Used to Initialize Worker

We can get the details of the current thread initializing a task in the ThreadPool.

This can be achieved by calling the threading.current_thread() function to get the threading.Thread instance for the thread executing the function used to initialize in the worker thread.

For example, we can call this function in a custom function executed used to initialize workers in the ThreadPool:

We can then configure the ThreadPool to call the custom function when initializing worker threads.

You can learn more about how to initialize worker threads in the ThreadPool in the tutorial:

• ThreadPool Initialize Worker Threads in Python

Threads Used to Manage ThreadPool

The multiprocessing.pool.Pool class (that the ThreadPool extends) uses worker threads internally to help manage the pool.

Looking at the source code for the Pool class, we can see that at least three threads are created with responsibilities such as:

• Handling the creation of workers.
• Handling the dispatching of tasks to workers.
• Handing the task results from workers.

In the main thread that creates the ThreadPool, we can get a list of all running threads and inspect the details of the threads associated with the ThreadPool.

We can call the threading.enumerate() function to get a list of all actively running threads. This will return a list of threading.Thread instances, one for each running thread.

For example:

We can get this list of threads after the ThreadPool has been created, and ideally after tasks have already been issued so that all workers and maintenance threads have been started.

Now that we know how to get the details of threads associated with the ThreadPool, let’s look at some worked examples.

Example of Getting Thread Used to Execute Task

We can explore how to get the details of the thread used to execute tasks in the ThreadPool.

In this example, we will define a custom function to be executed in the ThreadPool. The task will get the current thread and report its details. The main thread will create the ThreadPool and issue the task into the ThreadPool.

Firstly, we can define a custom function to execute as a task in the ThreadPool.

The function will take an argument and then get the threading.Thread instance for the current thread and report the thread name. We could report other details of the thread as well.

The task() function below implements this.

You can learn more about the details of a thread in the tutorial:

• How to Query Thread Attributes in Python

Next, in the main thread, we will create the ThreadPool with the default configuration.

We will use the context manager interface to ensure that the ThreadPool is closed automatically once we are finished with it.

You can learn more about the context manager interface in the tutorial:

• How to Use the ThreadPool Context Manager

We will issue a single call to the custom function as a task in the ThreadPool.

This can be achieved via the apply_async() function that will issue the task asynchronously and return an AsyncResult as a handle on the task.

You can learn more about issuing tasks via the apply_async() function in the tutorial:

• How to Use ThreadPool apply_async() in Python

The main thread will then wait for the task to finish via the returned AsyncResult object.

Tying this together, the complete example is listed below.

Running the example first creates the ThreadPool.

The main thread then issues a task to the ThreadPool and then blocks until the task completes.

The custom function runs in the ThreadPool. It gets the current thread and reports its name.

In this case, we can see, as we might expect, that a worker thread is used in the pool to execute tasks, in this case with the name “Thread-1“, a default name assigned to threads

Next, let’s explore how we might check which thread is used to initialize the worker thread.

Example of Getting Thread Used to Initialize Workers

We can explore how to get the details of the thread used to initialize the worker threads.

In this example, we will define a custom function to initialize the worker threads. The initialization function will get the current thread and report its details. To ensure that the worker threads are initialized, we will issue one task into the ThreadPool that will just block for a fraction of a second.

Firstly, we can define the custom function that will initialize the worker threads.

The function does not take any arguments or return any values. It will get the threading.Thread instance for the current thread, then report its name.

The init_worker() function below implements this.

Next, we can define a custom function to execute a task in the ThreadPool.

The task will take an argument and block for a fraction of a second.

The task() function below implements this.

Finally, in the main thread, we can create the ThreadPool with the default configuration using the context manager interface.

We can issue the task using the apply_async() function, get an AsyncResult in return immediately, then wait on the object for the task to complete.

Tying this together, the complete example is listed below.

Running the example first creates the ThreadPool.

The main thread then issues the task into the ThreadPool and waits for it to complete.

The ThreadPool is created and starts a default number of worker threads, eight in this case, although the number of threads will differ depending on the number of CPU cores in your system.

Each worker thread is then initialized using the custom initialization function that gets the current thread and reports its name.

In this case, we can see that each worker thread is initialized using the main thread of the worker thread.

This is not surprising.

Next, let’s investigate the worker threads created to help manage the ThreadPool.

Example of Getting Threads Used To Manage the ThreadPool

We can explore the threads created automatically by the ThreadPool.

Looking at the source code for the multiprocessing.pool.Pool class (that is the parent of the ThreadPool class), we can see that helper threads are created by the process pool.

A brief review of the code suggests that the roles of these helper functions might be to:

• Help to create workers.
• Help with dispatching work to workers.
• Help handling results sent back from workers.

In this example, we will create a ThreadPool and issue tasks and wait for them to complete. We will then get a list of all active threads in the main thread and report their details.

Firstly, we can define a custom function to execute in the ThreadPool.

The task does not need to do anything other than help start up the ThreadPool completely.

In this case, the task will take an argument and block for a fraction of a second.

The task() function below implements this.

Next in the main thread, we will create the ThreadPool using the default configuration. We will use the context manager interface to ensure the pool is closed automatically once we are finished with it.

Next, we will issue 10 tasks to the ThreadPool, calling our custom task() function with different integer values from 0 to 9.

This can be achieved with the map_async() function that will issue the tasks asynchronously and return immediately with an AsyncResult object. We can then wait on the object until the issued tasks are completed.

You can learn more about the map_async() function in the tutorial:

• How to Use ThreadPool map_async() in Python

Next, we can get a list of all active threads in the main thread and report the details of each.

Tying this together, the complete example is listed below.

Running the example first creates the ThreadPool

The 10 tasks are then issued to the ThreadPool and the main thread blocks until they are complete.

Each task runs in the ThreadPool, blocks for a moment, then returns.

The main thread continues on and gets a list of all active threads and reports their details.

We can see the main thread, used to execute the program with the type _MainThread and the name of MainThread.

In this case, we can see eight worker threads with the type DummyProcess and names Thread-1 to Thread-8.

We can then see the three helper threads that have the type Thread and names Thread-9 to Thread-11.

All worker threads and helper threads are daemon threads, meaning they will not prevent the main program from exiting.

Unfortunately, the helper threads and workers have ambiguous default, e.g. “Thread-1” to “Thread-11“. Hopefully, they are renamed in a future version of Python to indicate their association with the ThreadPool.

Further Reading

This section provides additional resources that you may find helpful.

Books

• Python ThreadPool Jump-Start, Jason Brownlee (my book!)
• Threading API Interview Questions
• ThreadPool PDF Cheat Sheet

I also recommend specific chapters from the following books:

• Python Cookbook, David Beazley and Brian Jones, 2013.
  • See: Chapter 12: Concurrency
• Effective Python, Brett Slatkin, 2019.
  • See: Chapter 7: Concurrency and Parallelism
• Python in a Nutshell, Alex Martelli, et al., 2017.
  • See: Chapter: 14: Threads and Processes

Guides

• Python ThreadPool: The Complete Guide
• Python Multiprocessing Pool: The Complete Guide
• Python ThreadPoolExecutor: The Complete Guide
• Python Threading: The Complete Guide

APIs

• multiprocessing - Process-based parallelism

References

• Thread (computing), Wikipedia.
• Process (computing), Wikipedia.
• Thread Pool, Wikipedia.

Takeaways

You now know how to get the details of threads used by the ThreadPool.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by David M. Chambers on Unsplash