Last Updated on September 12, 2022
You can access the process in a Python program using multiprocessing module functions.
In this tutorial you will discover how to access and query processes in Python using multiprocessing utility functions.
Let’s get started.
Need Thread Utility Functions
A process is a running instance of a computer program.
Every Python program is executed in a Process, which is a new instance of the Python interpreter. This process has the name MainProcess and has one thread used to execute the program instructions called the MainThread. Both processes and threads are created and managed by the underlying operating system.
Sometimes we may need to create new child processes in our program in order to execute code concurrently.
Python provides the ability to create and manage new processes via the multiprocessing.Process class.
You can learn more about multiprocessing in the tutorial:
In some applications we may have many processes and may need to access details about the number of processes that are running or access properties of multiprocessing.Process instances for specific threads.
How can we get access to the processes in a Python program?
Run loops using all CPUs, download your FREE book to learn how.
How to Use Process Utility Functions
There are a number of utilities we can use when working with Python processes.
These utilities are provided as multiprocessing module functions.
Examples of popular multiprocessing module utility functions are as follows:
- active_children(): Returns a list of all active child processes for the current process.
- cpu_count(): Return the number of logical CPU cores in the system.
- current_process(): Return the Process instance for the current process.
- parent_process(): Return the Process instance for the parent of the current process.
Some additional multiprocessing module functions that are less commonly used include the following:
- freeze_support(): Add support for multiprocessing for a frozen Python executable on windows.
- get_all_start_methods(): Return a list of process start methods supported on the system.
- get_context(): Return a process context object.
- get_start_method(): Return the currently configured process start method.
- set_executable(): Set the path to the Python executable for child processes.
- set_start_method(): Set the start method used for new processes.
Now that we are familiar with the multiprocessing module functions, let’s look at some worked examples.
How to Get Active Child Processes
We can get a list of all active child processes for a parent process.
This can be achieved via the multiprocessing.active_children() function that returns a list of all child processes that are currently running.
|
1 2 3 |
... # get a list of all active child processes children = active_children() |
We can demonstrate this with a short example.
|
1 2 3 4 5 6 7 8 9 10 |
# SuperFastPython.com # list all active child processes from multiprocessing import active_children # get a list of all active child processes children = active_children() # report a count of active children print(f'Active Children Count: {len(children)}') # report each in turn for child in children: print(child) |
Running the example reports the number of active child processes.
In this case, there are no active children processes
|
1 |
Active Children Count: 0 |
We can update the example so that we first start a number of children processes, have the children processes block for a moment, then get the list of active children.
This can be achieved by first defining a function to execute in a new process that blocks for a moment.
The task() function below implements this.
|
1 2 3 4 |
# function to execute in a new process def task(): # block for a moment sleep(1) |
Next, we can create a number of processes configured to run our task() function, then start them.
|
1 2 3 4 5 6 |
... # create a number of child processes processes = [Process(target=task) for _ in range(5)] # start the child processes for process in processes: process.start() |
We can then report the active child processes as before.
|
1 2 3 4 5 6 7 8 |
... # get a list of all active child processes children = active_children() # report a count of active children print(f'Active Children Count: {len(children)}') # report each in turn for child in children: print(child) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
# SuperFastPython.com # list all active child processes from time import sleep from multiprocessing import active_children from multiprocessing import Process # function to execute in a new process def task(): # block for a moment sleep(1) # entry point if __name__ == '__main__': # create a number of child processes processes = [Process(target=task) for _ in range(5)] # start the child processes for process in processes: process.start() # get a list of all active child processes children = active_children() # report a count of active children print(f'Active Children Count: {len(children)}') # report each in turn for child in children: print(child) |
Running the example first creates five processes to run our task() function, then starts them.
A list of all active child processes is then retrieved.
The count is reported, which is shown as five, as we expect, then the details of each process are then reported.
This highlights how we can access all active child processes from a parent process.
|
1 2 3 4 5 6 |
Active Children Count: 5 <Process name='Process-3' pid=16853 parent=16849 started> <Process name='Process-4' pid=16854 parent=16849 started> <Process name='Process-2' pid=16852 parent=16849 started> <Process name='Process-1' pid=16851 parent=16849 started> <Process name='Process-5' pid=16855 parent=16849 started> |
Free Python Multiprocessing Course
Download your FREE multiprocessing PDF cheat sheet and get BONUS access to my free 7-day crash course on the multiprocessing API.
Discover how to use the Python multiprocessing module including how to create and start child processes and how to use a mutex locks and semaphores.
How to Get The CPU Count
We may want to know the number of CPU cores available.
This can be determined via the multiprocessing.cpu_count() function.
The function returns an integer that indicates the number of logical CPU cores available in the system running the Python program.
Recall that a central processing unit or CPU executes program instructions. A CPU may have one or more physical CPU cores for executing code in parallel. Modern CPU cores may make use of hyperthreading, allowing each physical CPU core to operate like two (or more) logical CPU cores.
Therefore, a computer system with a CPU with four physical cores may report eight logical CPU cores via the multiprocessing.cpu_count() function function.
It may be useful to know the number of CPU cores available to configure a thread pool or the number of processes to create to execute tasks in parallel.
The example below demonstrates how to use the number of CPU cores.
|
1 2 3 4 5 6 7 |
# SuperFastPython.com # example of reporting the number of logical cpu cores from multiprocessing import cpu_count # get the number of cpu cores num_cores = cpu_count() # report details print(num_cores) |
Running the example reports the number of logical CPU cores available in the system.
In this case, my system provides eight logical CPU cores, your result may differ.
|
1 |
8 |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
How to Get The Current Process
We can get a multiprocessing.Process instance for the process running the current code.
This can be achieved via the multiprocessing.current_process() function that returns a multiprocessing.Process instance.
|
1 2 3 |
... # get the current process thread = current_process() |
We can use this function to access the multiprocessing.Process for the MainProcess.
This can be demonstrated with a short example, listed below.
|
1 2 3 4 5 6 7 |
# SuperFastPython.com # example of executing the current process from multiprocessing import current_process # get the current process process = current_process() # report details print(process) |
Running the example gets the process instance for the currently running process.
The details are then reported, showing that we have accessed the main process that has no parent process.
|
1 |
<_MainProcess name='MainProcess' parent=None started> |
We can use this function within a child process, allowing the child process to access a process instance for itself.
First, we can define a custom function that gets the current process and reports its details.
The task() function below implements this.
|
1 2 3 4 5 6 |
# function executed in a new process def task(): # get the current process process = current_process() # report details print(process) |
Next, we can create a new process to execute our custom task() function. Then start the process and wait for it to terminate.
|
1 2 3 4 5 6 7 |
... # create a new process process = Process(target=task) # start the new process process.start() # wait for the child process to terminate process.join() |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
# SuperFastPython.com # example of executing the current process within a child process from multiprocessing import Process from multiprocessing import current_process # function executed in a new process def task(): # get the current process process = current_process() # report details print(process) # entry point if __name__ == '__main__': # create a new process process = Process(target=task) # start the new process process.start() # wait for the child process to terminate process.join() |
Running the example first creates a new process instance, then starts the process and waits for it to finish.
The new process then gets an instance of its own multiprocessing.Process instance and then reports the details.
|
1 |
<Process name='Process-1' parent=16873 started> |
How to Get The Parent Process
We may need to access the parent process for a current child process.
This can be achieved via the multiprocessing.parent_process() function. This will return a multiprocessing.Process instance for the parent of the current process.
The MainProcess does not have a parent, therefore attempting to get the parent of the MainProcess will return None.
We can demonstrate this with a worked example.
|
1 2 3 4 5 6 7 |
# SuperFastPython.com # example of getting the parent process of the main process from multiprocessing import parent_process # get the the parent process process = parent_process() # report details print(process) |
Running the example attempts the multiprocessing.Process instance for the MainProcess.
The function returns None, as expected as the MainProcess does not have a parent process.
|
1 |
None |
We can make the example more interesting by creating a new child process, then getting the parent of the child process, which will be the MainProcess.
This can be achieved by first defining a function to get the parent process and then report its details.
The task() function below implements this.
|
1 2 3 4 5 6 |
# function to execute in a new process def task(): # get the the parent process process = parent_process() # report details print(process) |
Next, we can create a new process instance and configure it to execute our task() function. We can then start the new process and wait for it to terminate.
|
1 2 3 4 5 6 7 |
... # create a new process process = Process(target=task) # start the new process process.start() # wait for the new process to terminate process.join() |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
# SuperFastPython.com # example of getting the parent process of a child process from multiprocessing import parent_process from multiprocessing import Process # function to execute in a new process def task(): # get the the parent process process = parent_process() # report details print(process) # entry point if __name__ == '__main__': # create a new process process = Process(target=task) # start the new process process.start() # wait for the new process to terminate process.join() |
Running the example first creates a new process configured to execute our task function, then starts the process and waits for it to terminate.
The new child process then gets a multiprocessing.Process instance for the parent process, then reports its details.
We can see that the parent of the new process is the MainProcess, as we expected.
|
1 |
<_ParentProcess name='MainProcess' parent=None unknown> |
Further Reading
This section provides additional resources that you may find helpful.
Python Multiprocessing Books
- Python Multiprocessing Jump-Start, Jason Brownlee (my book!)
- Multiprocessing API Interview Questions
- Multiprocessing API Cheat Sheet
I would also recommend specific chapters in the books:
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- High Performance Python, Ian Ozsvald and Micha Gorelick, 2020.
- See: Chapter 9: The multiprocessing Module
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Multiprocessing: The Complete Guide
- Python Multiprocessing Pool: The Complete Guide
- Python ProcessPoolExecutor: The Complete Guide
APIs
References
Takeaways
You now know how to use process utility functions in Python.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Louis Morgner on Unsplash
Do you have any questions?