Last Updated on September 12, 2022
You can kill all child processes by first getting a list of all active child processes via the multiprocessing.active_children() function then calling either terminate() or kill() on each process instance.
In this tutorial you will discover how to kill all active child processes in Python.
Let’s get started.
Need to Kill All Child Processes
A process is a running instance of a computer program.
Every Python program is executed in a Process, which is a new instance of the Python interpreter. This process has the name MainProcess and has one thread used to execute the program instructions called the MainThread. Both processes and threads are created and managed by the underlying operating system.
Sometimes we may need to create new child processes in our program in order to execute code concurrently.
Python provides the ability to create and manage new processes via the multiprocessing.Process class.
You can learn more about multiprocessing in the tutorial:
In concurrent programming, we sometimes need to kill all child processes.
This may be for many reasons, such as:
- A fault in one process requires that all processes be stopped.
- The user requests that the application close immediately.
- A result is obtained, negating all remaining tasks.
How can we kill all child processes in Python?
Run loops using all CPUs, download your FREE book to learn how.
How to Stop All Child Processes
We can terminate or kill all active child processes for the current process.
This involves first getting a list of all active child processes via the multiprocessing.active_children() function.
For example:
|
1 2 3 |
... # get all active child processes active = active_children() |
This will return a list of multiprocessing.Process instances, one for each currently running child process.
You can learn more about getting all child processes in the tutorial:
We can iterate over the list of active child processes and request that each stop in turn.
This can be achieved by calling the terminate() function on each active child process. This will raise a SIGTERM signal in the child process.
For example:
|
1 2 3 4 |
... # terminate all active children for child in active: child.terminate() |
Alternatively, we can call the kill() function on each child process. This will raise a SIGKILL signal in the child process.
For example:
|
1 2 3 4 |
... # kill all active children for child in active: child.kill() |
You can learn more about terminating and killing processes in the tutorial:
Now that we know how to stop all active child processes, let’s look at some worked examples.
Example of Terminating All Child Processes
We can explore how to terminate all active child processes.
In this example we will start ten child processes, each configured to block for a number of seconds. The main process will then terminate all active child processes.
Firstly, we can define a function to run in each child process.
The function will block by sleeping for ten seconds.
The task() function below implements this.
|
1 2 3 4 |
# function executed in a child process def task(): # block for a while sleep(10) |
Next, we can configure ten child processes to execute our custom task() function. This can be achieved in a list comprehension.
|
1 2 3 |
... # start many child processes children = [Process(target=task) for _ in range(10)] |
We can then start all child processes and then block the main process for two seconds.
|
1 2 3 4 5 6 7 |
... # start all child processes for child in children: child.start() # wait a moment print('Main waiting...') sleep(2) |
The main process will then stop all child processes.
First, we will get a list of all active child processes.
|
1 2 3 4 |
... # get all active child processes active = active_children() print(f'Active Children: {len(active)}') |
We can then iterate over the list of active child processes and terminate each in turn.
|
1 2 3 4 |
... # terminate all active children for child in active: child.terminate() |
The main process can then wait for the child processes to shut down completely.
|
1 2 3 4 |
... # block until all children have closed for child in active: child.join() |
Finally, we can confirm that there are no longer any child processes running.
|
1 2 3 4 |
... # report active children active = active_children() print(f'Active Children: {len(active)}') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
# SuperFastPython.com # example of stopping all child processes with terminate from time import sleep from multiprocessing import Process from multiprocessing import active_children # function executed in a child process def task(): # block for a while sleep(10) # protect the entry point if __name__ == '__main__': # start many child processes children = [Process(target=task) for _ in range(10)] # start all child processes for child in children: child.start() # wait a moment print('Main waiting...') sleep(2) # get all active child processes active = active_children() print(f'Active Children: {len(active)}') # terminate all active children for child in active: child.terminate() # block until all children have closed for child in active: child.join() # report active children active = active_children() print(f'Active Children: {len(active)}') |
Running the example first creates and starts ten child processes.
The main process then blocks for two seconds.
Each child process is started and blocked for ten seconds.
The main process then wakes up and gets a list of active child processes. It reports the number of active child processes that matches the number of child processes we expect.
The child processes are then terminated, raising a SIGTERM signal in each child process.
The main process blocks for a fraction of a second until all child processes have terminated successfully, then reports the total number of active child processes, which is zero as we now expect.
|
1 2 3 |
Main waiting... Active Children: 10 Active Children: 0 |
Next, let’s look at how we might kill all child processes.
Free Python Multiprocessing Course
Download your FREE multiprocessing PDF cheat sheet and get BONUS access to my free 7-day crash course on the multiprocessing API.
Discover how to use the Python multiprocessing module including how to create and start child processes and how to use a mutex locks and semaphores.
Example of Killing All Child Processes
We can explore how to kill all active child processes.
In this example, we can update the example from the previous section to kill the child processes instead of terminating them.
Recall that the main difference between killing and terminating a child process is the signal sent to the process, e.g. SIGKILL vs SIGTERM. A SIGTERM signal can be handled by the child process, e.g. to close resources, whereas a SIGKILL process cannot.
This change involves calling the kill() function on each active child process.
For example:
|
1 2 3 4 |
... # kill all active children for child in active: child.kill() |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
# SuperFastPython.com # example of stopping all child processes with kill from time import sleep from multiprocessing import Process from multiprocessing import active_children # function executed in a child process def task(): # block for a while sleep(10) # protect the entry point if __name__ == '__main__': # start many child processes children = [Process(target=task) for _ in range(10)] # start all child processes for child in children: child.start() # wait a moment print('Main waiting...') sleep(2) # get all active child processes active = active_children() print(f'Active Children: {len(active)}') # kill all active children for child in active: child.kill() # block until all children have closed for child in active: child.join() # report active children active = active_children() print(f'Active Children: {len(active)}') |
Running the example first creates and starts ten child processes.
The main process then blocks for two seconds.
Each child process is started and blocked for ten seconds.
The main process then wakes up and gets a list of active child processes. It reports the number of active child processes that matches the number of child processes we expect.
The child processes are then killed, raising a SIGKILL signal in each child process.
The main process blocks for a fraction of a second until all child processes have been killed successfully, then reports the total number of active child processes, which is zero as we now expect.
|
1 2 3 |
Main waiting... Active Children: 10 Active Children: 0 |
Next, let’s look at how to stop all child processes that themselves have child processes.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Terminating All Child Processes That Have Child Processes
We can explore how to terminate all active child processes that themselves have child processes running.
In this example we will first show how terminating child processes that have child processes does not work as expected, and then we will show how we can stop all child and grandchild processes immediately.
Failure To Terminate Grandchild Processes
If we try to kill or terminate child processes that themselves have child processes, the child processes will not stop immediately.
This is because a process will not completely terminate until all of its child processes have terminated.
We can demonstrate this with a worked example.
In this example, the main process will start ten child processes as before. Each child process itself will create one child process. When the main process terminates the child processes, the grandchild processes continue running. The main process will then explicitly wait on the child processes to shutdown. The child process will implicitly or automatically wait for their own child processes to terminate before terminating completely themselves.
The effect is that the program does not terminate when expected, and instead waits for all grandchild processes to finish first.
First, we can define the function to be executed by grandchild processes.
This function will block for ten seconds.
The task2() function below implements this.
|
1 2 3 |
# function executed in a child process def task2(): sleep(10) |
Next, we can define the function to be executed by the child processes.
This function will create a child process to execute our task2() function above, then block.
The task() function below implements this.
|
1 2 3 4 5 6 |
# function executed in a child process def task(): # start another child process Process(target=task2).start() # block for a while sleep(10) |
Finally, the main process can configure and start ten child processes to run the task() function, then block for a moment.
|
1 2 3 4 5 6 7 8 9 |
... # start many child processes children = [Process(target=task) for _ in range(10)] # start all child processes for child in children: child.start() # wait a moment print('Main waiting...') sleep(2) |
The main process will then report the number of child processes, terminate them, then wait for all child processes to terminate.
|
1 2 3 4 5 6 7 8 9 10 |
... # get all active child processes active = active_children() print(f'Active Children: {len(active)}') # terminate all active children for child in active: child.terminate() # block until all children have closed for child in active: child.join() |
The main process will then report the total number of active child processes.
|
1 2 3 4 |
... # report active children active = active_children() print(f'Active Children: {len(active)}') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
# SuperFastPython.com # example of stopping all child processes that have child processes from time import sleep from multiprocessing import Process from multiprocessing import active_children # function executed in a child process def task2(): sleep(10) # function executed in a child process def task(): # start another child process Process(target=task2).start() # block for a while sleep(10) # protect the entry point if __name__ == '__main__': # start many child processes children = [Process(target=task) for _ in range(10)] # start all child processes for child in children: child.start() # wait a moment print('Main waiting...') sleep(2) # get all active child processes active = active_children() print(f'Active Children: {len(active)}') # terminate all active children for child in active: child.terminate() # block until all children have closed for child in active: child.join() # report active children active = active_children() print(f'Active Children: {len(active)}') |
Running the example first starts ten child processes.
Each child process then starts one grandchild process and then blocks for ten seconds. Each grandchild process itself blocks for ten seconds.
The main process then wakes up and gets a list of all child processes. It reports ten active child processes, which matches the number of child processes that we started.
The main process then terminates the child processes then blocks until the child processes are completely finished.
The child processes terminate, then block waiting for their own child processes, the grandchild processes, to finish.
The grandchild processes run as per normal and finish after ten seconds.
The grandchild processes terminate, the child processes then unblock, and finally the main process unblocks, and reports no remaining active child processes.
This highlights that even though we explicitly terminated the child processes, the child processes did not terminate as each child had a grandchild process running.
|
1 2 3 |
Main waiting... Active Children: 10 Active Children: 0 |
Next, let’s explore how we can terminate all child and grandchild processes on demand.
Fix To Terminate Grandchild Processes
We can explore how to terminate all child and grandchild processes.
This can be achieved by the child processes themselves responding to the request to terminate by in turn terminating their child processes.
This can be achieved by each child process registering a function to respond to the SIGTERM function. Within the registered function, the child process can get a list of active child processes and request that each is terminated, then terminate itself.
This way, the child process will first terminate its own child processes then terminate itself, allowing all children and grandchildren processes of the main process to be killed on demand. This approach can be extended to any level of descendant processes.
Firstly, we can define a function to handle the SIGTERM function.
The function will get a list of active child processes, terminate each, then terminate itself with a call to sys.exit().
You can learn more about calling sys.exit() to terminate a process from within a process in the tutorial:
The handler() function below implements this.
|
1 2 3 4 5 6 7 8 9 |
# handle signal def handler(sig, frame): # get all active child processes active = active_children() # terminate all active children for child in active: child.terminate() # terminate the process sys.exit(0) |
We can then register the handler function via the signal.signal() function.
This can be done by the child process while executing the task() function.
|
1 2 3 |
... # handle sigterm signal(SIGTERM, handler) |
The updated task() function with this change is listed below.
|
1 2 3 4 5 6 7 8 9 |
... # function executed in a child process def task(): # handle sigterm signal(SIGTERM, handler) # start another child process Process(target=task2).start() # block for a while sleep(10) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
# SuperFastPython.com # example of stopping all child processes that have child processes from time import sleep from multiprocessing import Process from multiprocessing import active_children from signal import signal from signal import SIGTERM import sys # function executed in a child process def task2(): sleep(10) # handle signal def handler(sig, frame): # get all active child processes active = active_children() # terminate all active children for child in active: child.terminate() # terminate the process sys.exit(0) # function executed in a child process def task(): # handle sigterm signal(SIGTERM, handler) # start another child process Process(target=task2).start() # block for a while sleep(10) # protect the entry point if __name__ == '__main__': # start many child processes children = [Process(target=task) for _ in range(10)] # start all child processes for child in children: child.start() # wait a moment print('Main waiting...') sleep(2) # get all active child processes active = active_children() print(f'Active Children: {len(active)}') # terminate all active children for child in active: child.terminate() # block until all children have closed for child in active: child.join() # report active children active = active_children() print(f'Active Children: {len(active)}') |
Running the example first starts ten child processes.
Each child process then starts one grandchild process and then blocks for ten seconds. Each grandchild process itself blocks for ten seconds.
The main process then wakes up and gets a list of all child processes. It reports ten active child processes, which matches the number of child processes that we started.
The main process then terminates the child processes then blocks until the child processes are completely finished.
The child processes respond to the SIGTERM signal with the custom function handler. Each child process then terminates their own child processes then terminates.
The grandchild processes are terminated immediately. The child process then terminated immediately.
The main process blocks only for a fraction of a second as all grandchild and child processes shutdown, then reports no remaining active child processes running.
|
1 2 3 |
Main waiting... Active Children: 10 Active Children: 0 |
Further Reading
This section provides additional resources that you may find helpful.
Python Multiprocessing Books
- Python Multiprocessing Jump-Start, Jason Brownlee (my book!)
- Multiprocessing API Interview Questions
- Multiprocessing API Cheat Sheet
I would also recommend specific chapters in the books:
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- High Performance Python, Ian Ozsvald and Micha Gorelick, 2020.
- See: Chapter 9: The multiprocessing Module
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Multiprocessing: The Complete Guide
- Python Multiprocessing Pool: The Complete Guide
- Python ProcessPoolExecutor: The Complete Guide
APIs
References
Takeaways
You now know how to stop all active child processes in Python.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Egle Sidaraviciute on Unsplash
Do you have any questions?