You can stop the main process and allow child processes to continue running.
This can be achieved by allowing the main process to stop executing custom code and automatically join the child processes, or to have the main process exit itself and release all resources.
In this tutorial, you will discover how to stop the main process and leave child processes running in Python.
Let’s get started.
Need to Stop Parent and Keep Child Process Running
We can run a task in a child process.
This can be achieved by creating an instance of the multiprocessing.Process class and specify the target function to execute via the “target” argument and any arguments to the target function via the “args” argument.
For example:
|
1 2 3 |
... # create and configure a child process process = multiprocessing.Process(target=task) |
You can learn more about running a function in a child process in the tutorial:
When running tasks in a child process, we may wish to stop and release the parent process.
This may be for many reasons, such as:
- The parent process is only needed to boot the task in the child process.
- The parent process is no longer required.
How can we stop the parent process and allow the child process to continue running?
Run loops using all CPUs, download your FREE book to learn how.
How to Stop Parent and Keep Child Process Running
We can stop the parent process and allow the child process to continue running.
There are 3 approaches that we might explore to achieve this, they are:
- Main process finishes running its task.
- Child terminates the main process.
- Main process explicitly exists.
One approach is to allow the main process to reach the end of its task.
In this case, the main process will no longer be executed, but will not stop. Instead, it will automatically join the child process and wait for it to complete.
You can learn more about this in the tutorial:
Another approach is to have the child process explicitly terminate or kill the parent process.
This can be achieved by getting the main process and calling the terminate() method or the kill() method. This will raise a signal in the process and may cause it to stop.
This will fail with an exception as the main process cannot be stopped in this way.
You can learn more about terminating and killing processes in the tutorial:
A more aggressive approach is to have the main process explicitly exit.
The main process can call the sys.exit() method to exit itself. This will have the same effect as the main process finishing its task. It will join the child process until it is complete.
Instead, the main process can call the os._exit() function which will force the process to exit immediately.
You can learn more about exiting the main process in the tutorial:
This will leave the child process running as an orphaned process.
You can learn more about orphaned processes in the tutorial:
Now that we know how to stop the parent process and leave the child process running, let’s look at some worked examples.
Example Parent Stop and Child Keeps Running
We can explore allowing the main process to stop while the child process continues running.
In this example, we can define a task to execute in a child process that runs for some time. The main process will start this task asynchronously, then stop. The child process will continue running until it is done. Although the main process is no longer executing code, it remains alive and waiting for the child process to terminate.
Firstly, we can define a task to execute in the child process.
The task will loop three times, each iteration it will report a message and sleep for one second. After the loop, the task will get the parent process and report whether it is alive or not.
This can be achieved by calling the parent_process() function to get the multiprocessing.Process instance, then call the is_alive() method.
The task() function below implements this.
|
1 2 3 4 5 6 7 8 9 10 11 |
# task executed in a child process def task(): for i in range(3): # report a message print('Task is running...', flush=True) # block for a moment sleep(1) # get the parent process parent = parent_process() # check if the main process is still running print(f'Parent alive: {parent.is_alive()}') |
You can learn more about getting the parent process in the tutorial:
You can learn more about querying the status of a process in the tutorial:
The main process will create and configure a new process to execute the task() function, start the process, report a message then stop.
|
1 2 3 4 5 6 7 8 |
# protect the entry point if __name__ == '__main__': # create and configure a child process process = Process(target=task) # start the child process process.start() # report a final message print('Main is done') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
# SuperFastPython.com # example of stopping the parent process and child remains running from time import sleep from multiprocessing import Process from multiprocessing import parent_process # task executed in a child process def task(): for i in range(3): # report a message print('Task is running...', flush=True) # block for a moment sleep(1) # get the parent process parent = parent_process() # check if the main process is still running print(f'Parent alive: {parent.is_alive()}') # protect the entry point if __name__ == '__main__': # create and configure a child process process = Process(target=task) # start the child process process.start() # report a final message print('Main is done') |
Running the example first creates the child process to execute our custom task function, then starts the process.
The main process then reports a message and stops executing our custom code.
The child process runs its loop, reporting a message and sleeping for one second each iteration.
After the loop, the child process gets the main process and reports whether it is alive.
In this case, we can see that although the parent process is no longer executing our custom code, it remains alive, waiting for the child process to terminate.
This highlights how we can allow the main process to stop running and for the child process to continue running, without explicitly terminating the main process.
|
1 2 3 4 5 |
Main is done Task is running... Task is running... Task is running... Parent alive: True |
Next, let’s look at an example of explicitly terminating the main process from the child process.
Free Python Multiprocessing Course
Download your FREE multiprocessing PDF cheat sheet and get BONUS access to my free 7-day crash course on the multiprocessing API.
Discover how to use the Python multiprocessing module including how to create and start child processes and how to use a mutex locks and semaphores.
Example of Child Cannot Terminate Parent Process
We can explore getting the main process from the child process and explicitly terminating it.
In this example, we can update the previous example to call the terminate() method on the parent process from the child process.
We may want to terminate the main process in order to release any resources it may be holding.
The updated task() function with this change is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
# task executed in a child process def task(): for i in range(3): # report a message print('Task is running...', flush=True) # block for a moment sleep(1) # get the parent process parent = parent_process() # terminate the parent process parent.terminate() # wait for the parent to stop parent.join() # check if the main process is still running print(f'Parent alive: {parent.is_alive()}') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
# SuperFastPython.com # example of terminating the parent process and child remains running from time import sleep from multiprocessing import Process from multiprocessing import parent_process # task executed in a child process def task(): for i in range(3): # report a message print('Task is running...', flush=True) # block for a moment sleep(1) # get the parent process parent = parent_process() # terminate the parent process parent.terminate() # wait for the parent to stop parent.join() # check if the main process is still running print(f'Parent alive: {parent.is_alive()}') # protect the entry point if __name__ == '__main__': # create and configure a child process process = Process(target=task) # start the child process process.start() # report a final message print('Main is done') |
Running the example first creates the child process to execute our custom task function, then starts the process.
The main process then reports a message and stops executing our custom code.
The child process runs its loop, reporting a message and sleeping for one second each iteration.
After the loop, the child process gets the main process. It then calls the terminate() method on the main process.
This fails with an AttributeError exception.
The reason is that the main process is a special process that has overridden some methods. It does not allow the terminate() or kill() methods to be called on it.
This means that a child process cannot explicitly terminate or kill the main process.
|
1 2 3 4 5 6 7 8 |
Main is done Task is running... Task is running... Task is running... Process Process-1: Traceback (most recent call last): ... AttributeError: 'NoneType' object has no attribute 'terminate' |
Next, let’s look at an example of having the main process explicitly exit.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Parent Forcefully Exiting While Child Still Running
We can explore the case of the parent explicitly exiting while the child process is running.
In this example, we will update the main process to call the os._exit() function in order to terminate itself without automatically joining the child process and waiting for it to complete.
|
1 2 3 |
... # terminate self os._exit(os.EX_OK) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
# SuperFastPython.com # example of parent terminating itself and child remains running from time import sleep from multiprocessing import Process from multiprocessing import parent_process import os # task executed in a child process def task(): for i in range(3): # report a message print('Task is running...', flush=True) # block for a moment sleep(1) # get the parent process parent = parent_process() # check if the main process is still running print(f'Parent alive: {parent.is_alive()}') # protect the entry point if __name__ == '__main__': # create and configure a child process process = Process(target=task) # start the child process process.start() # report a final message print('Main is done') # terminate self os._exit(os.EX_OK) |
Running the example first creates the child process to execute our custom task function, then starts the process.
The main process then reports a message and calls the exit() function. This terminates the main process immediately.
The child process runs its loop, reporting a message and sleeping for one second each iteration.
After the loop, the child process gets the main process and reports whether it is alive.
In this case, we can see that the main process has been terminated and is no longer alive.
This highlights how we can leave a child process running and have the main process terminate and release all resources.
|
1 2 3 4 5 |
Main is done Task is running... Task is running... Task is running... Parent alive: False |
Further Reading
This section provides additional resources that you may find helpful.
Python Multiprocessing Books
- Python Multiprocessing Jump-Start, Jason Brownlee (my book!)
- Multiprocessing API Interview Questions
- Multiprocessing API Cheat Sheet
I would also recommend specific chapters in the books:
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- High Performance Python, Ian Ozsvald and Micha Gorelick, 2020.
- See: Chapter 9: The multiprocessing Module
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Multiprocessing: The Complete Guide
- Python Multiprocessing Pool: The Complete Guide
- Python ProcessPoolExecutor: The Complete Guide
APIs
References
Takeaways
You now know how to stop the main process and leave child processes running in Python.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Doug Watanabe on Unsplash
Do you have any questions?