Last Updated on September 12, 2022
You can safely stop a process by using a multiprocessing.Event.
In this tutorial you will discover how to gracefully stop a process in Python.
Let’s get started.
Need to Stop a Process
A process is a running instance of a computer program.
Every Python program is executed in a Process, which is a new instance of the Python interpreter. This process has the name MainProcess and has one thread used to execute the program instructions called the MainThread. Both processes and threads are created and managed by the underlying operating system.
Sometimes we may need to create new child processes in our program in order to execute code concurrently.
Python provides the ability to create and manage new processes via the multiprocessing.Process class.
You can learn more about multiprocessing in the tutorial:
In concurrent programming, we may run a task in a new process, then later decide to stop the task.
This may be for many reasons, such as:
- The result from the task is no longer required.
- The application is shutting down.
- The outcome from the task has gone astray.
Python provides no way to stop a running process in the API.
How can we stop a process in Python?
Run loops using all CPUs, download your FREE book to learn how.
How to Stop a Process
A process can be stopped using a shared boolean variable such as a multiprocessing.Event.
A multiprocessing.Event is a thread-safe boolean variable flag that can be either set or not set. It can be shared between processes safely and correctly and checked and set without fear of a race condition.
You can learn more about multiprocessing events in the tutorial:
A new multiprocessing.Event can be created and then shared between processes, for example:
|
1 2 3 |
... # create a shared event event = multiprocessing.Event() |
The event is created in the ‘not set’ or False state.
We may have a task in a custom function that is run in a new process. The task may iterate, such as in a while-loop or a for-loop.
For example:
|
1 2 3 4 5 |
# custom task function def task(): # perform task in iterations while True: # ... |
We can update our task function to check the status of a multiprocessing.Event each iteration.
If the event is set True, we can exit the task loop or return from the task() function, allowing the new child process to terminate.
The status of the multiprocessing.Event can be checked via the is_set() function.
For example:
|
1 2 3 4 5 6 7 8 |
# custom task function def task(): # perform task in iterations while True: # ... # check for stop if event.is_set(): break |
The main parent process, or another process, can then set the event in order to stop the new process from running.
The multiprocessing.Event can be set or made True via the set() function.
For example:
|
1 2 3 4 5 |
... # set the event event.set() # wait for the new process to stop process.join() |
Note, this same approach can be used to stop a thread within a process. You can learn more in the tutorial:
Now that we know how to stop a Python process, let’s look at some worked examples.
Stop a Process With a Custom Function
In this section we can explore how to safely stop a child process that is executing a custom function.
First, we will develop an example that runs a custom function in a child process, then we will look at how to update the example to stop the process on demand.
Custom Function in a New Process
We can develop an example that runs a function in a new child process.
The new task function will execute in a loop and once finished the new process will terminate and the main parent process will terminate.
Firstly, we can define the task function.
The function will execute a loop five times. Each iteration it will block for a second, then report a message in an effort to simulate work and show progress.
Once the task is finished, the function will report a final message.
The task() function below implements this.
|
1 2 3 4 5 6 7 8 9 |
# custom task function def task(): # execute a task in a loop for i in range(5): # block for a moment sleep(1) # report a message print('Worker process running...', flush=True) print('Worker closing down', flush=True) |
Next, in the main parent process we can create a new multiprocessing.Process instance that is configured to execute our task() function in a new process via the “target” argument.
|
1 2 3 |
... # create and configure a new process process = Process(target=task) |
We can then immediately start the new process via the start() function.
|
1 2 3 |
... # start the new process process.start() |
The parent process will then wait for the new child process to finish by joining the process.
If you are new to joining processes, you can learn more here:
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
# SuperFastPython.com # example of a process executing a custom function from time import sleep from multiprocessing import Process # custom task function def task(): # execute a task in a loop for i in range(5): # block for a moment sleep(1) # report a message print('Worker process running...', flush=True) print('Worker closing down', flush=True) # entry point if __name__ == '__main__': # create and configure a new process process = Process(target=task) # start the new process process.start() # wait for the new process to finish process.join() |
Running the example first creates and starts the new child process.
The main parent process then blocks until the new process finishes.
The new process executes in a loop, blocking for a moment and reporting progress. After five iterations of the loop, the new process reports a final message and terminates.
The parent process notices that the new process has finished and then terminates itself, closing the program.
|
1 2 3 4 5 6 |
Worker process running... Worker process running... Worker process running... Worker process running... Worker process running... Worker closing down |
Next, let’s look at how we might update this example to stop the process on demand.
Stop Custom Function in New Process
We can update the example from the previous section to stop the new process on demand.
This can be achieved by creating a new multiprocessing.Event instance, passing it as an argument to the new process, and then updating the custom function to check if the event is set each iteration. Once the event is set, the function can break its task loop and exit, terminating the new process.
Firstly, we can update the task function to take the shared multiprocessing.Event instance as an argument.
|
1 2 3 |
# custom task function def task(event): # ... |
We can then update the task loop to check the status of the event each iteration, and if set to break the task loop.
|
1 2 3 4 |
... # check for stop if event.is_set(): break |
Tying this together, the updated task() function with these changes is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
# custom task function def task(event): # execute a task in a loop for i in range(5): # block for a moment sleep(1) # check for stop if event.is_set(): break # report a message print('Worker process running...', flush=True) print('Worker closing down', flush=True) |
Next, the main process can first create a new multiprocessing.Event instance to be shared between the processes.
|
1 2 3 |
... # create the event event = Event() |
We can then update the parent process to block for a few seconds.
|
1 2 3 |
... # block for a while sleep(3) |
Finally, we can set the event and trigger the new process to stop.
|
1 2 3 4 |
... # stop the worker process print('Main stopping process') event.set() |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
# SuperFastPython.com # example of stopping a new process from time import sleep from multiprocessing import Process from multiprocessing import Event # custom task function def task(event): # execute a task in a loop for i in range(5): # block for a moment sleep(1) # check for stop if event.is_set(): break # report a message print('Worker process running...', flush=True) print('Worker closing down', flush=True) # entry point if __name__ == '__main__': # create the event event = Event() # create and configure a new process process = Process(target=task, args=(event,)) # start the new process process.start() # block for a while sleep(3) # stop the worker process print('Main stopping process') event.set() # wait for the new process to finish process.join() |
Running the example first creates and starts the new child process.
The main parent process then blocks for a few seconds.
Meanwhile, the new process executes its task loop, blocking and reporting a message each iteration. It checks the event each iteration, which remains false and does not trigger the process to stop.
The main parent process wakes up, and then sets the event. It then joins the new process, waiting for it to terminate.
The task process checks the event which is discovered to be set (e.g. True). The process breaks the task loop, reports a final message then terminates the new process.
The parent process then terminates, closing the Python program.
|
1 2 3 4 |
Worker process running... Worker process running... Main stopping process Worker closing down |
Now that we know how to stop a custom function running in a new process, let’s look at an example of stopping a custom process class.
Free Python Multiprocessing Course
Download your FREE multiprocessing PDF cheat sheet and get BONUS access to my free 7-day crash course on the multiprocessing API.
Discover how to use the Python multiprocessing module including how to create and start child processes and how to use a mutex locks and semaphores.
Stop a Process With an Extended Class
In this section we can explore how to stop a process that is an object that extends the multiprocessing.Process class.
First, we will develop an example that extends the multiprocessing.Process class, then we will look at how to update the example to stop the process on demand.
Custom Process Class
We can run a task in a new process by extending the multiprocessing.Process class and overriding the constructor and the run() function.
In this section, we will show how to run the same task as in the previous section within a custom process class.
Firstly, we can define a new class that extends multiprocessing.Process.
|
1 2 3 |
# custom process class class CustomProcess(Process): # ... |
The new class may implement a constructor, and if so it must call the constructor of the multiprocessing.Process class to properly initialize the underlying process instance.
|
1 2 3 4 |
# constructor def __init__(self): # call the parent constructor Process.__init__(self) |
We can then override the run() method on the multiprocessing.Process class.
This function will run in a new process once the process is started via the start() function. Specifically we call start() and start() will call run() for us in a new spawned or forked child process.
Our task will be to execute a loop five times, block for a second to simulate work, and report a message. Then report a message once the task is complete.
|
1 2 3 4 5 6 7 8 9 |
# execute task def run(self): # execute a task in a loop for i in range(5): # block for a moment sleep(1) # report a message print('Worker process running...', flush=True) print('Worker closing down', flush=True) |
Tying this together the new CustomProcess class is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
# custom process class class CustomProcess(Process): # constructor def __init__(self): # call the parent constructor Process.__init__(self) # execute task def run(self): # execute a task in a loop for i in range(5): # block for a moment sleep(1) # report a message print('Worker process running...', flush=True) print('Worker closing down', flush=True) |
Next, in the main parent process, we can create a new instance of our custom process class and start it in order to execute our task in a new process.
|
1 2 3 4 5 |
... # create a new process process = CustomProcess() # start the new process process.start() |
We will then join() the new process and wait for the new process to terminate, before terminating the main parent process.
|
1 2 3 |
... # wait for the new process to finish process.join() |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
# SuperFastPython.com # example of a extending the process class from time import sleep from multiprocessing import Process # custom process class class CustomProcess(Process): # constructor def __init__(self): # call the parent constructor Process.__init__(self) # execute task def run(self): # execute a task in a loop for i in range(5): # block for a moment sleep(1) # report a message print('Worker process running...', flush=True) print('Worker closing down', flush=True) # entry point if __name__ == '__main__': # create a new process process = CustomProcess() # start the new process process.start() # wait for the new process to finish process.join() |
Running the example first creates our custom process class then starts executing the new process.
The main process then blocks waiting for the new process to finish.
Our custom process class runs our task in a new process of execution, looping five times. Each iteration it blocks for a moment then reports a message.
Once the task is finished, it reports a final message.
The new process terminates, then the main process terminates, closing the process.
|
1 2 3 4 5 6 |
Worker process running... Worker process running... Worker process running... Worker process running... Worker process running... Worker closing down |
Next, let’s look at how we might update this example to stop the task on demand.
Stop Custom Process Class
We can update the example in the previous section to stop the process on demand.
This can be achieved by sharing a multiprocessing.Event with the new process and updating the run() function to check if the event is set each iteration. Once set, the run() function can exit, which will terminate the new process.
Firstly, we can update the constructor of our new process class to create a new instance of the multiprocessing.Event class and store it as an instance variable attribute.
|
1 2 3 4 5 6 |
# constructor def __init__(self): # call the parent constructor Process.__init__(self) # create and store an event self.event = Event() |
Next, we can update the run() function to check the status of the multiprocessing.Event each iteration and to break the task loop once the event is set.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
# execute task def run(self): # execute a task in a loop for i in range(5): # block for a moment sleep(1) # check for stop if self.event.is_set(): break # report a message print('Worker process running...', flush=True) print('Worker closing down', flush=True) |
Tying this together, the updated CustomProcess class that can be stopped on demand is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
# custom process class class CustomProcess(Process): # constructor def __init__(self): # call the parent constructor Process.__init__(self) # create and store an event self.event = Event() # execute task def run(self): # execute a task in a loop for i in range(5): # block for a moment sleep(1) # check for stop if self.event.is_set(): break # report a message print('Worker process running...', flush=True) print('Worker closing down', flush=True) |
The main parent process can then set the event via the attribute on the new CustomProcess instance.
|
1 2 3 4 |
... # stop the worker process print('Main stopping process') process.event.set() |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
# SuperFastPython.com # example of stopping a custom process class from time import sleep from multiprocessing import Process from multiprocessing import Event # custom process class class CustomProcess(Process): # constructor def __init__(self): # call the parent constructor Process.__init__(self) # create and store an event self.event = Event() # execute task def run(self): # execute a task in a loop for i in range(5): # block for a moment sleep(1) # check for stop if self.event.is_set(): break # report a message print('Worker process running...', flush=True) print('Worker closing down', flush=True) # entry point if __name__ == '__main__': # create a new process process = CustomProcess() # start the new process process.start() # block for a while sleep(3) # stop the worker process print('Main stopping process') process.event.set() # wait for the new process to finish process.join() |
Running the example first involves creating an instance of the custom process and starting it.
The main process will then block for a few seconds.
The new process will start executing its task loop, blocking for a moment and reporting progress.
The main process then wakes up and sets the event, triggering the new process to stop.
The new process notices that the event is set and then exits the task loop, reporting a final message and terminating the new process.
The main process waits for the new process to terminate, before then stopping itself and closing the application.
|
1 2 3 4 |
Worker process running... Worker process running... Main stopping process Worker closing down |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Common Questions About Stopping Processes
This section lists common questions about stopping a child process in Python.
Do you have any questions about stopping a process? Let me know in the comments and I will do my best to help.
What if My Task Does Not Have a Loop?
Your task does not need to have a loop in order to be stopped.
Instead, you need to add frequent checks of the shared multiprocessing.Event and return from the task() function or run() function once the event is set.
This will depend on the specifics of the task you wish to exit in a new process.
If your task involves blocking, such as reading or writing IO or waiting on another process, you may want to adopt a busy wait and have the multiprocessing.Event checked each iteration.
If your task remains idle for some time, such as waiting for sporadic events, it may be able to wait on the multiprocessing.Event itself via the wait() function.
What if My Task Raises An Exception?
If your task executing in a new process raises an exception that is not handled, it will terminate the process.
This could be an alternate way for you to trigger a new process to terminate, if convenient.
What if My New Process Has Already Stopped?
If the new process has already stopped and you set the event, it will not have any effect.
It would not be a problem.
What if My New Process is a Daemon Process?
If your task is executing in a daemon process, then there is no need to stop it manually as it will be terminated for you once all non-daemon processes (e.g. the main parent process) terminate.
You can learn more about daemon process here:
How Can We Stop Multiple Processes?
You can share the same multiprocessing.Event between all of the child processes that might want to stop.
Each process can frequently check the status of the event.
Once set, all processes can exit their task and terminate.
How Can We Stop Tasks in a ProcessPoolExecutor?
The same technique of using a multiprocessing.Event can be used to stop tasks executing in a ProcessPoolExecutor.
You can see an example of this here:
Do We Need To Use a Manager
No.
A shared Event object can be created via a multiprocessing.Manager.
For example:
|
1 2 3 4 5 6 |
... # create a manager with Manager() as manager: # create an event from the manager event = manager.Event() # ... |
This is only needed if the event is to be shared across processes running on different machines.
Why Not use Process.terminate() or Process.kill()
A child process can be stopped by calling the terminate() method, or the kill() method.
If called, the process will stop immediately.
This is not a graceful and safe way to stop a child process as it does not give the child process any way to respond to the request to stop, such as close and release resources.
Further Reading
This section provides additional resources that you may find helpful.
Python Multiprocessing Books
- Python Multiprocessing Jump-Start, Jason Brownlee (my book!)
- Multiprocessing API Interview Questions
- Multiprocessing API Cheat Sheet
I would also recommend specific chapters in the books:
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- High Performance Python, Ian Ozsvald and Micha Gorelick, 2020.
- See: Chapter 9: The multiprocessing Module
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Multiprocessing: The Complete Guide
- Python Multiprocessing Pool: The Complete Guide
- Python ProcessPoolExecutor: The Complete Guide
APIs
References
Takeaways
You now know how to stop a process in Python.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Aleks Marinkovic on Unsplash
What if we have many Processes and Event attached to it and want to stop the Processes separately i.e. not in a single time when needed?
No problem, each process can have its own Event object used to stop it.