How to Safely Stop a Process in Python

May 21, 2022 Python Multiprocessing

You can safely stop a process by using a multiprocessing.Event.

In this tutorial you will discover how to gracefully stop a process in Python.

Let's get started.

Need to Stop a Process

A process is a running instance of a computer program.

Every Python program is executed in a Process, which is a new instance of the Python interpreter. This process has the name MainProcess and has one thread used to execute the program instructions called the MainThread. Both processes and threads are created and managed by the underlying operating system.

Sometimes we may need to create new child processes in our program in order to execute code concurrently.

Python provides the ability to create and manage new processes via the multiprocessing.Process class.

You can learn more about multiprocessing in the tutorial:

Multiprocessing in Python: The Complete Guide

In concurrent programming, we may run a task in a new process, then later decide to stop the task.

This may be for many reasons, such as:

The result from the task is no longer required.
The application is shutting down.
The outcome from the task has gone astray.

Python provides no way to stop a running process in the API.

How can we stop a process in Python?

How to Stop a Process

A process can be stopped using a shared boolean variable such as a multiprocessing.Event.

A multiprocessing.Event is a thread-safe boolean variable flag that can be either set or not set. It can be shared between processes safely and correctly and checked and set without fear of a race condition.

You can learn more about multiprocessing events in the tutorial:

Multiprocessing Event Object In Python

A new multiprocessing.Event can be created and then shared between processes, for example:

...
# create a shared event
event = multiprocessing.Event()

The event is created in the 'not set' or False state.

We may have a task in a custom function that is run in a new process. The task may iterate, such as in a while-loop or a for-loop.

For example:

# custom task function
def task():
	# perform task in iterations
	while True:
		# ...

We can update our task function to check the status of a multiprocessing.Event each iteration.

If the event is set True, we can exit the task loop or return from the task() function, allowing the new child process to terminate.

The status of the multiprocessing.Event can be checked via the is_set() function.

For example:

# custom task function
def task():
	# perform task in iterations
	while True:
		# ...
        # check for stop
        if event.is_set():
            break

The main parent process, or another process, can then set the event in order to stop the new process from running.

The multiprocessing.Event can be set or made True via the set() function.

For example:

...
# set the event
event.set()
# wait for the new process to stop
process.join()

Note, this same approach can be used to stop a thread within a process. You can learn more in the tutorial:

How to Stop a Thread in Python

Now that we know how to stop a Python process, let's look at some worked examples.

Stop a Process With a Custom Function

In this section we can explore how to safely stop a child process that is executing a custom function.

First, we will develop an example that runs a custom function in a child process, then we will look at how to update the example to stop the process on demand.

Custom Function in a New Process

We can develop an example that runs a function in a new child process.

The new task function will execute in a loop and once finished the new process will terminate and the main parent process will terminate.

Firstly, we can define the task function.

The function will execute a loop five times. Each iteration it will block for a second, then report a message in an effort to simulate work and show progress.

Once the task is finished, the function will report a final message.

The task() function below implements this.

# custom task function
def task():
    # execute a task in a loop
    for i in range(5):
        # block for a moment
        sleep(1)
        # report a message
        print('Worker process running...', flush=True)
    print('Worker closing down', flush=True)

Next, in the main parent process we can create a new multiprocessing.Process instance that is configured to execute our task() function in a new process via the "target" argument.

...
# create and configure a new process
process = Process(target=task)

We can then immediately start the new process via the start() function.

...
# start the new process
process.start()

The parent process will then wait for the new child process to finish by joining the process.

If you are new to joining processes, you can learn more here:

How to Join a Process in Python

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of a process executing a custom function
from time import sleep
from multiprocessing import Process

# custom task function
def task():
    # execute a task in a loop
    for i in range(5):
        # block for a moment
        sleep(1)
        # report a message
        print('Worker process running...', flush=True)
    print('Worker closing down', flush=True)

# entry point
if __name__ == '__main__':
    # create and configure a new process
    process = Process(target=task)
    # start the new process
    process.start()
    # wait for the new process to finish
    process.join()

Running the example first creates and starts the new child process.

The main parent process then blocks until the new process finishes.

The new process executes in a loop, blocking for a moment and reporting progress. After five iterations of the loop, the new process reports a final message and terminates.

The parent process notices that the new process has finished and then terminates itself, closing the program.

Worker process running...
Worker process running...
Worker process running...
Worker process running...
Worker process running...
Worker closing down

Next, let's look at how we might update this example to stop the process on demand.

Stop Custom Function in New Process

We can update the example from the previous section to stop the new process on demand.

This can be achieved by creating a new multiprocessing.Event instance, passing it as an argument to the new process, and then updating the custom function to check if the event is set each iteration. Once the event is set, the function can break its task loop and exit, terminating the new process.

Firstly, we can update the task function to take the shared multiprocessing.Event instance as an argument.

# custom task function
def task(event):
	# ...

We can then update the task loop to check the status of the event each iteration, and if set to break the task loop.

...
# check for stop
if event.is_set():
    break

Tying this together, the updated task() function with these changes is listed below.

# custom task function
def task(event):
    # execute a task in a loop
    for i in range(5):
        # block for a moment
        sleep(1)
        # check for stop
        if event.is_set():
            break
        # report a message
        print('Worker process running...', flush=True)
    print('Worker closing down', flush=True)

Next, the main process can first create a new multiprocessing.Event instance to be shared between the processes.

...
# create the event
event = Event()

We can then update the parent process to block for a few seconds.

...
# block for a while
sleep(3)

Finally, we can set the event and trigger the new process to stop.

...
# stop the worker process
print('Main stopping process')
event.set()

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of stopping a new process
from time import sleep
from multiprocessing import Process
from multiprocessing import Event

# custom task function
def task(event):
    # execute a task in a loop
    for i in range(5):
        # block for a moment
        sleep(1)
        # check for stop
        if event.is_set():
            break
        # report a message
        print('Worker process running...', flush=True)
    print('Worker closing down', flush=True)

# entry point
if __name__ == '__main__':
    # create the event
    event = Event()
    # create and configure a new process
    process = Process(target=task, args=(event,))
    # start the new process
    process.start()
    # block for a while
    sleep(3)
    # stop the worker process
    print('Main stopping process')
    event.set()
    # wait for the new process to finish
    process.join()

Running the example first creates and starts the new child process.

The main parent process then blocks for a few seconds.

Meanwhile, the new process executes its task loop, blocking and reporting a message each iteration. It checks the event each iteration, which remains false and does not trigger the process to stop.

The main parent process wakes up, and then sets the event. It then joins the new process, waiting for it to terminate.

The task process checks the event which is discovered to be set (e.g. True). The process breaks the task loop, reports a final message then terminates the new process.

The parent process then terminates, closing the Python program.

Worker process running...
Worker process running...
Main stopping process
Worker closing down

Now that we know how to stop a custom function running in a new process, let's look at an example of stopping a custom process class.

Stop a Process With an Extended Class

In this section we can explore how to stop a process that is an object that extends the multiprocessing.Process class.

First, we will develop an example that extends the multiprocessing.Process class, then we will look at how to update the example to stop the process on demand.

Custom Process Class

We can run a task in a new process by extending the multiprocessing.Process class and overriding the constructor and the run() function.

In this section, we will show how to run the same task as in the previous section within a custom process class.

Firstly, we can define a new class that extends multiprocessing.Process.

# custom process class
class CustomProcess(Process):
	# ...

The new class may implement a constructor, and if so it must call the constructor of the multiprocessing.Process class to properly initialize the underlying process instance.

# constructor
def __init__(self):
    # call the parent constructor
    Process.__init__(self)

We can then override the run() method on the multiprocessing.Process class.

This function will run in a new process once the process is started via the start() function. Specifically we call start() and start() will call run() for us in a new spawned or forked child process.

Our task will be to execute a loop five times, block for a second to simulate work, and report a message. Then report a message once the task is complete.

# execute task
def run(self):
    # execute a task in a loop
    for i in range(5):
        # block for a moment
        sleep(1)
        # report a message
        print('Worker process running...', flush=True)
    print('Worker closing down', flush=True)

Tying this together the new CustomProcess class is listed below.

# custom process class
class CustomProcess(Process):
    # constructor
    def __init__(self):
        # call the parent constructor
        Process.__init__(self)

    # execute task
    def run(self):
        # execute a task in a loop
        for i in range(5):
            # block for a moment
            sleep(1)
            # report a message
            print('Worker process running...', flush=True)
        print('Worker closing down', flush=True)

Next, in the main parent process, we can create a new instance of our custom process class and start it in order to execute our task in a new process.

...
# create a new process
process = CustomProcess()
# start the new process
process.start()

We will then join() the new process and wait for the new process to terminate, before terminating the main parent process.

...
# wait for the new process to finish
process.join()

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of a extending the process class
from time import sleep
from multiprocessing import Process

# custom process class
class CustomProcess(Process):
    # constructor
    def __init__(self):
        # call the parent constructor
        Process.__init__(self)

    # execute task
    def run(self):
        # execute a task in a loop
        for i in range(5):
            # block for a moment
            sleep(1)
            # report a message
            print('Worker process running...', flush=True)
        print('Worker closing down', flush=True)

# entry point
if __name__ == '__main__':
    # create a new process
    process = CustomProcess()
    # start the new process
    process.start()
    # wait for the new process to finish
    process.join()

Running the example first creates our custom process class then starts executing the new process.

The main process then blocks waiting for the new process to finish.

Our custom process class runs our task in a new process of execution, looping five times. Each iteration it blocks for a moment then reports a message.

Once the task is finished, it reports a final message.

The new process terminates, then the main process terminates, closing the process.

Worker process running...
Worker process running...
Worker process running...
Worker process running...
Worker process running...
Worker closing down

Next, let's look at how we might update this example to stop the task on demand.

Stop Custom Process Class

We can update the example in the previous section to stop the process on demand.

This can be achieved by sharing a multiprocessing.Event with the new process and updating the run() function to check if the event is set each iteration. Once set, the run() function can exit, which will terminate the new process.

Firstly, we can update the constructor of our new process class to create a new instance of the multiprocessing.Event class and store it as an instance variable attribute.

# constructor
def __init__(self):
    # call the parent constructor
    Process.__init__(self)
    # create and store an event
    self.event = Event()

Next, we can update the run() function to check the status of the multiprocessing.Event each iteration and to break the task loop once the event is set.

# execute task
def run(self):
    # execute a task in a loop
    for i in range(5):
        # block for a moment
        sleep(1)
        # check for stop
        if self.event.is_set():
            break
        # report a message
        print('Worker process running...', flush=True)
    print('Worker closing down', flush=True)

Tying this together, the updated CustomProcess class that can be stopped on demand is listed below.

# custom process class
class CustomProcess(Process):
    # constructor
    def __init__(self):
        # call the parent constructor
        Process.__init__(self)
        # create and store an event
        self.event = Event()

    # execute task
    def run(self):
        # execute a task in a loop
        for i in range(5):
            # block for a moment
            sleep(1)
            # check for stop
            if self.event.is_set():
                break
            # report a message
            print('Worker process running...', flush=True)
        print('Worker closing down', flush=True)

The main parent process can then set the event via the attribute on the new CustomProcess instance.

...
# stop the worker process
print('Main stopping process')
process.event.set()

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of stopping a custom process class
from time import sleep
from multiprocessing import Process
from multiprocessing import Event

# custom process class
class CustomProcess(Process):
    # constructor
    def __init__(self):
        # call the parent constructor
        Process.__init__(self)
        # create and store an event
        self.event = Event()

    # execute task
    def run(self):
        # execute a task in a loop
        for i in range(5):
            # block for a moment
            sleep(1)
            # check for stop
            if self.event.is_set():
                break
            # report a message
            print('Worker process running...', flush=True)
        print('Worker closing down', flush=True)

# entry point
if __name__ == '__main__':
    # create a new process
    process = CustomProcess()
    # start the new process
    process.start()
    # block for a while
    sleep(3)
    # stop the worker process
    print('Main stopping process')
    process.event.set()
    # wait for the new process to finish
    process.join()

Running the example first involves creating an instance of the custom process and starting it.

The main process will then block for a few seconds.

The new process will start executing its task loop, blocking for a moment and reporting progress.

The main process then wakes up and sets the event, triggering the new process to stop.

The new process notices that the event is set and then exits the task loop, reporting a final message and terminating the new process.

The main process waits for the new process to terminate, before then stopping itself and closing the application.

Worker process running...
Worker process running...
Main stopping process
Worker closing down

Common Questions About Stopping Processes

This section lists common questions about stopping a child process in Python.

Do you have any questions about stopping a process? Let me know in the comments and I will do my best to help.

What if My Task Does Not Have a Loop?

Your task does not need to have a loop in order to be stopped.

Instead, you need to add frequent checks of the shared multiprocessing.Event and return from the task() function or run() function once the event is set.

This will depend on the specifics of the task you wish to exit in a new process.

If your task involves blocking, such as reading or writing IO or waiting on another process, you may want to adopt a busy wait and have the multiprocessing.Event checked each iteration.

If your task remains idle for some time, such as waiting for sporadic events, it may be able to wait on the multiprocessing.Event itself via the wait() function.

What if My Task Raises An Exception?

If your task executing in a new process raises an exception that is not handled, it will terminate the process.

This could be an alternate way for you to trigger a new process to terminate, if convenient.

What if My New Process Has Already Stopped?

If the new process has already stopped and you set the event, it will not have any effect.

It would not be a problem.

What if My New Process is a Daemon Process?

If your task is executing in a daemon process, then there is no need to stop it manually as it will be terminated for you once all non-daemon processes (e.g. the main parent process) terminate.

You can learn more about daemon process here:

Daemon Process in Python

How Can We Stop Multiple Processes?

You can share the same multiprocessing.Event between all of the child processes that might want to stop.

Each process can frequently check the status of the event.

Once set, all processes can exit their task and terminate.

How Can We Stop Tasks in a ProcessPoolExecutor?

The same technique of using a multiprocessing.Event can be used to stop tasks executing in a ProcessPoolExecutor.

You can see an example of this here:

How To Stop Running Tasks in the ProcessPoolExecutor in Python

Do We Need To Use a Manager

No.

A shared Event object can be created via a multiprocessing.Manager.

For example:

...
# create a manager
with Manager() as manager:
    # create an event from the manager
    event = manager.Event()
    # ...

This is only needed if the event is to be shared across processes running on different machines.

Why Not use Process.terminate() or Process.kill()

A child process can be stopped by calling the terminate() method, or the kill() method.

If called, the process will stop immediately.

This is not a graceful and safe way to stop a child process as it does not give the child process any way to respond to the request to stop, such as close and release resources.

Takeaways

You now know how to stop a process in Python.

If you enjoyed this tutorial, you will love my book: Python Multiprocessing Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.