Multiprocessing Pool Restarts Workers if Killed

Last Updated on September 12, 2022

Child worker processes in the multiprocessing pool will be restarted automatically if killed.

In this tutorial you will discover what happens if a child worker process is killed in the multiprocessing pool in Python.

Let’s get started.

Table of Contents

What If One Worker is Killed

The multiprocessing pool provides a pool of reusable workers for executing ad hoc tasks with process-based concurrency.

An instance of the multiprocessing.Pool class can be created, specifying the number of workers to create, otherwise a default number of workers will be created to match the number of logical CPUs in the system.

Once created, ad hoc tasks can be issued to the pool for execution via the Pool.apply() function. Multiple tasks may be executed in the book by calling the same function with different arguments via the Pool.map() function.

Tasks may be executed asynchronously in the pool via Pool.apply_async() and Pool.map_async(), allowing the caller to carry on with other tasks.

Once concern with the multiprocessing pool is what happens if one child worker process in the pool is killed.

A child worker process may be killed any number of ways, for example:

An external user or program may manually kill the process.
The program may programmatically kill the process, inadvertently.

There are two main outcomes that may occur if one child worker process is killed.

The worker is permanently unavailable and perhaps the pool is broken.
The worker is restarted automatically by the pool.

What happens if a child worker process in the pool is killed?

Run loops using all CPUs, download your FREE book to learn how.

Child Workers Are Restarted if Killed

Child worker processes in the multiprocessing pool are restarted if they are killed.

This is a capability and feature of the multiprocessing.Pool class, although it is not documented.

This capability is likely related to the class’s ability to limit the number of tasks executed by each child worker before the worker is replaced via the “maxtasksperchild” argument to the class constructor.

For example, if a parent Python process gets a handle on a child worker processes, such as via its multiprocessing.Process instance, and kills it, such as via the kill() function, then the multiprocessing.Pool will identify that this has occurred and will restart the child worker process.

Now that we know what happens if a child worker process is killed, let’s look at a worked example.

Download Now: Free Process Pool PDF Cheat Sheet

Example of Killing a Child Worker Process

We can explore what happens if we kill one child worker process in the multiprocessing pool.

In this example, we will create a multiprocessing pool with a fixed number of workers. We will then list all child processes for the process to confirm that the workers were created and running. A single child process is then killed directly. The child processes for the current process are then listed again to confirm that a new worker was started in its place.

Firstly, we can create a multiprocessing pool with four worker processes.

Don’t worry if you have more or fewer CPU cores, as we will not be running any tasks.

...

# create a pool

pool = Pool(4)

Next, we can wait a moment for the pool to initialize completely. This is probably not required, but will likely avoid any possible race conditions in checking if the workers are running.

...

# wait a moment

sleep(0.5)

We can then get a list of all child processes for the current process. This will include all workers in the process pool.

This can be achieved via the multiprocessing.active_children() function

...

# report all active child processes

children = active_children()

We can then report the details of each process in this list.

...

for child in children:

print(child)

We can then get one child worker process instance from the list and explicitly kill it.

This will send a SIGKILL signal and terminate the child process immediately.

...

# kill one child

child = children[0]

child.kill()

We will then wait a moment for the child process to be terminated.

...

# wait a moment

sleep(0.5)

The details of the terminated process can be reported, confirming it was killed.

1 2	... print(child)

We can then get a list of all child processes for the current process and report their details.

This may include any new child workers created to replace the killed process.

...

# report all active child processes

for child in active_children():

print(child)

Finally, we can close the multiprocessing pool now that we are finished with it.

...

# close the pool

pool.close()

Tying this together, the complete example is listed below.

# SuperFastPython.com

# what happens if we kill a child worker process

from time import sleep

from multiprocessing import Pool

from multiprocessing import active_children

# protect the entry point

if __name__ == '__main__':

# create a pool

pool = Pool(4)

# wait a moment

sleep(0.5)

# report all active child processes

children = active_children()

for child in children:

print(child)

# kill one child

child = children[0]

child.kill()

# wait a moment

sleep(0.5)

print(child)

# report all active child processes

for child in active_children():

print(child)

# close the pool

pool.close()

Running the example first creates a multiprocessing pool with four child worker processes.

The main process then blocks a moment to wait for the pool to initialize completely.

Next, a list of all child processes is retrieved and their details are reported.

In this case, we can see the details of the four child worker processes in the pool.

One of the workers is then killed and the main process blocks for a moment.

It then reports the details of the killed process and indeed we can see that its status is “stopped” and that its exit code indicates it received a SIGKILL signal.

Finally, all child processes are retrieved again and their details are reported. We can see four processes, and a new process (id 10095) was created to replace the process that was killed (id 10092).

This confirms that the multiprocessing pool will create new child worker processes to replace those workers that are killed.

Note, process ids will differ each time the program is run as the ids are allocated by the underlying operating system.