Run a Function in a Child Process

May 4, 2022 Python Multiprocessing

You can run a function in a new child Process via the "target" argument on the multiprocessing.Process class.

In this tutorial you will discover how to run a function in a new process.

Let's get started.

Need to Run A Function in a New Process

A process is a running instance of a computer program.

Every Python program is executed in a Process, which is a new instance of the Python interpreter. Each Python process has one thread used to execute the program instructions called the MainThread. Both processes and threads are created and managed by the underlying operating system.

Sometimes we may need to create additional processes in our program in order to execute code concurrently.

Python provides the ability to create and manage new processes via the multiprocessing.Process class.

You can learn more about multiprocessing in the tutorial:

Multiprocessing in Python: The Complete Guide

One way of running a function in a new process is via an argument on the multiprocessing.Process class.

How can we run a function in a new process using the multiprocessing.Process class?

How to Run a Function In a Process

To run a function in another process:

Create an instance of the multiprocessing.Process class.
Specify the name of the function via the "target" argument.
Call the start() function.

First, we must create a new instance of the multiprocessing.Process class and specify the function we wish to execute in a new process via the "target" argument.

...
# create a process
process = multiprocessing.Process(target=task)

The function executed in another process may have arguments in which case they can be specified as a tuple and passed to the "args" argument of the multiprocessing.Process class constructor or as a dictionary to the "kwargs" argument.

...
# create a process
process = multiprocessing.Process(target=task, args=(arg1, arg2))

We can then start executing the process by calling the start() function.

The start() function will return immediately and the operating system will execute the function in a separate process as soon as it is able.

...
# run the new process
process.start()

A new instance of the Python interrupter will be created and a new thread within the new process will be created to execute our target function.

And that's all there is to it.

We do not have control over when the process will execute precisely or which CPU core will execute it. Both of these are low-level responsibilities that are handled by the underlying operating system.

Next, let's look at a worked example of executing a function in a new process.

Example of Running a Function in a Process

First, we can define a custom function that will be executed in another process.

We will define a simple function that blocks for a moment then prints a statement.

The function can have any name we like, in this case we'll name it "task".

# a custom function that blocks for a moment
def task():
    # block for a moment
    sleep(1)
    # display a message
    print('This is from another process')

Next, we can create an instance of the multiprocessing.Process class and specify our function name as the "target" argument in the constructor.

...
# create a process
process = Process(target=task)

Once created we can run the process which will execute our custom function in a new native process, as soon as the operating system can.

...
# run the process
process.start()

The start() function does not block, meaning it returns immediately.

We can explicitly wait for the new process to finish executing by calling the join() function.

...
# wait for the process to finish
print('Waiting for the process...')
process.join()

Tying this together, the complete example of executing a function in another process is listed below.

# SuperFastPython.com
# example of running a function in another process
from time import sleep
from multiprocessing import Process

# a custom function that blocks for a moment
def task():
    # block for a moment
    sleep(1)
    # display a message
    print('This is from another process')

# entry point
if __name__ == '__main__':
    # create a process
    process = Process(target=task)
    # run the process
    process.start()
    # wait for the process to finish
    print('Waiting for the process...')
    process.join()

Running the example first creates the multiprocessing.Process then calls the start() function. This does not start the process immediately, but instead allows the operating system to schedule the function to execute as soon as possible.

At some point a new instance of the Python interpreter is created that has a new thread which will execute our target function.

The main thread of our initial process then prints a message waiting for the new process to complete, then calls the join() function to explicitly block and wait for the new process to terminate.

Once the custom function returns, the new process is closed. The join() function then returns and the main thread exits.

Waiting for the process...
This is from another process

Example of Running a Function in a Process With Arguments

We can execute functions in another process that takes arguments.

This can be demonstrated by first updating our task() function from the previous section to take two arguments, one for the time in seconds to block and the second for a message to display.

# a custom function that blocks for a moment
def task(sleep_time, message):
    # block for a moment
    sleep(sleep_time)
    # display a message
    print(message)

Next, we can update the call to the multiprocessing.Process constructor to specify the two arguments in the order that our task() function expects them as a tuple via the "args" argument.

...
# create a process
process = Process(target=task, args=(1.5, 'New message from another process'))

Tying this together, the complete example of executing a custom function that takes arguments in a separate process is listed below.

# SuperFastPython.com
# example of running a function with arguments in another process
from time import sleep
from multiprocessing import Process

# a custom function that blocks for a moment
def task(sleep_time, message):
    # block for a moment
    sleep(sleep_time)
    # display a message
    print(message)

# entry point
if __name__ == '__main__':
    # create a process
    process = Process(target=task, args=(1.5, 'New message from another process'))
    # run the process
    process.start()
    # wait for the process to finish
    print('Waiting for the process...')
    process.join()

Running the example creates the process specifying the function name and the arguments to the function.

The new process is started and the function blocks for the parameterized number of seconds and prints the parameterized message.

Waiting for the process...
New message from another process

Takeaways

You now know how to execute functions in another process via the "target" argument on the multiprocessing.Process class.

If you enjoyed this tutorial, you will love my book: Python Multiprocessing Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.