Automatically Start Processes in Python

May 11, 2022 Python Multiprocessing

You can define a new class that extends multiprocessing.Process that automatically starts new processes.

In this tutorial you will discover how to auto-start new processes in Python.

Let's get started.

Need to Auto-Start a Process

A process is a running instance of a computer program.

Every Python program is executed in a Process, which is a new instance of the Python interpreter. This process has the name MainProcess and has one thread used to execute the program instructions called the MainThread. Both processes and threads are created and managed by the underlying operating system.

Sometimes we may need to create new child processes in our program in order to execute code concurrently.

Python provides the ability to create and manage new processes via the multiprocessing.Process class.

You can learn more about multiprocessing in the tutorial:

Multiprocessing in Python: The Complete Guide

A problem with processes is that you must first create an instance of the multiprocessing.Process class, then call the start() function before the process is started.

In many cases, it would be easier if processes could be started immediately once the class was created.

Is there a way to automatically start processes in Python?

How to Auto-Start a Process

A process can automatically be started by creating a subclass of multiprocessing.Process that calls the start() function in the constructor.

The typical way a process is started involves two steps:

Create the instance of multiprocessing.Process.
Call the start() function.

For example:

...
# create the process instance
process = multiprocessing.Process(...)
# start the process
process.start()

This adds an additional step if the goal is to create and start the process immediately.

In fact, in most cases that a process is created it is to be started as the next step. Rarely do we need a delay between creating a process instance and starting the process.

We can create and start a process in one step with a compound statement, for example:

...
# create and start a process
multiprocessing.Process(...).start()

The downsides of this approach is that we cannot maintain a reference to the new process instance, and we must combine multiple statements into a single compound statement, which is bad for readability and debugging.

An alternative is to simplify creating and starting a new process in a single step that involves:

Define a custom class that extends multiprocessing.Process.
Override the class constructor and call start().

Importantly, when extending the multiprocessing.Process class we must call the parent constructor first to ensure that the process is set up correctly.

We must also pass on all arguments and keyword arguments to the parent constructor so that the new process is set up correctly, e.g. calls the target task function with any arguments.

You can learn more about extending the multiprocessing.Process class in this tutorial:

How to Extend the Process Class in Python

The example below defines a custom process class that automatically starts the process.

# custom process class that auto starts the process
class AutoStartProcess(Process):
    # constructor
    def __init__(self, *args, **kwargs):
        # call the the parent constructor
        super().__init__(*args, **kwargs)
        # start the process
        self.start()

We can then create an instance of this class that will start the process immediately.

For example:

...
# create and start the new process
process = AutoStartProcess(...)

Next, let's look at using the automatically starting process in a worked example.

Example of Auto-Starting a Process

We can develop an example to demonstrate how to automatically start a new process.

In this example, we will define a new AutoStartProcess class and then use it to execute a custom target function that blocks for a moment and reports a message.

First, we can define the custom class that extends the multiprocessing.Process class, passes along all arguments to the parent constructor, then starts the process.

We will use the AutoStartProcess class defined in the previous section, provided again below.

# custom process class that automatically starts processes when they are constructed
class AutoStartProcess(Process):
    # constructor
    def __init__(self, *args, **kwargs):
        # call the the parent constructor
        super().__init__(*args, **kwargs)
        # start the process
        self.start()

Next, we can define a custom task function to execute in a new process.

The function will first print a message to let us know that the process is running, it will then sleep for one second by calling time.sleep() function, then reports a final message before the process terminates.

The task() function below implements this.

# task function
def task():
    print('Task starting')
    # block for a moment
    sleep(1)
    # report
    print('Task all done')

Finally, the parent process will create an instance of the AutoStartProcess class configured to execute our task() function.

Creating a new instance will start the new process immediately.

...
# create and start the new process
process = AutoStartProcess(target=task)

You can learn more about executing functions in a new process in this tutorial:

Run a Function in a Child Process

The main process will then block until the new process terminates. This can be achieved by calling the join() function.

You can learn more about joining processes in this tutorial:

How to Join a Process in Python

...
# wait for the new process to finish
process.join()

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of automatically starting a process
from time import sleep
from multiprocessing import Process

# custom process class that automatically starts processes when they are constructed
class AutoStartProcess(Process):
    # constructor
    def __init__(self, *args, **kwargs):
        # call the the parent constructor
        super().__init__(*args, **kwargs)
        # start the process
        self.start()

# task function
def task():
    print('Task starting')
    # block for a moment
    sleep(1)
    # report
    print('Task all done')

# entry point
if __name__ == '__main__':
    # create and start the new process
    process = AutoStartProcess(target=task)
    # wait for the new process to finish
    process.join()

Running the example first creates an instance of the new process class.

The constructor of the new process class calls the parent constructor of the multiprocessing.Process class, which ensures that the new process is set up correctly. The constructor then starts the new process immediately.

The parent process then joins the new child process and waits for it to terminate.

The new process executes our custom task function. A message is printed, the process sleeps for a moment, then a final message is reported before the process terminates.

Task starting
Task all done

This demonstrates how to define a new class for auto-starting a new process, and how it can be used to execute task functions.

Takeaways

You now know how to auto-start new processes in Python