Last Updated on September 12, 2022
You can print() to stdout from child processes by setting the “flush” argument to True, or by using the ‘fork‘ start method.
In this tutorial you will discover how to print() from child processes in Python.
Let’s get started.
Problem with print() in Child Processes
Printing to standard out (stdout) with the built-in print() function may not work property from child processes.
For example, you may print output messages for the user or debug messages from a child process and they may never appear, or may only appear when the child process is terminated.
|
1 2 3 |
... # report a message from a child process print('Hello from the child process') |
Do you have this problem?
Let me know in the comments below.
This is a very common situation and the cause is well understood and easy to workaround.
Next, let’s look at exactly why this problem occurs.
Run loops using all CPUs, download your FREE book to learn how.
Why print() Doesn’t Work From Child Processes
The print() function is a built-in function for displaying messages on standard output or stdout.
When you call print() from a child process created using the ‘spawn‘ start method, the message will not appear.
This is because the messages are block buffered by default and the buffer is not flushed by default after every message. This is unlike the main process that is interactive and will flush messages after each line, e.g. line buffered.
When interactive, the stdout stream is line-buffered. Otherwise, it is block-buffered like regular text files.
— sys — System-specific parameters and functions
Instead, the buffered messages are only flushed occasionally, such as when the child process terminates and the buffer is garbage collected.
Next, let’s look at how we can make the print() function work as expected from child processes.
How To Print From Child Processes
There are two techniques you can use to make print() messages appear immediately when called from a child process.
They are:
- Flush stdout.
- Use the ‘fork‘ start method.
Flush Standard Output
You can flush stdout automatically with each call to print().
This can be achieved by setting the ‘flush‘ argument to True.
For example:
|
1 2 3 |
... # report a message from a child process print('Hello from the child process', flush=True) |
An alternate approach is to call the flush() function on the sys.stdout object directly.
For example:
|
1 2 3 4 5 |
... # report a message from a child process print('Hello from the child process') # flush output sys.stdout.flush() |
Use Fork Start Method
There are three main techniques for starting a child process in Python.
They are:
- Spawn
- Fork
- ForkServer
On Windows and macOS, the ‘spawn‘ technique is used by default.
Alternatively, on another platform you may choose to use the ‘spawn‘ start method.
For example:
|
1 2 3 |
... # set the start method to spawn set_start_method('spawn') |
The problem with the print() function only occurs when using the ‘spawn‘ start method.
You can change the start method to ‘fork‘ which will cause print() to work as expected.
Note, the ‘fork‘ start method is not supported on Windows at the time of writing.
You can set the start method via the multiprocessing.set_start_method() function.
For example:
|
1 2 3 |
... # set the start method to fork set_start_method('fork') |
You can learn more about process start methods in the tutorial:
Now that we know how to fix the print() function from child processes, let’s look at some worked examples.
Free Python Multiprocessing Course
Download your FREE multiprocessing PDF cheat sheet and get BONUS access to my free 7-day crash course on the multiprocessing API.
Discover how to use the Python multiprocessing module including how to create and start child processes and how to use a mutex locks and semaphores.
Example of Broken Print From Child Process
Before we explore how to make the print() function behave as expected from child processes, let’s demonstrate the problem of printing from child processes.
In this example, we will start a child process to execute a custom function that prints a message, then blocks for a second. The main process will start the child process, then wait for it to terminate before printing a message itself.
Instead of the message from the child process appearing immediately after the child process is started, as we might expect, instead, the message from the child process does not appear until the child process is terminated.
First, let’s define a function to be executed in a child process.
This function will report a message then sleep for one second before exiting.
The task() function below implements this.
|
1 2 3 4 5 6 |
# function executed in a child process def task(): # report a message print('Hello from child process') # block for a moment sleep(1) |
Next, in the main process we will set the start method to ‘spawn‘, to ensure the print() function does not work as expected.
|
1 2 3 |
... # set the start method to spawn set_start_method('spawn') |
Next, we will print a message from the main process, which will appear immediately.
|
1 2 3 |
... # report a message print('Hello from main process') |
We will then configure a new multiprocessing.Process instance to execute our custom function, start the child process, then wait for it to terminate.
|
1 2 3 4 5 6 7 |
... # create and configure child process process = Process(target=task) # start child process process.start() # wait for child process to finish process.join() |
Finally, the main process will report another message.
|
1 2 3 |
... # report another message print('Hello again from main process') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
# SuperFastPython.com # example of print from child process not working from time import sleep from multiprocessing import Process from multiprocessing import set_start_method # function executed in a child process def task(): # report a message print('Hello from child process') # block for a moment sleep(1) # protected entry point if __name__ == '__main__': # set the start method to spawn set_start_method('spawn') # report a message print('Hello from main process') # create and configure child process process = Process(target=task) # start child process process.start() # wait for child process to finish process.join() # report another message print('Hello again from main process') |
Running the example first prints a message from the main process, which appears immediately.
Next, the child process is created and started and the main process blocks and waits for it to finish.
The child process calls print(), but the message does not appear. It then blocks for one second, then terminates.
Finally, the message from the child process appears after the child process has terminated.
The main process then reports a second message, which appears immediately.
This highlights that printing from a child process created with the ‘spawn‘ start method does not work as we might expect.
|
1 2 3 |
Hello from main process Hello from child process Hello again from main process |
Next, let’s look at how we might fix the behavior of the print() statement by using the ‘fork’ start method.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Fixed Print With Fork Start Method
We can explore how we can make print() from child processes behave as expected by changing the start method.
In this example, we will update the example from the previous section to use the ‘fork‘ start method instead of the ‘spawn‘ start method.
For example:
|
1 2 3 |
... # set the start method to fork set_start_method('fork') |
And that’s it.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
# SuperFastPython.com # example of print from child process not working from time import sleep from multiprocessing import Process from multiprocessing import set_start_method # function executed in a child process def task(): # report a message print('Hello from child process') # block for a moment sleep(1) # protected entry point if __name__ == '__main__': # set the start method to fork set_start_method('fork') # report a message print('Hello from main process') # create and configure child process process = Process(target=task) # start child process process.start() # wait for child process to finish process.join() # report another message print('Hello again from main process') |
Running the example first prints a message from the main process, which appears immediately.
Next, the child process is created and started and the main process blocks and waits for it to finish.
The child process calls print() and the message appears immediately. It then blocks for one second, then terminates.
The main process then reports a second message, which appears immediately.
This highlights that changing the start method to fork will cause printing from a child process to work as expected.
Note, the fork start method is not supported on Windows.
|
1 2 3 |
Hello from main process Hello from child process Hello again from main process |
Next, let’s look at how we might fix printing from a child process by flushing each message.
Fix Print With flush=True
We can explore how to fix printing from child processes by flushing stdout each time the print() function is called.
In this example, we will update the broken example above and change the print() statement so that stdout is flushed directly.
This can be achieved by setting the ‘flush‘ argument to True.
For example:
|
1 2 3 |
... # report a message print('Hello from child process', flush=True) |
The complete task() function with this change is listed below.
|
1 2 3 4 5 6 |
# function executed in a child process def task(): # report a message print('Hello from child process', flush=True) # block for a moment sleep(1) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
# SuperFastPython.com # example of print from child process with flushed output from time import sleep from multiprocessing import Process from multiprocessing import set_start_method # function executed in a child process def task(): # report a message print('Hello from child process', flush=True) # block for a moment sleep(1) # protected entry point if __name__ == '__main__': # set the start method to spawn set_start_method('spawn') # report a message print('Hello from main process') # create and configure child process process = Process(target=task) # start child process process.start() # wait for child process to finish process.join() # report another message print('Hello again from main process') |
Running the example first prints a message from the main process, which appears immediately.
Next, the child process is created and started and the main process blocks and waits for it to finish.
The child process calls print() and the message appears immediately. It then blocks for one second, then terminates.
The main process then reports a second message, which appears immediately.
This highlights that flushing the standard output stream when calling print() will cause printed messages to appear immediately as we expect.
|
1 2 3 |
Hello from main process Hello from child process Hello again from main process |
Next, let’s explore how we might manually flush the standard output stream.
Fix Print With Explicit Call to Flush
We can explore how to fix printing from a child process by explicitly flushing standard out.
In this example, we can fix the broken example above by adding a call to sys.stdout.flush() after the call to print() in the child process.
For example:
|
1 2 3 4 5 |
... # report a message print('Hello from child process') # flush output sys.stdout.flush() |
The complete task() function with this change is listed below.
|
1 2 3 4 5 6 7 8 |
# function executed in a child process def task(): # report a message print('Hello from child process') # flush output sys.stdout.flush() # block for a moment sleep(1) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
# SuperFastPython.com # example of print from child process with flushed output from time import sleep from multiprocessing import Process from multiprocessing import set_start_method import sys # function executed in a child process def task(): # report a message print('Hello from child process') # flush output sys.stdout.flush() # block for a moment sleep(1) # protected entry point if __name__ == '__main__': # set the start method to spawn set_start_method('spawn') # report a message print('Hello from main process') # create and configure child process process = Process(target=task) # start child process process.start() # wait for child process to finish process.join() # report another message print('Hello again from main process') |
Running the example first prints a message from the main process, which appears immediately.
Next, the child process is created and started and the main process blocks and waits for it to finish.
The child process calls print(), then stdout is flushed and the message appears immediately. It then blocks for one second, then terminates.
The main process then reports a second message, which appears immediately.
This highlights that flushing the standard output stream explicitly after calling print() will cause printed messages to appear immediately as we expect.
|
1 2 3 |
Hello from main process Hello from child process Hello again from main process |
You can learn more about multiprocessing in the tutorial:
Further Reading
This section provides additional resources that you may find helpful.
Python Multiprocessing Books
- Python Multiprocessing Jump-Start, Jason Brownlee (my book!)
- Multiprocessing API Interview Questions
- Multiprocessing API Cheat Sheet
I would also recommend specific chapters in the books:
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- High Performance Python, Ian Ozsvald and Micha Gorelick, 2020.
- See: Chapter 9: The multiprocessing Module
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Multiprocessing: The Complete Guide
- Python Multiprocessing Pool: The Complete Guide
- Python ProcessPoolExecutor: The Complete Guide
APIs
References
Takeaways
You now know how to print() from child processes in Python.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Knut Troim on Unsplash
Hello, I have tried each of these methods, including combinations of the listed methods. I am unable to print any line from the child processes. I have no earthly idea if this is system incapability on my end or if I am misunderstanding the syntax of the library. Any help is greatly appreciated.
Sorry to hear that. Perhaps the cause is something else, e.g. the development environment or operating system?
Perhaps you can try making a broader change and see if it has an effect, e.g. try running your program from the command line? Or try the simplest possible example to print from a second process?
Let me know how you go.
I experienced the same problem as reported by a user a year ago. None of your methods for making print syayements appear worked for me. Also, fork. while not deprecated has strong warnings and will not be the Linux default from 3.14 on. I can give you source code but I do not know how to send it. BTW, Python was run from the command line on Ubuntu 22.04. I badly want print to work as it will show me whether my multiprocessing structure is sound. The final version of the program will have a lot of code where the print statements now exist.
Sorry to hear that.
The most reliable method across platforms is to explicitly flush stdout, e.g.:
Are you able to to test this and provide the smallest possible example that does not work for you?