Last Updated on March 17, 2023
You can use a mutual exclusion (mutex) lock for processes via the multiprocessing.Lock class.
In this tutorial you will discover how to use mutex locks with processes in Python.
Let’s get started.
Need for a Mutual Exclusion Lock
A process is a running instance of a computer program.
Every Python program is executed in a Process, which is a new instance of the Python interpreter. This process has the name MainProcess and has one thread used to execute the program instructions called the MainThread. Both processes and threads are created and managed by the underlying operating system.
Sometimes we may need to create new child processes in our program in order to execute code concurrently.
Python provides the ability to create and manage new processes via the multiprocessing.Process class.
You can learn more about multiprocessing in the tutorial:
When writing concurrent programs we may need to share data or resources between processes, which typically must be protected with a mutual exclusion lock.
The most commonly used mechanism for ensuring mutual exclusion is a mutual exclusion lock or mutex, or simply lock. A mutex is a special type of object that has support in the underlying hardware. The basic idea is that each critical section is protected by a lock.
— Page 53, An Introduction to Parallel Programming, 2020.
You can learn more about mutex locks in the tutorial:
How can we use a mutex lock with processes in Python?
Run loops using all CPUs, download your FREE book to learn how.
How to Use Mutex Locks
A mutual exclusion lock or mutex lock is a synchronization primitive intended to prevent a race condition.
You can learn more about race conditions between processes in the tutorial:
Python provides a mutual exclusion lock for use with processes via the multiprocessing.Lock class.
An instance of the lock can be created and then acquired by processes before accessing a critical section, and released after the critical section.
For example:
|
1 2 3 4 5 6 7 8 |
... # create a lock lock = multiprocessing.Lock() # acquire the lock lock.acquire() # ... # release the lock lock.release() |
Only one process can have the lock at any time. If a process does not release an acquired lock, it cannot be acquired again.
The process attempting to acquire the lock will block until the lock is acquired, such as if another process currently holds the lock then releases it.
We can attempt to acquire the lock without blocking by setting the “block” argument to False. If the lock cannot be acquired, a value of False is returned.
|
1 2 3 |
... # acquire the lock without blocking lock.acquire(block=false) |
We can also attempt to acquire the lock with a timeout, that will wait the set number of seconds to acquire the lock before giving up. If the lock cannot be acquired, a value of False is returned.
|
1 2 3 |
... # acquire the lock with a timeout lock.acquire(timeout=10) |
We can also use the lock via the context manager protocol via the with statement, allowing the critical section to be a block within the usage of the lock and for the lock to be released automatically once the block has completed.
For example:
|
1 2 3 4 5 6 |
... # create a lock lock = multiprocessing.Lock() # acquire the lock with lock: # ... |
This is the preferred usage as it makes it clear where the protected code begins and ends, and ensures that the lock is always released, even if there is an exception or error within the critical section.
We can also check if the lock is currently acquired by a process via the locked() function.
|
1 2 3 4 |
... # check if a lock is currently acquired if lock.locked(): # ... |
Python also provides a mutex lock for use with threads. You can learn more about it in this tutorial:
Now that we know how to use the multiprocessing.Lock class, let’s look at a worked example.
Example of Using a Multiprocessing Lock
We can develop an example to demonstrate how to use the mutex lock.
First, we can define a target task function that takes a lock as an argument and uses the lock to protect a critical section.
In this case, the critical section involves reporting a message and blocking for a fraction of a second.
|
1 2 3 4 5 6 |
# work function def task(lock, identifier, value): # acquire the lock with lock: print(f'>process {identifier} got the lock, sleeping for {value}') sleep(value) |
Next, we can then create one instance of the multiprocessing.Lock to be shared among the processes.
|
1 2 3 |
... # create the shared lock lock = Lock() |
We can then create many processes, each configured to execute our task() function and compete to execute the critical section.
Each process will receive the shared lock as an argument as well as an integer id between 0 and 9 and a random time to sleep in seconds between 0 and 1.
We can implement this via a list comprehension, creating a list of ten configured multiprocessing.Process instances.
|
1 2 3 |
... # create a number of processes with different sleep times processes = [Process(target=task, args=(lock, i, random())) for i in range(10)] |
Next, we can start all of the processes.
|
1 2 3 4 |
... # start the processes for process in processes: process.start() |
Finally, we can wait for all of the new child processes to terminate.
|
1 2 3 4 |
... # wait for all processes to finish for process in processes: process.join() |
Tying this together, the complete example of using a lock is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
# SuperFastPython.com # example of a mutual exclusion (mutex) lock for processes from time import sleep from random import random from multiprocessing import Process from multiprocessing import Lock # work function def task(lock, identifier, value): # acquire the lock with lock: print(f'>process {identifier} got the lock, sleeping for {value}') sleep(value) # entry point if __name__ == '__main__': # create the shared lock lock = Lock() # create a number of processes with different sleep times processes = [Process(target=task, args=(lock, i, random())) for i in range(10)] # start the processes for process in processes: process.start() # wait for all processes to finish for process in processes: process.join() |
Running the example starts ten processes that are all configured to execute our custom function.
The child processes are then started and the main process blocks until all child processes finish.
Each child process attempts to acquire the lock within the task() function. Only one process can acquire the lock at a time and once they do, they report a message including their id and how long they will sleep. The process then blocks for a fraction of a second before releasing the lock.
Your specific results may vary given the use of random numbers. Try running the example a few times
|
1 2 3 4 5 6 7 8 9 10 |
>process 2 got the lock, sleeping for 0.34493199862842716 >process 0 got the lock, sleeping for 0.1690829274493061 >process 1 got the lock, sleeping for 0.586700038562483 >process 3 got the lock, sleeping for 0.8439760508777033 >process 4 got the lock, sleeping for 0.49642440261633747 >process 6 got the lock, sleeping for 0.7291278047802177 >process 5 got the lock, sleeping for 0.4495745681185115 >process 7 got the lock, sleeping for 0.6844618818829677 >process 8 got the lock, sleeping for 0.21518155457911792 >process 9 got the lock, sleeping for 0.30577395898093285 |
Free Python Multiprocessing Course
Download your FREE multiprocessing PDF cheat sheet and get BONUS access to my free 7-day crash course on the multiprocessing API.
Discover how to use the Python multiprocessing module including how to create and start child processes and how to use a mutex locks and semaphores.
Further Reading
This section provides additional resources that you may find helpful.
Python Multiprocessing Books
- Python Multiprocessing Jump-Start, Jason Brownlee (my book!)
- Multiprocessing API Interview Questions
- Multiprocessing API Cheat Sheet
I would also recommend specific chapters in the books:
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- High Performance Python, Ian Ozsvald and Micha Gorelick, 2020.
- See: Chapter 9: The multiprocessing Module
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Multiprocessing: The Complete Guide
- Python Multiprocessing Pool: The Complete Guide
- Python ProcessPoolExecutor: The Complete Guide
APIs
References
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Takeaways
You now know how to use a mutex lock with processes in Python.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Viktor Ritsvall on Unsplash
The explications are pretty easy but the links overwhelm a little, try to reduce the links to external tutorials and explain as clear as you can without bothering the readers with all that additional links.
Thank you for the tip!