Threading Mutex Lock in Python

February 24, 2022 Python Threading

You can use a mutual exclusion (mutex) lock in Python via the threading.Lock class.

In this tutorial you will discover how to use the threading.Lock class to create and use mutex locks.

Let's get started.

Need for a Mutual Exclusion Lock

A thread is a thread of execution in a computer program.

Every Python program has at least one thread of execution called the main thread. Both processes and threads are created and managed by the underlying operating system.

Sometimes we may need to create additional threads in our program in order to execute code concurrently.

We can create and manage new Python threads via the threading module and the threading.Thread class.

When writing concurrent programs we may need to share data or resources between threads, which typically must be protected with a mutual exclusion lock.

What is a mutual exclusion lock and how can we use it in Python?

What is a Mutual Exclusion Lock

A mutual exclusion lock or mutex lock is a synchronization primitive intended to prevent a race condition.

A race condition is a concurrency failure case when two threads run the same code and access or update the same resource (e.g. data variables, stream, etc.) leaving the resource in an unknown and inconsistent state.

Race conditions often result in unexpected behavior of a program and/or corrupt data.

These sensitive parts of code that can be executed by multiple threads concurrently and may result in race conditions are called critical sections. A critical section may refer to a single block of code, but it also refers to multiple accesses to the same data variable or resource from multiple functions.

The most commonly used mechanism for ensuring mutual exclusion is a mutual exclusion lock or mutex, or simply lock. A mutex is a special type of object that has support in the underlying hardware. The basic idea is that each critical section is protected by a lock.
-- Page 53, An Introduction to Parallel Programming, 2020.

A mutex lock can be used to ensure that only one thread at a time executes a critical section of code at a time, while all other threads trying to execute the same code must wait until the currently executing thread is finished with the critical section and releases the lock.

While one thread “owns” the lock—that is, has returned from a call to the lock function but hasn't yet called the unlock function—any other thread attempting to execute the code in the critical section will wait in its call to the lock function.
-- Page 53, An Introduction to Parallel Programming, 2020.

Each thread must attempt to acquire the lock at the beginning of the critical section. If the lock has not been obtained, then a thread will acquire it and other threads must wait until the thread that acquired the lock releases it.

Before a thread can execute the code in the critical section, it must "obtain" the mutex by calling a mutex function, and when it's done executing the code in the critical section, it should “relinquish” the mutex by calling an unlock function.
-- Page 53, An Introduction to Parallel Programming, 2020.

If the lock has not been acquired, we might refer to it as being in the "unlocked" state. Whereas if the lock has been acquired, we might refer to it as being in the "locked" state.

Unlocked: The lock has not been acquired and can be acquired by the next thread that makes an attempt.
Locked: The lock has been acquired by one thread and any thread that makes an attempt to acquire it must wait until it is released.

Locks are created in the unlocked state.

Now that we know what a mutex lock is, let's take a look at how we can use it in Python.

How to Use the threading.Lock Class

Python provides a mutual exclusion lock via the threading.Lock class.

An instance of the lock can be created and then acquired by threads before accessing a critical section, and released after the critical section.

For example:

...
# create a lock
lock = Lock()
# acquire the lock
lock.acquire()
# ...
# release the lock
lock.release()

Only one thread can have the lock at any time. If a thread does not release an acquired lock, it cannot be acquired again.

The thread attempting to acquire the lock will block until the lock is acquired, such as if another thread currently holds the lock then releases it.

A primitive lock is in one of two states, “locked” or “unlocked”. It is created in the unlocked state.
-- LOCK OBJECTS, THREADING - THREAD-BASED PARALLELISM.

We can attempt to acquire the lock without blocking by setting the "blocking" argument to False. If the lock cannot be acquired, a value of False is returned.

...
# acquire the lock without blocking
lock.acquire(blocking=false)

We can also attempt to acquire the lock with a timeout, that will wait the set number of seconds to acquire the lock before giving up. If the lock cannot be acquired, a value of False is returned.

...
# acquire the lock with a timeout
lock.acquire(timeout=10)

We can also use the lock via the context manager protocol via the with statement, allowing the critical section to be a block within the usage of the lock and for the lock to be released automatically once the block has completed.

For example:

...
# create a lock
lock = Lock()
# acquire the lock
with lock:
    # ...

This is the preferred usage as it makes it clear where the protected code begins and ends, and ensures that the lock is always released, even if there is an exception or error within the critical section.

We can also check if the lock is currently acquired by a thread via the locked() function.

...
# check if a lock is currently acquired
if lock.locked():
    # ...

Now that we know how to use the threading.Lock class, let's look at a worked example.

Example of Using a Mutual Exclusion Lock

We can develop an example to demonstrate how to use the mutex lock.

First, we can define a target task function that takes a lock as an argument and uses the lock to protect a critical section. In this case, the critical section involves reporting a message and blocking for a fraction of a second.

# work function
def task(lock, identifier, value):
    # acquire the lock
    with lock:
        print(f'>thread {identifier} got the lock, sleeping for {value}')
        sleep(value)

We can then create one instance of the threading.Lock shared among the threads, and pass it to each thread that we intend to execute the target task function.

...
# create a shared lock
lock = Lock()
# start a few threads that attempt to execute the same critical section
for i in range(10):
    # start a thread
    Thread(target=task, args=(lock, i, random())).start()

Tying this together, the complete example of using a lock is listed below.

# SuperFastPython.com
# example of a mutual exclusion (mutex) lock
from time import sleep
from random import random
from threading import Thread
from threading import Lock

# work function
def task(lock, identifier, value):
    # acquire the lock
    with lock:
        print(f'>thread {identifier} got the lock, sleeping for {value}')
        sleep(value)

# create a shared lock
lock = Lock()
# start a few threads that attempt to execute the same critical section
for i in range(10):
    # start a thread
    Thread(target=task, args=(lock, i, random())).start()
# wait for all threads to finish...

Running the example starts ten threads that all execute a custom target function.

Each thread attempts to acquire the lock, and once they do, they report a message including their id and how long they will sleep before releasing the lock.

Your specific results may vary given the use of random numbers.

>thread 0 got the lock, sleeping for 0.8859193801237439
>thread 1 got the lock, sleeping for 0.02868415293867832
>thread 2 got the lock, sleeping for 0.04469783674319383
>thread 3 got the lock, sleeping for 0.20456291750962474
>thread 4 got the lock, sleeping for 0.3689208984892195
>thread 5 got the lock, sleeping for 0.21105944750222927
>thread 6 got the lock, sleeping for 0.052093068060339864
>thread 7 got the lock, sleeping for 0.871251970586552
>thread 8 got the lock, sleeping for 0.932718580790764
>thread 9 got the lock, sleeping for 0.9514093969897454

Common Questions

This section lists common questions about the threading.Lock mutex in Python.

Do We Need To Protect Critical Sections In Light of the GIL?

Yes.

The current version of the Python interpreter implements a global interpreter lock (GIL) which ensures that only one thread can execute at a time within a Python process.

After learning about the global interpreter lock (GIL), many new Python programmers assume they can forgo using mutual-exclusion locks (also called mutexes) in their code altogether. If the GIL is already preventing Python threads from running on multiple CPU cores in parallel, it must also act as a lock for a program's data structures, right? Some testing on types like lists and dictionaries may even show that this assumption appears to hold. But beware, this is not truly the case.
-- Page 235, Effective Python, 2019.

Nevertheless, it is possible for the operating system to context switch between threads while a thread is in the middle of updating a variable. This means another thread attempting to access or update the same variable may access a corrupt version of the data (race condition).

Critical sections must be protected in Python, even in light of the GIL.

Why Not Use a Boolean Variable?

Reading and writing a boolean variable is an atomic operation in the Python interpreter, and only one thread can execute at one time given the global interpreter lock in the Python interpreter. Therefore, the same effect could be achieved using a single boolean variable.

I believe this is a true statement at the time of writing, but only with the current version of the Python interpreter.

This behavior might change in later versions of Python and may be different with third-party Python interpreters that do not implement a global interpreter lock.

I recommend implementing correct synchronization primitives like the mutex to protect critical sections of code regardless of the GIL.

What If We Don't Release The Lock?

Not releasing the lock is a concurrency bug.

If the lock is not released by a thread that acquired it, then it can not be acquired again and the code protected by the critical section cannot be executed again.

What If We Ignore The Lock?

Ignoring the lock is a concurrency bug.

You may write code so that some threads may obey the lock and some may not. This will likely result in a race condition and defeat the purposes of having a lock in the first place.

The lock only protects the critical section (data, resources, etc.) if it is enforced in all accesses to the critical section.

Is the threading.Lock Reentrant?

No.

A reentrant lock is a lock that if held by one thread can be acquired again by the same thread.

The threading.Thread lock is not reentrant, meaning if a thread acquires the lock and attempts to acquire it again in another part of the code, that it will block, resulting in a deadlock bug (a concurrency failure condition).

Reentrant mutual exclusion locks are available in the threading.RLock class.

Takeaways

You now know how to use a mutex lock in Python.

If you enjoyed this tutorial, you will love my book: Python Threading Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.