Last Updated on September 12, 2022

You can use reentrant locks in Python via the threading.RLock class.

In this tutorial you will discover how to use reentrant mutex locks in Python.

Let’s get started.

Need for a Reentrant Lock

A thread is a thread of execution in a computer program.

Every Python program has at least one thread of execution called the main thread. Both processes and threads are created and managed by the underlying operating system.

Sometimes we may need to create additional threads in our Python program in order to execute code concurrently.

Python provides the ability to create and manage new threads via the threading module and the threading.Thread class.

When writing concurrent programs we may need to share data or resources between threads, which typically must be protected with a lock.

A standard mutex lock does not allow a thread to acquire the lock more than once. This means that code in one critical section cannot call or execute code with another critical section protected by the same lock. Instead, a different type of mutex lock is required called a reentrant lock.

What is a reentrant lock and how can we use it in Python?

First, let’s take a quick look at a standard mutex lock as a point of reference.

What is a Mutex Lock

A mutual exclusion lock or mutex lock is a synchronization primitive intended to prevent a race condition.

A race condition is a concurrency failure case when two threads run the same code and access or update the same resource (e.g. data variables, stream, etc.) leaving the resource in an unknown and inconsistent state.

Race conditions often result in unexpected behavior of a program and/or corrupt data.

These sensitive parts of code that can be executed by multiple threads concurrently and may result in race conditions are called critical sections. A critical section may refer to a single block of code, but it also refers to multiple accesses to the same data variable or resource from multiple functions.

A mutex lock can be used to ensure that only one thread at a time executes a critical section of code at a time, while all other threads trying to execute the same code must wait until the currently executing thread is finished with the critical section.

Each thread must attempt to acquire the lock at the beginning of the critical section. If the lock has not been obtained, then a thread will acquire it and other threads must wait until the thread that acquired the lock releases it.

Now that we know what a mutex lock is, let’s take a closer look at a reentrant mutex lock.

What is a Reentrant Lock

A reentrant mutual exclusion lock, “reentrant mutex” or “reentrant lock” for short, is like a mutex lock except it allows a thread to acquire the lock more than once.

A reentrant lock is a synchronization primitive that may be acquired multiple times by the same thread. […] In the locked state, some thread owns the lock; in the unlocked state, no thread owns it.

— RLock Objects, threading – Thread-based parallelism.

A thread may need to acquire the same lock more than once for many reasons.

We can imagine critical sections spread across a number of functions, each protected by the same lock. A thread may call across these functions in the course of normal execution and may call into one critical section from another critical section.

A limitation of a (non-reentrant) mutex lock is that if a thread has acquired the lock that it cannot acquire it again. In fact, this situation will result in a deadlock as it will wait forever for the lock to be released so that it can be acquired, but it holds the lock and will not release it.

A reentrant lock will allow a thread to acquire the same lock again if it has already acquired it. This allows the thread to execute critical sections from within critical sections, as long as they are protected by the same reentrant lock.

Each time a thread acquires the lock it must also release it, meaning that there are recursive levels of acquire and release for the owning thread. As such, this type of lock is sometimes called a “recursive mutex lock“.

Now that we are familiar with the reentrant lock, let’s take a closer look at the difference between a lock and a reentrant lock in Python.

Difference Between Lock and RLock

Python provides a mutex lock via the threading.Lock class and a reentrant lock via the threading.RLock class.

• threading.Lock: Python mutex lock.
• threading.RLock: Python reentrant mutex lock.

A threading.Lock can only be acquired once, and once acquired it cannot be acquired again by the same thread or any other thread until it has been released.

A threading.RLock can be acquired more than once by the same thread, although once acquired by a thread it cannot be acquired by a different thread until it is been released.

Importantly, each time the threading.RLock is acquired by the same thread it must be released the same number of times until it is available to be acquired again by a different thread. This means that the number of calls to acquire() must have the same number of calls to release() for the RLock to return to the unlocked state.

Now that we know about the reentrant lock, let’s take a look at how we can use it in Python.

How to Use the Reentrant Lock

Python provides a reentrant lock via the threading.RLock class.

An instance of the RLock can be created and then acquired by threads before accessing a critical section, and released after the critical section.

For example:

The thread attempting to acquire the lock will block until the lock is acquired, such as if another thread currently holds the lock (once or more than once) then releases it.

We can attempt to acquire the lock without blocking by setting the “blocking” argument to False. If the lock cannot be acquired, a value of False is returned.

We can also attempt to acquire the lock with a timeout, that will wait the set number of seconds to acquire the lock before giving up. If the lock cannot be acquired, a value of False is returned.

We can also use the reentrant lock via the context manager protocol via the with statement, allowing the critical section to be a block within the usage of the lock and for the lock to be released once the block is exited.

For example:

Now that we know how to use the threading.RLock class, let’s look at a worked example.

Example of Using a Reentrant Lock

We can develop an example to demonstrate how to use the threading.RLock.

First, we can define a function to report that a thread is done that protects the reporting process with a lock.

Next, we can define a task function that reports a message, blocks for a moment, then calls the reporting function. All of the work is protected with the lock.

Given that the target task function is protected with a lock and calls the reporting function that is also protected by the same lock, we can use a reentrant lock so that if a thread acquires the lock in task(), it will be able to re-enter the lock in the report() function.

Finally, we can create the reentrant lock, then startup ten threads that execute the target task function.

Tying this together, the complete example of demonstrating a reentrant lock is listed below.

Running the example starts up ten threads that execute the target task function.

Only one thread can acquire the lock at a time, and then once acquired, blocks and then reenters the same lock again to report the done message.

If a non-reentrant lock, e.g. a threading.Lock was used instead, then the thread would block forever waiting for the lock to become available, which it can’t because the thread already holds the lock.

Further Reading

This section provides additional resources that you may find helpful.

Python Threading Books

• Python Threading Jump-Start, Jason Brownlee (my book!)
• Threading API Interview Questions
• Threading Module API Cheat Sheet

I also recommend specific chapters in the following books:

• Python Cookbook, David Beazley and Brian Jones, 2013.
  • See: Chapter 12: Concurrency
• Effective Python, Brett Slatkin, 2019.
  • See: Chapter 7: Concurrency and Parallelism
• Python in a Nutshell, Alex Martelli, et al., 2017.
  • See: Chapter: 14: Threads and Processes

Guides

• Python Threading: The Complete Guide
• Python ThreadPoolExecutor: The Complete Guide
• Python ThreadPool: The Complete Guide

APIs

• threading - Thread-based parallelism
• queue — A synchronized queue class

References

• Thread (computing), Wikipedia.
• Process (computing), Wikipedia.

Takeaways

You now know how to use the threading.RLock reentrant mutex lock in Python.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by Harley-Davidson on Unsplash