Last Updated on September 12, 2022
You can make a counter thread-safe by using a mutex lock via the threading.Lock class.
In this tutorial you will discover how to develop a thread-safe counter.
Let’s get started.
Need a Thread-Safe Counter
A thread is a thread of execution in a computer program.
Every Python program has at least one thread of execution called the main thread. Both processes and threads are created and managed by the underlying operating system.
Sometimes we may need to create additional threads in our program in order to execute code concurrently.
Python provides the ability to create and manage new threads via the threading module and the threading.Thread class.
You can learn more about Python threads in the guide:
In concurrent programming, we often need to update a counter from multiple threads.
This may be for many reasons, such as:
- Counting tasks completed by multiple worker threads.
- Collating values from different data sources.
- Counting the occurrence of events.
Updating a count variable from multiple threads is not thread safe and may result in a race condition.
You can learn more about race conditions here:
How can we create a thread-safe counter?
Run loops using all CPUs, download your FREE book to learn how.
How to Create a Thread-Safe Counter
A counter can be made thread-safe using a mutual exclusion (mutex) lock via the threading.Lock class.
First, an instance of a lock can be created.
For example:
|
1 2 3 |
... # create a lock lock = Lock() |
Then each time the counter variable is accessed or updated, it can be protected by the lock.
This can be achieved by calling the acquire() function before accessing the counter variable and calling release() after work with the counter variable has completed.
For example:
|
1 2 3 4 5 6 7 |
... # acquire the lock protecting the counter lock.acquire() # update the counter counter += 1 # release the lock protecting the counter lock.release() |
A simpler approach to using the threading.Lock is to use the context manager which will release the lock automatically once the block is exited.
For example:
|
1 2 3 4 5 |
... # acquire the lock protecting the counter with lock: # update the counter counter += 1 |
You can learn more about locks here:
Next, let’s explore how to develop a thread-safe counter with some worked examples.
Example of Thread-Unsafe Counter
Before we develop a thread-safe counter, let’s confirm that indeed a simple counter is not thread-safe in Python.
Firstly, we can develop a class that will wrap our counter variable.
We will call the class ThreadUnsafeCounter.
|
1 2 3 |
# thread unsafe counter class class ThreadUnsafeCounter(): # ... |
The class constructor will define an instance variable that will keep track of the count and initialize the value to zero.
|
1 2 3 4 |
# constructor def __init__(self): # initialize counter self._counter = 0 |
We can then add a method to add a value to the counter named increment().
|
1 2 3 |
# increment the counter def increment(self): self._counter += 1 |
Finally, we can add a method that will return the current value of the counter.
|
1 2 3 |
# get the counter value def value(self): return self._counter |
Tying this together, the complete ThreadUnsafeCounter class is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
# thread unsafe counter class class ThreadUnsafeCounter(): # constructor def __init__(self): # initialize counter self._counter = 0 # increment the counter def increment(self): self._counter += 1 # get the counter value def value(self): return self._counter |
Next, we can then configure and start ten threads to increment the counter concurrently.
Firstly, we can define a function that will take the shared counter instance as an argument and then increment the counter 100,000 times.
|
1 2 3 4 5 |
# task executed by threads def task(counter): # increment the counter for _ in range(100000): counter.increment() |
In the main thread, we can first create an instance of our ThreadUnsafeCounter to be shared among all threads.
|
1 2 3 |
... # create the counter counter = ThreadUnsafeCounter() |
We can then create 10 threads that will be configured to call this task() function and pass the counter instance as an argument.
|
1 2 3 |
... # create 10 threads to increment the counter threads = [Thread(target=task, args=(counter,)) for _ in range(10)] |
Next, we can start all ten threads and then wait for them to finish.
|
1 2 3 4 5 6 7 |
... # start all threads for thread in threads: thread.start() # wait for all threads to finish for thread in threads: thread.join() |
Finally, we can report the final value of the counter.
|
1 2 3 |
... # report the value of the counter print(counter.value()) |
With ten threads, each incrementing the counter 100,000 times, we expect the final value of the counter to be 1,000,000.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
# SuperFastPython.com # example of a thread-unsafe counter (with race conditions) from threading import Thread # thread unsafe counter class class ThreadUnsafeCounter(): # constructor def __init__(self): # initialize counter self._counter = 0 # increment the counter def increment(self): self._counter += 1 # get the counter value def value(self): return self._counter # task executed by threads def task(counter): # increment the counter for _ in range(100000): counter.increment() # create the counter counter = ThreadUnsafeCounter() # create 10 threads to increment the counter threads = [Thread(target=task, args=(counter,)) for _ in range(10)] # start all threads for thread in threads: thread.start() # wait for all threads to finish for thread in threads: thread.join() # report the value of the counter print(counter.value()) |
Running the example first creates an instance of our ThreadUnsafeCounter class.
Next, we create and configure ten threads that all attempt to increment the counter 100,000 times. The threads are then started and the main thread blocks until all new threads are finished.
Finally, the value of the counter is reported.
In this case, we can see that the counter does not have the expected value. Instead, it is about 280,000 short of the value of 1,000,000 we expected.
|
1 |
720138 |
In fact, every time the program is run, you will see a different value.
|
1 |
965689 |
This is because the _counter variable is updated in a thread unsafe manner.
There is a race condition.
You can learn more about this type of race condition here:
Next, let’s update our class to be thread-safe.
Free Python Threading Course
Download your FREE threading PDF cheat sheet and get BONUS access to my free 7-day crash course on the threading API.
Discover how to use the Python threading module including how to create and start new threads and how to use a mutex locks and semaphores
Example of Thread-Safe Counter
The ThreadUnsafeCounter class that we developed in the previous section can be updated to be thread-safe.
That is we can add a threading.Lock instance to the class to protect the counter variable.
Firstly, we can initialize the lock as an instance variable in the constructor of the class.
|
1 2 3 4 5 6 |
# constructor def __init__(self): # initialize counter self._counter = 0 # initialize lock self._lock = Lock() |
Next, when updating the counter in the increment() method, we can first acquire the lock then change the value using the context manager.
|
1 2 3 4 |
# increment the counter def increment(self): with self._lock: self._counter += 1 |
Finally, when getting the current value of the from the value() method, we can again protect the counter variable by first acquiring the lock.
|
1 2 3 4 |
# get the counter value def value(self): with self._lock: return self._counter |
We can rename the class to ThreadSafeCounter. The updated thread-safe version of the class is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
# thread safe counter class class ThreadSafeCounter(): # constructor def __init__(self): # initialize counter self._counter = 0 # initialize lock self._lock = Lock() # increment the counter def increment(self): with self._lock: self._counter += 1 # get the counter value def value(self): with self._lock: return self._counter |
And that’s it.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
# SuperFastPython.com # example of a thread-safe counter from threading import Thread from threading import Lock # thread safe counter class class ThreadSafeCounter(): # constructor def __init__(self): # initialize counter self._counter = 0 # initialize lock self._lock = Lock() # increment the counter def increment(self): with self._lock: self._counter += 1 # get the counter value def value(self): with self._lock: return self._counter # task executed by threads def task(counter): # increment the counter for _ in range(100000): counter.increment() # create the counter counter = ThreadSafeCounter() # create 10 threads to increment the counter threads = [Thread(target=task, args=(counter,)) for _ in range(10)] # start all threads for thread in threads: thread.start() # wait for all threads to finish for thread in threads: thread.join() # report the value of the counter print(counter.value()) |
Running the example first creates the shared counter instance before.
Then creates and starts ten threads, each attempting to increment the counter 100,000 times each concurrently.
After the threads finish, the main thread reports the current value.
In this case, we can see that the counter variable was indeed protected and that no race condition was present.
The expected value of 1,000,000 was reported, and is reported every time the example is run.
|
1 |
1000000 |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Further Reading
This section provides additional resources that you may find helpful.
Python Threading Books
- Python Threading Jump-Start, Jason Brownlee (my book!)
- Threading API Interview Questions
- Threading Module API Cheat Sheet
I also recommend specific chapters in the following books:
- Python Cookbook, David Beazley and Brian Jones, 2013.
- See: Chapter 12: Concurrency
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Threading: The Complete Guide
- Python ThreadPoolExecutor: The Complete Guide
- Python ThreadPool: The Complete Guide
APIs
References
Takeaways
You now know how to make a counter thread safe.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Harley-Davidson on Unsplash
Do you have any questions?