Benchmark Context Manager in Python
You can develop a custom context manager to automatically benchmark code in Python.
In this tutorial, you will discover how to benchmark Python code using a context manager.
Let's get started.
Benchmark Python Code With time.perf_counter()
We can benchmark Python code using the time module.
The time.perf_counter() function will return a value from a high-performance counter.
Return the value (in fractional seconds) of a performance counter, i.e. a clock with the highest available resolution to measure a short duration.
-- time — Time access and conversions
The difference between the two calls to the time.perf_counter() function can provide a high-precision estimate of the execution time of a block of code.
Unlike the time.time() function, the time.perf_counter() function is not subject to updates, such as daylight saving and synchronizing the system clock with a time server. This makes the time.perf_counter() function is a reliable approach to benchmarking Python code.
We can call the time.perf_counter() function at the beginning of the code we wish to benchmark, and again at the end of the code we wish to benchmark.
For example:
...
# record start time
time_start = time.perf_counter()
# call benchmark code
task()
# record end time
time_end = time.perf_counter()The difference between the start and end time is the total duration of the program in seconds.
For example:
...
# calculate the duration
time_duration = time_end - time_start
# report the duration
print(f'Took {time_duration:.3f} seconds')You can learn more about benchmarking Python code with the time.perf_counter() function in the tutorial:
How can we hide all of this code so that we can benchmark with a simple interface?
Can we develop a benchmark context manager?
How to Benchmark Code Using a Context Manager
We can hide manual benchmarking of Python code in a context manager.
A context manager is an object that defines the runtime context to be established when executing a with statement. The context manager handles the entry into, and the exit from, the desired runtime context for the execution of the block of code.
-- With Statement Context Managers
Recall that a context manager is a Python object that has __enter__() and __exit__() methods and is used via the with expression.
You can learn more about context managers here:
Examples include opening a file via the open() built-in function and using a ThreadPoolExecutor.
For example:
# create thread pool
with ThreadPoolExecutor() as tpe:
# ...
We can define a new class that implements a constructor __init__() the __enter__() and __exit__() methods.
The __init__() constructor can take a name argument for the benchmark case and store it in an object attribute.
For example:
# constructor
def __init__(self, name):
# store the name of this benchmark
self.name = name
The __enter__() method can initialize the start time and store it in object attributes. It can then return an instance of the context manager itself, as a good practice.
For example:
...
# enter the context manager
def __enter__(self):
# record the start time
self.time_start = perf_counter()
# return this object
return self
The __exit__() method must take some standard arguments about any exception that occurred while running the context code. It can then record the end time, calculate and store the duration and report the calculated duration along with the name of the benchmark case.
...
# exit the context manager
def __exit__(self, exc_type, exc_value, traceback):
# record the end time
self.time_end = perf_counter()
# calculate the duration
self.duration = self.time_end - self.time_start
# report the duration
print(f'{self.name} took {self.duration:.3f} seconds')
# do not suppress any exception
return False
Tying this together, we can define a Benchmark context manager below.
# define the benchmark context manager
class Benchmark(object):
# constructor
def __init__(self, name):
# store the name of this benchmark
self.name = name
# enter the context manager
def __enter__(self):
# record the start time
self.time_start = perf_counter()
# return this object
return self
# exit the context manager
def __exit__(self, exc_type, exc_value, traceback):
# record the end time
self.time_end = perf_counter()
# calculate the duration
self.duration = self.time_end - self.time_start
# report the duration
print(f'{self.name} took {self.duration:.3f} seconds')
# do not suppress any exception
return False
We can then use it by creating an instance of the Benchmark class within the "with" expression and then list any code within the context we wish to benchmark.
For example:
...
# create the benchmark context
with Benchmark('Task'):
# run the task
...
The code within the context will run as per normal, and once finished, the total execution time will be reported automatically.
This is a clever idea, but I am not the first to think of it. The first time I came across the idea was in a post by Dave Beazley. All credit to him:
- A Context Manager for Timing Benchmarks, Dave Beazley, 2010.
Now that we know how to develop and use a benchmark context manager, let's look at some examples.
Example of Benchmarking a Function Using a Context Manager
We can explore how to use our Benchmark context manager to benchmark the execution time of a custom function.
In this example, we will define a custom function that takes a moment to complete.
The function creates a list of 100 million squared integers in a list comprehension.
For example:
# function to benchmark
def task():
# create a large list
data = [i*i for i in range(100000000)]
We can then call this function within the Benchmark context manager to have the execution time automatically recorded and reported.
For example:
# create the benchmark context
with Benchmark('Task'):
# run the task
task()
Tying this together, the complete example is listed below.
# SuperFastPython.com
# example of a benchmark context manager
from time import perf_counter
# define the benchmark context manager
class Benchmark(object):
# constructor
def __init__(self, name):
# store the name of this benchmark
self.name = name
# enter the context manager
def __enter__(self):
# record the start time
self.time_start = perf_counter()
# return this object
return self
# exit the context manager
def __exit__(self, exc_type, exc_value, traceback):
# record the end time
self.time_end = perf_counter()
# calculate the duration
self.duration = self.time_end - self.time_start
# report the duration
print(f'{self.name} took {self.duration:.3f} seconds')
# do not suppress any exception
return False
# function to benchmark
def task():
# create a large list
data = [i*i for i in range(100000000)]
# protect the entry point
if __name__ == '__main__':
# create the benchmark context
with Benchmark('Task'):
# run the task
task()
Running the example first creates the Benchmark context manager within the with expression and provides the name "Task", which is stored in an object attribute.
The context manager is entered, automatically calling the __enter__() method where the start time is recorded in an object attribute.
The task() function is then called and the list is created.
Finally, the context manager is exited, automatically calling the __exit__() method, recording the end time, calculating the duration, and reporting it to standard out.
In this case, we can see that the task() function took about 6.091 seconds to complete.
This highlights how we can benchmark arbitrary Python code using a custom context manager.
Task took 6.091 secondsTakeaways
You now know how to benchmark Python code using a context manager.
If you enjoyed this tutorial, you will love my book: Python Benchmarking. It covers everything you need to master the topic with hands-on examples and clear explanations.