You can benchmark snippets of Python code using the timeit.timeit() function.
In this tutorial, you will discover how to benchmark Python code using the timeit.timeit() function.
Let’s get started.
What is timeit
The timeit module is provided in the Python standard library.
It provides an easy way to benchmark single statements and snippets of Python code.
This module provides a simple way to time small bits of Python code. It has both a Command-Line Interface as well as a callable one. It avoids a number of common traps for measuring execution times.
— timeit — Measure execution time of small code snippets
The timeit module provides two interfaces for benchmarking.
- API interface.
- Command-line interface.
The first is an API that can be used via the timeit.Timer object or timeit.timeit() and timeit.repeat() module functions.
The second is a command line interface.
Both are intended to benchmark single Python statements, although multiple lines and multiple statements can be benchmarked using the module.
Run loops using all CPUs, download your FREE book to learn how.
How to Benchmark With timeit.timeit()
The timeit.timeit() benchmarks Python code and reports the duration in seconds.
Create a Timer instance with the given statement, setup code and timer function and run its timeit() method with number executions.
— timeit — Measure execution time of small code snippets
The timeit.timeit() takes the Python statement to be benchmarked as a string.
For example:
|
1 2 3 |
... # benchmark a python statement result = timeit.timeit('[i*i for i in range(1000)]') |
Setup Argument
Any Python code required to execute the benchmark code can be provided as a string to the “setup” argument.
This might include defining a variable.
The setup code is only executed once prior to the benchmark.
For example:
|
1 2 3 |
... # benchmark a python statement with setup code result = timeit.timeit('[i*i for i in range(total)]', setup='total=10000') |
It might include importing the main module so that required functions are imported.
For example:
|
1 2 3 |
... # benchmark a python statement with import in setup result = timeit.timeit('task()', setup='from __main__ import task') |
Globals Argument
Alternatively, if we have defined code in our program that is required to execute the benchmark code, we can specify the “globals” argument to provide the entire namespace.
We can specify locals() or globals() which will include a namespace from our current program.
For example:
|
1 2 3 |
... # benchmark a python statement with a namespace result = timeit.timeit('task()', globals=globals()) |
Number Argument
Finally, we can specify the number of repetitions of the benchmark code via the “number” argument.
By default, this is set to one million, e.g. 1,000,000, although can be set to a smaller number if the benchmark code takes a long time to execute.
For example:
|
1 2 3 |
... # benchmark a python statement with a smaller number result = timeit.timeit('[i*i for i in range(1000)]', number=100) |
The “number” argument should be set so that the overall duration is at least 0.2 or 0.5 seconds, perhaps even more than one second.
Now that we know how to use the timeit.timeit() function, let’s look at some worked examples.
Example of Standalone Code with timeit.timeit()
In this example, we will benchmark standalone Python code.
This is code that does not depend on any other variables or functions.
We will benchmark two different ways to calculate a list of 1,000 squared numbers.
The first method will multiply each number by itself.
|
1 |
[i*i for i in range(1000)] |
The second method will use the exponent operator (**) and raise each number to the power of 2.
|
1 |
[i**2 for i in range(1000)] |
These snippets execute very quickly, therefore we will repeat each snippet 100,000 times when calling the timeit.timeit() function.
|
1 2 3 4 5 6 |
... # benchmark the i*i method time_method1 = timeit('[i*i for i in range(1000)]', number=100000) ... # benchmark the i**2 method time_method2 = timeit('[i**2 for i in range(1000)]', number=100000) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 |
# SuperFastPython.com # example of benchmarking a list of numbers with timeit.timeit() from timeit import timeit # benchmark the i*i method time_method1 = timeit('[i*i for i in range(1000)]', number=100000) # report the duration print(f'i*i: {time_method1} seconds') # benchmark the i**2 method time_method2 = timeit('[i**2 for i in range(1000)]', number=100000) # report the duration print(f'i**2: {time_method2} seconds') |
Running the example first executes the i*i 100,000 times and reports the overall duration.
It then runs the i**2 100,000 times and reports the overall duration.
In this case, we can see that the i*i took about 4.180 seconds, whereas the i** took about 25.427 seconds.
This suggests that the i*i method is faster.
This highlights how we can benchmark standalone code using timeit.timeit().
|
1 2 |
i*i: 4.180790501181036 seconds i**2: 5.427739103091881 seconds |
Next, let’s look at including a benchmark that requires setup.
Free Python Benchmarking Course
Get FREE access to my 7-day email course on Python Benchmarking.
Discover benchmarking with the time.perf_counter() function, how to develop a benchmarking helper function and context manager and how to use the timeit API and command line.
Example of Code With Setup using timeit.timeit()
We can explore a timeit.timeit() benchmark that requires setup code.
In this case, we will add one more method to the benchmark suite for squaring a list of 1,000 numbers.
The new method will use the math.pow() function, which takes the number and the exponent as arguments.
|
1 |
[pow(i,2) for i in range(1000)] |
This requires that the pow() function be imported from the math module.
We can achieve this by including the important statement via the “setup” argument to the timeit.timeit() function.
|
1 2 3 |
... # benchmark the math.pow() method time_method3 = timeit('[pow(i,2) for i in range(1000)]', setup='from math import pow', number=100000) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
# SuperFastPython.com # example of benchmarking a list of numbers with timeit.timeit() from timeit import timeit # benchmark the i*i method time_method1 = timeit('[i*i for i in range(1000)]', number=100000) # report the duration print(f'i*i: {time_method1} seconds') # benchmark the i**2 method time_method2 = timeit('[i**2 for i in range(1000)]', number=100000) # report the duration print(f'i**2: {time_method2} seconds') # benchmark the math.pow() method time_method3 = timeit('[pow(i,2) for i in range(1000)]', setup='from math import pow', number=100000) # report the duration print(f'math.pow(): {time_method3} seconds') |
In this case, we benchmark the i*i and i**2 methods as before, achieving similar but not identical results, specifically, 4.180 seconds and 5.515 seconds respectively.
Then the new math.pow() method is benchmarked. In this case, it takes about 9.239 seconds to complete overall.
We can see that it is more than twice as slow as the i*i method.
This highlights how we can benchmark a function that requires importing using timeit.timeit().
|
1 2 3 |
i*i: 4.180096419993788 seconds i**2: 5.515898708021268 seconds math.pow(): 9.239141091005877 seconds |
Next, we will explore how we can benchmark a custom function.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of a Custom Function with timeit.timeit()
We can explore a timeit.timeit() benchmark of a custom function.
In this case, we will define a new function that squares a given number.
|
1 2 3 |
# define a custom function for squaring numbers def square(value): return value * value |
We will then create a list of 1,000 squared numbers that use our custom function.
|
1 |
[square(i) for i in range(1000)] |
This requires that the square() function be made available to the benchmark snippet.
We can achieve this via the “globals” argument and provide the current namespace via the globals() built-in function, which will include the definition of our custom square() function.
|
1 2 3 |
... # benchmark the square() method time_method4 = timeit('[square(i) for i in range(1000)]', globals=globals(), number=100000) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
# SuperFastPython.com # example of benchmarking a list of numbers with timeit.timeit() from timeit import timeit # define a custom function for squaring numbers def square(value): return value * value # benchmark the i*i method time_method1 = timeit('[i*i for i in range(1000)]', number=100000) # report the duration print(f'i*i: {time_method1} seconds') # benchmark the i**2 method time_method2 = timeit('[i**2 for i in range(1000)]', number=100000) # report the duration print(f'i**2: {time_method2} seconds') # benchmark the math.pow() method time_method3 = timeit('[pow(i,2) for i in range(1000)]', setup='from math import pow', number=100000) # report the duration print(f'math.pow(): {time_method3} seconds') # benchmark the square() method time_method4 = timeit('[square(i) for i in range(1000)]', globals=globals(), number=100000) # report the duration print(f'square(): {time_method4} seconds') |
In this case, we benchmark the i*i, i**2, and math.pow() methods as before, achieving similar but not identical results, specifically, 4.159 seconds, 5.522 seconds, and 9.287 seconds respectively.
Then the new custom square() function approach is benchmarked. In this case, it completes in about 7.319 seconds.
This is slower than the simple i*i method, although it is faster than the math.pow() function.
This highlights how we can benchmark a custom function using timeit.timeit().
|
1 2 3 4 |
i*i: 4.1599748940061545 seconds i**2: 5.5225951920001535 seconds math.pow(): 9.287094817002071 seconds square(): 7.319860859002802 seconds |
Further Reading
This section provides additional resources that you may find helpful.
Books
- Python Benchmarking, Jason Brownlee (my book!)
Also, the following Python books have chapters on benchmarking that may be helpful:
- Python Cookbook, 2013. (sections 9.1, 9.10, 9.22, 13.13, and 14.13)
- High Performance Python, 2020. (chapter 2)
Guides
- 4 Ways to Benchmark Python Code
- 5 Ways to Measure Execution Time in Python
- Python Benchmark Comparison Metrics
Benchmarking APIs
- time — Time access and conversions
- timeit — Measure execution time of small code snippets
- The Python Profilers
References
Takeaways
You now know how to benchmark Python code using the timeit.timeit() function.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Enrico Carnemolla on Unsplash
Do you have any questions?