You can benchmark NumPy array copy functions and discover the fastest approaches to use in different circumstances.
Generally, numpy.copy() is about as fast as ndarray.copy() and numpy.array() when creating a copy of a NumPy array with the same shape and content.
It is possible that numpy.copyto() and slicing have similar or slightly worse performance.
In this tutorial, you will discover how to benchmark and discover the fastest ways to copy NumPy arrays.
Let’s get started.
Need to Copy NumPy Array Fast
Copying NumPy arrays a common activity in our Python programs.
There are many ways to copy a NumPy array, although there is little guidance about which approach is fastest.
We will focus our attention on creating a new array that has the same shape and content as the source array.
Although there are many ways that this can be achieved, perhaps the more common among them include the following:
- numpy.copy()
- ndarray.copy()
- numpy.array()
- numpy.copyto()
- 5. dst[:] = src[:]
Did I miss your preferred approach to copy NumPy arrays?
Let me know in the comments and I will add it to the benchmarking.
In this case, we will leave out creating a new array with the same shape as a source array although initialized with different values, such as numpy.zeros_like(), numpy.ones_like(), etc.
We will also leave out approaches that create a view of a source array, instead of a copy, e.g. dst = src[:].
Given that we need to create copies of arrays all the time in our NumPy programs, which approach or approach is fast?
Specific requirements aside, what function should we use when copying arrays?
Run loops using all CPUs, download your FREE book to learn how.
Benchmark NumPy Array Copying
We can explore the question of how fast the different array copy methods are using benchmarking.
In this case, we will use an approach to copy an array of a modest fixed size, then repeat this process many times to give an estimated time. We can then compare the times to see the relative performance of the approaches tested.
You can use this approach to benchmark your own favorite NumPy array operations.
If you use or extend the NumPy benchmarking approach used in this tutorial, let me know in the comments below. I’d love to see what you come up with.
We could use the time.perf_counter() function directly and develop a helper function to perform the benchmarking and report results.
You can learn more about benchmarking with the time.perf_counter() function in the tutorial:
Instead, in this case, we will use the timeit API, specifically the timeit.timeit() function and specify the string of array copy code to run and a fixed number of times to run it.
We will also provide the globals argument for any constants defined in our benchmark code, such as array size or shape.
For example:
|
1 2 3 4 |
... # benchmark a thing result = timeit.timeit('...', globals=globals(), number=N) print(f'approach {result:.3f} seconds') |
You can learn more about benchmarking with the timeit.timeit() function in the tutorial:
The number of runs in each benchmark was tuned to ensure that each snippet was executed in more than one second and less than about 10 seconds.
Let’s get started.
Fastest Way to Copy 1D NumPy Array
We can explore the fastest way to create a copy of a modestly sized initialized one-dimensional NumPy array.
In this case, we will define a fixed size 1d array with one million elements (1,000,000) initialized to zero. Each approach will be used to create a copy of this 8,000 times.
The approaches we will compare include the most common NumPy functions for copying a new 1d array of a given size.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
# SuperFastPython.com # benchmark copying a 1d numpy array import numpy import timeit # array to copy A = numpy.zeros(1000000) # number of times to run each snippet N = 8000 # numpy.copy() result = timeit.timeit('numpy.copy(A)', globals=globals(), number=N) print(f'numpy.copy() {result:.3f} seconds') # ndarray.copy() result = timeit.timeit('A.copy()', globals=globals(), number=N) print(f'ndarray.copy() {result:.3f} seconds') # numpy.array() result = timeit.timeit('numpy.array(A)', globals=globals(), number=N) print(f'numpy.array() {result:.3f} seconds') # numpy.empty();numpy.copyto() result = timeit.timeit('B=numpy.empty(A.shape);numpy.copyto(B,A)', globals=globals(), number=N) print(f'numpy.empty();numpy.copyto() {result:.3f} seconds') # numpy.empty();B[:]=A[:] result = timeit.timeit('B=numpy.empty(A.shape);B[:]=A[:]', globals=globals(), number=N) print(f'numpy.empty(A.shape);B[:]=A[:] {result:.3f} seconds') |
Running the example benchmarks each approach and reports the sum execution time.
|
1 2 3 4 5 |
numpy.copy() 10.933 seconds ndarray.copy() 10.481 seconds numpy.array() 10.531 seconds numpy.empty();numpy.copyto() 11.027 seconds numpy.empty(A.shape);B[:]=A[:] 11.044 seconds |
We can restructure the output into a table for comparison.
|
1 2 3 4 5 6 7 |
Approach | Time (sec) ---------------|------------ numpy.copy() | 10.933 ndarray.copy() | 10.481 numpy.array() | 10.531 numpy.copyto() | 11.027 B[:]=A[:] | 11.044 |
We can see that all tested approaches have a similar benchmark time.
In this case, it is possible that the ndarray.copy() and numpy.array() approaches are slightly faster than numpy.copy() and numpy.copyto().
Given the closeness of the results, the effect may be a statistical fluke and all approaches may have a near identical overall execution time.
Re-running the same benchmark gives a different pattern of results, supporting this suggestion, although numpy.array() remains consistently fast, even with subsequent additional runs.
It may be that the numpy.array() approach to copying an array is doing so in a slightly more efficient manner.
|
1 2 3 4 5 |
numpy.copy() 10.326 seconds ndarray.copy() 10.508 seconds numpy.array() 10.315 seconds numpy.empty();numpy.copyto() 10.652 seconds numpy.empty(A.shape);B[:]=A[:] 10.948 seconds |
Free Python Benchmarking Course
Get FREE access to my 7-day email course on Python Benchmarking.
Discover benchmarking with the time.perf_counter() function, how to develop a benchmarking helper function and context manager and how to use the timeit API and command line.
Fastest Way to Copy 2D NumPy Array
We can explore the fastest way to copy a modestly sized initialized two-dimensional NumPy array.
Each array will have the size (1000,1000) and we will run each method 10,000 times.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
# SuperFastPython.com # benchmark copying a 2d numpy array import numpy import timeit # array to copy A = numpy.zeros((1000,1000)) # number of times to run each snippet N = 10000 # numpy.copy() result = timeit.timeit('numpy.copy(A)', globals=globals(), number=N) print(f'numpy.copy() {result:.3f} seconds') # ndarray.copy() result = timeit.timeit('A.copy()', globals=globals(), number=N) print(f'ndarray.copy() {result:.3f} seconds') # numpy.array() result = timeit.timeit('numpy.array(A)', globals=globals(), number=N) print(f'numpy.array() {result:.3f} seconds') # numpy.empty();numpy.copyto() result = timeit.timeit('B=numpy.empty(A.shape);numpy.copyto(B,A)', globals=globals(), number=N) print(f'numpy.empty();numpy.copyto() {result:.3f} seconds') # numpy.empty();B[:]=A[:] result = timeit.timeit('B=numpy.empty(A.shape);B[:]=A[:]', globals=globals(), number=N) print(f'numpy.empty(A.shape);B[:]=A[:] {result:.3f} seconds') |
Running the example benchmarks each approach and reports the sum execution time.
|
1 2 3 4 5 |
numpy.copy() 12.689 seconds ndarray.copy() 12.568 seconds numpy.array() 12.830 seconds numpy.empty();numpy.copyto() 13.323 seconds numpy.empty(A.shape);B[:]=A[:] 13.402 seconds |
We can restructure the output into a table for comparison.
|
1 2 3 4 5 6 7 |
Approach | Time (sec) ---------------|------------ numpy.copy() | 12.689 ndarray.copy() | 12.568 numpy.array() | 12.830 numpy.copyto() | 13.323 B[:]=A[:] | 13.402 |
Again, we don’t see much difference in the results.
All approaches have a similar execution time with perhaps numpy.copyto() taking slightly longer.
Like the one-dimensional example above, we expect there to be no significant difference between the approaches.
Repeating the benchmark shows a different pattern of results, confirming this suspicion.
It does appear that the slicing method, B[:]=A[:], may have slightly worse performance in each set of benchmark runs.
|
1 2 3 4 5 |
numpy.copy() 12.785 seconds ndarray.copy() 12.620 seconds numpy.array() 12.406 seconds numpy.empty();numpy.copyto() 12.970 seconds numpy.empty(A.shape);B[:]=A[:] 13.388 seconds |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Further Reading
This section provides additional resources that you may find helpful.
Books
- Python Benchmarking, Jason Brownlee (my book!)
Also, the following Python books have chapters on benchmarking that may be helpful:
- Python Cookbook, 2013. (sections 9.1, 9.10, 9.22, 13.13, and 14.13)
- High Performance Python, 2020. (chapter 2)
Guides
- 4 Ways to Benchmark Python Code
- 5 Ways to Measure Execution Time in Python
- Python Benchmark Comparison Metrics
Benchmarking APIs
- time — Time access and conversions
- timeit — Measure execution time of small code snippets
- The Python Profilers
References
Takeaways
You now know how to benchmark and discover the fastest ways to copy NumPy arrays.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any additional tips?
I’d love to hear about them!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Łukasz Nieścioruk on Unsplash
Do you have any questions?