Benchmark Fastest Way To Copy NumPy Array

November 4, 2023 Python Benchmarking

You can benchmark NumPy array copy functions and discover the fastest approaches to use in different circumstances.

Generally, numpy.copy() is about as fast as ndarray.copy() and numpy.array() when creating a copy of a NumPy array with the same shape and content.

It is possible that numpy.copyto() and slicing have similar or slightly worse performance.

In this tutorial, you will discover how to benchmark and discover the fastest ways to copy NumPy arrays.

Let's get started.

Need to Copy NumPy Array Fast

Copying NumPy arrays a common activity in our Python programs.

There are many ways to copy a NumPy array, although there is little guidance about which approach is fastest.

We will focus our attention on creating a new array that has the same shape and content as the source array.

Although there are many ways that this can be achieved, perhaps the more common among them include the following:

Did I miss your preferred approach to copy NumPy arrays?
Let me know in the comments and I will add it to the benchmarking.

In this case, we will leave out creating a new array with the same shape as a source array although initialized with different values, such as numpy.zeros_like(), numpy.ones_like(), etc.

We will also leave out approaches that create a view of a source array, instead of a copy, e.g. dst = src[:].

Given that we need to create copies of arrays all the time in our NumPy programs, which approach or approach is fast?

Specific requirements aside, what function should we use when copying arrays?

Benchmark NumPy Array Copying

We can explore the question of how fast the different array copy methods are using benchmarking.

In this case, we will use an approach to copy an array of a modest fixed size, then repeat this process many times to give an estimated time. We can then compare the times to see the relative performance of the approaches tested.

You can use this approach to benchmark your own favorite NumPy array operations.

If you use or extend the NumPy benchmarking approach used in this tutorial, let me know in the comments below. I'd love to see what you come up with.

We could use the time.perf_counter() function directly and develop a helper function to perform the benchmarking and report results.

You can learn more about benchmarking with the time.perf_counter() function in the tutorial:

Benchmark Python with time.perf_counter()

Instead, in this case, we will use the timeit API, specifically the timeit.timeit() function and specify the string of array copy code to run and a fixed number of times to run it.

We will also provide the globals argument for any constants defined in our benchmark code, such as array size or shape.

For example:

...
# benchmark a thing
result = timeit.timeit('...', globals=globals(), number=N)
print(f'approach {result:.3f} seconds')

You can learn more about benchmarking with the timeit.timeit() function in the tutorial:

Benchmark Python with timeit.timeit()

The number of runs in each benchmark was tuned to ensure that each snippet was executed in more than one second and less than about 10 seconds.

Let's get started.

Fastest Way to Copy 1D NumPy Array

We can explore the fastest way to create a copy of a modestly sized initialized one-dimensional NumPy array.

In this case, we will define a fixed size 1d array with one million elements (1,000,000) initialized to zero. Each approach will be used to create a copy of this 8,000 times.

The approaches we will compare include the most common NumPy functions for copying a new 1d array of a given size.

The complete example is listed below.

# SuperFastPython.com
# benchmark copying a 1d numpy array
import numpy
import timeit
# array to copy
A = numpy.zeros(1000000)
# number of times to run each snippet
N = 8000
# numpy.copy()
result = timeit.timeit('numpy.copy(A)', globals=globals(), number=N)
print(f'numpy.copy() {result:.3f} seconds')
# ndarray.copy()
result = timeit.timeit('A.copy()', globals=globals(), number=N)
print(f'ndarray.copy() {result:.3f} seconds')
# numpy.array()
result = timeit.timeit('numpy.array(A)', globals=globals(), number=N)
print(f'numpy.array() {result:.3f} seconds')
# numpy.empty();numpy.copyto()
result = timeit.timeit('B=numpy.empty(A.shape);numpy.copyto(B,A)', globals=globals(), number=N)
print(f'numpy.empty();numpy.copyto() {result:.3f} seconds')
# numpy.empty();B[:]=A[:]
result = timeit.timeit('B=numpy.empty(A.shape);B[:]=A[:]', globals=globals(), number=N)
print(f'numpy.empty(A.shape);B[:]=A[:] {result:.3f} seconds')

Running the example benchmarks each approach and reports the sum execution time.

numpy.copy() 10.933 seconds
ndarray.copy() 10.481 seconds
numpy.array() 10.531 seconds
numpy.empty();numpy.copyto() 11.027 seconds
numpy.empty(A.shape);B[:]=A[:] 11.044 seconds

We can restructure the output into a table for comparison.

Approach       | Time (sec)
---------------|------------
numpy.copy()   | 10.933
ndarray.copy() | 10.481
numpy.array()  | 10.531
numpy.copyto() | 11.027
B[:]=A[:]      | 11.044

We can see that all tested approaches have a similar benchmark time.

In this case, it is possible that the ndarray.copy() and numpy.array() approaches are slightly faster than numpy.copy() and numpy.copyto().

Given the closeness of the results, the effect may be a statistical fluke and all approaches may have a near identical overall execution time.

Re-running the same benchmark gives a different pattern of results, supporting this suggestion, although numpy.array() remains consistently fast, even with subsequent additional runs.

It may be that the numpy.array() approach to copying an array is doing so in a slightly more efficient manner.

numpy.copy() 10.326 seconds
ndarray.copy() 10.508 seconds
numpy.array() 10.315 seconds
numpy.empty();numpy.copyto() 10.652 seconds
numpy.empty(A.shape);B[:]=A[:] 10.948 seconds

Fastest Way to Copy 2D NumPy Array

We can explore the fastest way to copy a modestly sized initialized two-dimensional NumPy array.

Each array will have the size (1000,1000) and we will run each method 10,000 times.

The complete example is listed below.

# SuperFastPython.com
# benchmark copying a 2d numpy array
import numpy
import timeit
# array to copy
A = numpy.zeros((1000,1000))
# number of times to run each snippet
N = 10000
# numpy.copy()
result = timeit.timeit('numpy.copy(A)', globals=globals(), number=N)
print(f'numpy.copy() {result:.3f} seconds')
# ndarray.copy()
result = timeit.timeit('A.copy()', globals=globals(), number=N)
print(f'ndarray.copy() {result:.3f} seconds')
# numpy.array()
result = timeit.timeit('numpy.array(A)', globals=globals(), number=N)
print(f'numpy.array() {result:.3f} seconds')
# numpy.empty();numpy.copyto()
result = timeit.timeit('B=numpy.empty(A.shape);numpy.copyto(B,A)', globals=globals(), number=N)
print(f'numpy.empty();numpy.copyto() {result:.3f} seconds')
# numpy.empty();B[:]=A[:]
result = timeit.timeit('B=numpy.empty(A.shape);B[:]=A[:]', globals=globals(), number=N)
print(f'numpy.empty(A.shape);B[:]=A[:] {result:.3f} seconds')

Running the example benchmarks each approach and reports the sum execution time.

numpy.copy() 12.689 seconds
ndarray.copy() 12.568 seconds
numpy.array() 12.830 seconds
numpy.empty();numpy.copyto() 13.323 seconds
numpy.empty(A.shape);B[:]=A[:] 13.402 seconds

We can restructure the output into a table for comparison.

Approach       | Time (sec)
---------------|------------
numpy.copy()   | 12.689
ndarray.copy() | 12.568
numpy.array()  | 12.830
numpy.copyto() | 13.323
B[:]=A[:]      | 13.402

Again, we don't see much difference in the results.

All approaches have a similar execution time with perhaps numpy.copyto() taking slightly longer.

Like the one-dimensional example above, we expect there to be no significant difference between the approaches.

Repeating the benchmark shows a different pattern of results, confirming this suspicion.

It does appear that the slicing method, B[:]=A[:], may have slightly worse performance in each set of benchmark runs.

numpy.copy() 12.785 seconds
ndarray.copy() 12.620 seconds
numpy.array() 12.406 seconds
numpy.empty();numpy.copyto() 12.970 seconds
numpy.empty(A.shape);B[:]=A[:] 13.388 seconds

Takeaways

You now know how to benchmark and discover the fastest ways to copy NumPy arrays.

If you enjoyed this tutorial, you will love my book: Python Benchmarking. It covers everything you need to master the topic with hands-on examples and clear explanations.