Asyncio gather() Return Values

May 14, 2024 Python Asyncio

We can retrieve return values from coroutines executed concurrently by asyncio.gather().

The asyncio.gather() function returns an asyncio.Future that executes all provided coroutines concurrently. Once the asyncio.Future is done, it returns a list that contains one return value for each coroutine provided to asyncio.gather(), in the order that they were provided.

In this tutorial, you will discover the return values from asyncio.gather() in Python.

Let's get started.

What is asyncio.gather()

The asyncio.gather() function allows us to run multiple coroutines or tasks concurrently.

Coroutines can be provided as positional arguments to asyncio.gather() and a list of return values is returned, for example:

...
# execute coroutines concurrently
results = await asyncio.gather(coro1(), coro2())

Alternately, we can create a list of coroutines or tasks and unpack them as positional arguments using the star operator (*), for example:

...
# creates a list of coroutine objects
coros = [coro(i) for i in range(100)]
# execute coroutines concurrently
results = await asyncio.gather(*coros)

You can learn more about how to use asyncio.gather() in the tutorial:

How to Use asyncio.gather() in Python

How To Get asyncio.gather() Return Values

A feature of asyncio.gather() is that it will return values from coroutines.

The return value from the awaitable from asyncio.gather() is a list.

The list contains one position value for each coroutine or task provided to asyncio.gather(), in the same order.

This is helpful as it allows us to concurrently execute a suite of coroutines and collect and process the return values once all tasks are done.

What If Coroutines Don't Return Values?

If a target task or coroutine does not return a value, then a result of None is added to the list returned from asyncio.gather().

What Type Is Returned Exactly?

The asyncio.gather() function returns an awaitable, specifically an asyncio.Future.

The result of the asyncio.Future is a list.

The list contains return values.

We can retrieve the list of return values by awaiting the returned awaitable directly.

For example:

...
# execute coroutines concurrently
results = await asyncio.gather(coro1(), coro2())

...
# get the future
future = asyncio.gather(coro1(), coro2())
# await future and get return values
results = await future

Alternatively, we can call the result() method on the asyncio.Future once it is done.

For example:

...
# get the future
future = asyncio.gather(coro1(), coro2())
# await future
await future
# get the return values
results = future.result()

What Order Are Values Returned?

The list returned from asyncio.gather() contains values in the same order as coroutines and tasks provided to asyncio.gather().

Are Return Values Provided As Coroutines Complete?

No, the list of return values is provided only once all provided coroutines and tasks are done.

If return values are required as tasks are completed, we can use asyncio.as_completed() which you can learn more about in the tutorial:

How to Use Asyncio as_completed() in Python

What If return_exceptions Is True?

If the return_exceptions argument to asyncio.gather() is true, then any coroutines or tasks that fail with an unhandled exception will have the raised exception as a return value instead of the intended return value.

You can learn more about handling exceptions with asyncio.gather() in the tutorial:

Asyncio gather() Handle Exceptions

Now that we know about the return values from asyncio.gather(), let's look at a worked example.

Example of asyncio.gather() Return Values

We can explore an example of getting return values from asyncio.gather().

In this example, we can define a coroutine that returns an integer value. We can issue many instances of the coroutines. Once all coroutines are done, we can confirm the return value from the awaitable returned from asyncio.gather() and then retrieve the specific return values.

Firstly, we can define our custom coroutine.

Our coroutine takes an integer argument, suspends with a sleep for two seconds, then returns the provided argument multiplied by 100.

The task_coro() coroutine below implements this.

# coroutine used for a task
async def task_coro(value):
    # sleep for a moment
    await asyncio.sleep(2)
    # return the result
    return value * 100

Next in the main coroutine, we can create a list of 5 instances of our coroutine with arguments from 0 to 4.

...
# create many coroutines
coros = [task_coro(i) for i in range(5)]

We can then issue these coroutines for execution using asyncio.gather() and retrieve the return value.

...
# run the tasks in the background
results = await asyncio.gather(*coros)

We will then report the type of the return value, and report each positional return value from the completed coroutines.

...
# report type of return value
print(type(results))
# report results
for result in results:
    print(result)

The main() coroutine that implements this is listed below.

# coroutine used for the entry point
async def main():
    # report a message
    print('Main starting')
    # create many coroutines
    coros = [task_coro(i) for i in range(5)]
    # run the tasks in the background
    results = await asyncio.gather(*coros)
    # report type of return value
    print(type(results))
    # report results
    for result in results:
        print(result)
    # report a message
    print('Main done')

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of asyncio gather return values
import asyncio

# coroutine used for a task
async def task_coro(value):
    # sleep for a moment
    await asyncio.sleep(2)
    # return the result
    return value * 100

# coroutine used for the entry point
async def main():
    # report a message
    print('Main starting')
    # create many coroutines
    coros = [task_coro(i) for i in range(5)]
    # run the tasks in the background
    results = await asyncio.gather(*coros)
    # report type of return value
    print(type(results))
    # report results
    for result in results:
        print(result)
    # report a message
    print('Main done')

# start the asyncio program
asyncio.run(main())

Running the example first runs the asyncio event loop and runs the main() coroutine.

The main() coroutine runs and reports a message.

A list of 5 coroutines is then created and then provided to asyncio.gather() for concurrent execution.

The main() coroutine then suspends and waits for all coroutines to complete.

All 5 coroutines begin executing, suspending for two seconds. They resume and return their integer values.

The main() coroutine results and receives the return value from asyncio.gather().

The type of the return value is confirmed to be a list.

The list is traversed and all return values are reported, in the order that the coroutines were provided to asyncio.gather().

A final message is reported and the program is terminated.

This confirms that asyncio.gather() returns a list of return values and highlights that return values in the list maintain the order of coroutines provided to asyncio.gather().

Main starting
<class 'list'>
0
100
200
300
400
Main done

Takeaways

You now know about the return values from asyncio.gather() in Python.

If you enjoyed this tutorial, you will love my book: Python Asyncio Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.