We can avoid a TypeError exception when using asyncio.gather() by unpacking the collection of awaitables with the star (*) operator.
The asyncio.gather() function is used to execute multiple coroutines concurrently and return a list of return values once all tasks are done.
The function will fail with a TypeError exception if a list or set of coroutines is provided directly. Instead, the list must be unpacked into positional arguments before being provided to the asyncio.gather() function. This can be achieved using the star or asterisk operator (e.g. *) to unpack the list or set.
In this tutorial, you will discover why you get a TypeError exception with asyncio.gather() and how to fix it.
Let’s get started.
TypeError with asyncio.gather()
It is common to get a TypeError exception when using asyncio.gather().
Specifically:
|
1 |
TypeError: unhashable type: 'list' |
Another common example includes:
|
1 |
TypeError: unhashable type: 'set' |
The error is confusing because the asyncio.gather() function takes multiple coroutines or tasks.
If a collection is provided directly to this function, then we get a TypeError.
Why do we get a TypeError when using asyncio.gather()?
Run loops using all CPUs, download your FREE book to learn how.
How to Fix TypeError With asyncio.gather()
We may get a TypeError exception when using asyncio.gather() if we provide a collection of awaitables directly as an argument.
For example:
|
1 2 3 4 5 |
... # create many coroutines coros = [task_coro(i) for i in range(10)] # run the tasks and get the results results = await asyncio.gather(coros) |
The reason is because the asyncio.gather() cannot take a collection as an argument.
We cannot provide a list, set, or dict directly to asyncio.gather().
Instead, we must provide awaitables as positional arguments.
For example:
|
1 2 3 |
... # run the tasks and get the results results = await asyncio.gather(coro1(), coro2(), coro3()) |
We can use a collection if we expand the collection into separate expressions before providing them to asyncio.gather().
This can be achieved via the star operator or asterisk operator (e.g. *). It is also sometimes called the splat operator or the unpack operator.
You can learn more in the Python API documentation here:
For example:
|
1 2 3 4 5 |
... # create many coroutines coros = [task_coro(i) for i in range(10)] # run the tasks and get the results results = await asyncio.gather(*coros) |
Notice the star (*) in front of the “coros” argument.
This expands the “coros” collection into separate expressions that are provided as positional arguments to the asyncio.gather() function.
You can learn more about how to use the asyncio.gather() function in the tutorial:
Next, let’s consider why we cannot provide a list directly to asyncio.gather().
Why Can’t We Provide a List to asyncio.gather()
The asyncio.gather() function takes an unbounded collection of awaitables as the first argument.
We can see this if we review the API documentation or the source code for the function.
For example, the API documentation:
|
1 2 |
awaitable asyncio.gather(*aws, return_exceptions=False) ... |
And, the source code:
|
1 2 |
def gather(*coros_or_futures, return_exceptions=False): ... |
Notice that gather() is a function that returns an awaitable, it is not a coroutine function.
Also, notice that the first argument is not a list.
Instead, it is a variable number of positional arguments, typically written as *args.
A “*args” argument in a function definition is a special variable argument (var args or varargs) that generally means “an unspecified or undefined number of positional arguments“.
Most programming languages offer a way of implementing varargs for function definitions, and “*args” is Python’s approach.
For example, see:
The difference between using a list and varags is subtle.
- A list is a single function argument.
- A *args is an unbounded sequence of positional arguments.
The asyncio.gather() uses “*args” instead of a list argument because it is more general. It does not require a list as an argument, it can take any number of positional arguments.
If asyncio.gather() took a list, then the function might be defined with an empty list as a default parameter.
For example:
|
1 2 |
def gather(coros_or_futures=[], return_exceptions=False): ... |
But this would be more limiting, it could not be called as follows:
|
1 2 |
... await asyncio.gather(coro1(), coro2(), coro3()) |
It would always require a list, which is more limiting.
If we really need to provide a list or set directly, we could use asyncio.wait() instead and retrieve return values from each task manually. This would require that we create a list of asyncio.Task objects instead of coroutines.
For example:
|
1 2 3 4 5 |
... # wait for all tasks to be done _ = asyncio.wait(tasks) # retrieve return values from completed tasks results = [task.result() for task in tasks] |
Alternately we could use a wrapper function for asyncio.gather(), for example:
|
1 2 3 |
# wrapper for asyncio.gather() that takes a list def gather_wrapper(awaitables=[], **args): return asyncio.gather(*awaitables, **args) |
Now that we know how to avoid the TypeError exception when using asyncio.gather(), let’s look at some worked examples.
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Example of TypeError: unhashable type: ‘list’
Before we explore an example of how to fix the TypeError exception, let’s look at an example that causes the exception.
In this case, we can define a simple custom coroutine that takes an argument, reports a message with the argument, suspends a moment, and then returns the argument.
|
1 2 3 4 5 6 7 8 |
# custom coroutine async def task_coro(value): # report a message print(f'>task {value} executing') # sleep for a moment await asyncio.sleep(1) # return the value return value |
We can then create a list of coroutine objects from our coroutine definition with different arguments and provide this to the asyncio.gather() function for concurrent execution.
|
1 2 3 4 5 |
... # create many coroutines coros = [task_coro(i) for i in range(10)] # run the tasks and get the results results = await asyncio.gather(coros) |
We expect this to fail with a TypeError exception.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
# SuperFastPython.com # example of typeerror with gather import asyncio # custom coroutine async def task_coro(value): # report a message print(f'>task {value} executing') # sleep for a moment await asyncio.sleep(1) # return the value return value # coroutine used for the entry point async def main(): # report a message print('main starting') # create many coroutines coros = [task_coro(i) for i in range(10)] # run the tasks and get the results results = await asyncio.gather(coros) # report the results print(f'results: {results}') # start the asyncio program asyncio.run(main()) |
Running the example first starts the asyncio event loop and executes our main() coroutine.
The main() coroutine runs and reports a message.
It then creates a list of 10 coroutine objects from our coroutine definition with different arguments from 0 to 9.
The list of coroutine objects is then provided to asyncio.gather() and awaited for a list of results.
This fails with a TypeError exception.
The event loop is terminated and the program exits, emitting a RuntimeWarning that our coroutine objects were never awaited.
This highlights that we cannot provide a list of coroutine objects as an argument directly to the asyncio.gather() function.
|
1 2 3 |
main starting TypeError: unhashable type: 'list' sys:1: RuntimeWarning: coroutine 'task_coro' was never awaited |
Next, let’s look at how we fix this error.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Fixed TypeError with asyncio.gather()
We can explore an example of providing a list to asyncio.gather() without getting a TimeError exception.
In this case, we can update the above example to use the star operator (*) to first unpack the list into positional arguments before providing them to asyncio.gather().
|
1 2 3 |
... # run the tasks and get the results results = await asyncio.gather(*coros) |
And that’s it.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
# SuperFastPython.com # example of typeerror with gather import asyncio # custom coroutine async def task_coro(value): # report a message print(f'>task {value} executing') # sleep for a moment await asyncio.sleep(1) # return the value return value # coroutine used for the entry point async def main(): # report a message print('main starting') # create many coroutines coros = [task_coro(i) for i in range(10)] # run the tasks and get the results results = await asyncio.gather(*coros) # report the results print(f'results: {results}') # start the asyncio program asyncio.run(main()) |
Running the example first starts the asyncio event loop and executes our main() coroutine.
The main() coroutine runs and reports a message.
It then creates a list of 10 coroutine objects from our coroutine definition with different arguments from 0 to 9.
The list of coroutine objects is then unpacked into positional arguments and provided to asyncio.gather() and awaited for a list of results.
The main() coroutine suspends and each coroutine in the list is scheduled as a task and executed in the event loop.
Each task runs, reporting a message, suspending a moment, and returning the value.
All tasks are complete and a list of return values from all issued tasks is retrieved and reported.
This highlights how we can unpack a list of awaitables before providing them to asyncio.gather() in order to avoid a TypeError exception.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
main starting >task 0 executing >task 1 executing >task 2 executing >task 3 executing >task 4 executing >task 5 executing >task 6 executing >task 7 executing >task 8 executing >task 9 executing results: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] |
Takeaways
You now know why you get a TypeError exception with asyncio.gather() and how to fix it.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any additional tips?
I’d love to hear about them!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Do you have any questions?