We can execute asyncio tasks and coroutines concurrently, a main benefit of using asyncio.

There are four main ways that we can achieve this, including issuing coroutines as independent tasks and awaiting them directly, awaiting them automatically via a TaskGroup, using asyncio.wait() or using asyncio.gather().

In this tutorial, you will discover how to execute asyncio tasks and coroutines concurrently in Python.

Let’s get started.

What is a Coroutine

A coroutine represents a special type of function that can pause its execution at specific points without blocking other tasks.

It allows for concurrent and non-blocking operations, enabling asynchronous programming.

Coroutines are declared using the “async def” expression in Python, distinguishing them from regular functions.

For example:

They can be paused or suspended using the await expression within their body, allowing other coroutines or tasks to run while waiting for potentially blocking operations like I/O or delays.

Coroutines are a fundamental building block in asynchronous programming with Python, enabling efficient handling of concurrent tasks without resorting to thread-based parallelism.

They facilitate cooperative multitasking and make it easier to write asynchronous code that remains responsive and scalable.

You can learn more about coroutines in the tutorial:

• What is a Coroutine in Python

What is an Asyncio Task

An asyncio task in Python refers to a unit of work that runs asynchronously within an event loop.

It represents a coroutine wrapped in an asyncio.Task object, allowing the asyncio event loop to manage its execution.

This can be achieved via the asyncio.create_task() function.

For example:

Tasks enable concurrent execution of coroutines and provide features for monitoring and controlling their execution, such as cancellation, waiting for completion, and handling exceptions.

Tasks are fundamental in asyncio programming, allowing us to schedule and manage multiple asynchronous operations efficiently.

You can learn more about asyncio tasks in the tutorial:

• What is an Asyncio Task

How to Run Tasks Concurrently

There are 4 ways to run coroutines and concurrently, they are:

1. Issue As Concurrent Tasks And Wait Manually
2. Issue As Concurrent Tasks And Wait Via TaskGroup
3. Issue As Concurrent Tasks And asyncio.wait()
4. Use asyncio.gather()

Let’s take a closer look at each in turn.

Concurrent Tasks And Wait Manually

A simple approach to execute asyncio coroutines concurrently is to issue each as an independent task, and then wait for each task to complete.

For example, we can create and issue an asyncio.Task for each coroutine and save the asyncio.Task objects in a list via a list comprehension:

This will allow all tasks to be scheduled for concurrent execution in the background.

We can then manually await each task, allowing all tasks an opportunity to run.

For example:

Concurrent Tasks And Wait Via TaskGroup

Another similar approach is to create and schedule each coroutine to run as a background task via the asyncio.TaskGroup.

The benefit of the TaskGroup is that it uses an asynchronous context manager interface. Once we exit the block of the context manager, the asyncio.TaskGroup will automatically wait for all issued tasks to be done.

This uses less code than the above example.

For example:

You can learn more about how to use the asyncio.TaskGroup in the tutorial:

• How to use asyncio.TaskGroup

Concurrent Tasks And asyncio.wait()

A related approach is to issue each coroutine as a background task and then call the asyncio.wait() method to wait for all tasks to complete.

This is a simpler version of waiting for all tasks to be done that uses less code, e.g. a single function call.

For example:

You can learn more about the asyncio.wait() function in the tutorial:

• How to Use Asyncio wait() in Python

Concurrency With asyncio.gather()

Perhaps the most common approach to execute coroutines concurrently is to use the asyncio.gather() function.

This function takes one or more coroutines directly, or tasks and will return once all provided awaitables are done.

It also returns an iterator of return values from all issued coroutines.

For example:

You can learn more about how to use the asyncio.gather() function in the tutorial:

• How to Use asyncio.gather() in Python

Now that we know how to execute asyncio coroutines and tasks concurrently, let’s look at some worked examples.

Example of Concurrent Tasks and Wait Manually

We can explore an example of issuing coroutines as tasks and manually waiting for them to complete.

In this case, we will define a simple work() coroutine that takes an integer argument, sleeps for one second, and then reports its unique argument.

We can then execute 20 instances of this coroutine concurrently.

In this case, we will issue each as an independent task via the asyncio.create_task() function and collect the asyncio.Task objects into a list using a list comprehension.

We will then iterate over the list and await each task in turn.

This will allow all coroutines to be executed concurrently in the asyncio event loop. Once the loop is complete, we know that all tasks have been done.

We can then report a final message.

Tying this together, the complete example is listed below.

Running the example first starts the asyncio event loop and runs our main() coroutine as the entry point.

The main() coroutine runs and creates a list of 20 coroutines, issued as independent tasks.

The main() coroutine then iterates over the list of tasks, awaiting each in turn. This suspends the main() coroutine, allowing the issued tasks an opportunity to run.

Each task is given an opportunity to run in the event loop and sleeps immediately. Tasks finish their sleep, resume, report a message, and terminate.

The loop in the main() coroutine completes and a final message is reported.

This highlights how we can execute many coroutines concurrently in asyncio by issuing them as independent tasks and awaiting them manually.

Next, let’s explore how we might await the issued tasks automatically with a TaskGroup.

Example of Concurrent Tasks and Wait via TaskGroup

In this case, we can issue coroutines as independent tasks using an asyncio.TaskGroup, then automatically waits for all issued tasks to be done.

In this case, we can update the above example to create and use an asyncio.TaskGroup via its asynchronous context manager. This can be achieved via the “async with” expression and the task group can be assigned to an argument “group“.

This group can then be used to issue our coroutines as tasks by calling the create_task() method on the asyncio.TaskGroup instance.

For example:

Exiting the asynchronous context manager will automatically await all tasks issued using the asyncio.TaskGroup.

If you are new to asynchronous context managers, you can learn more about how they work in the tutorial:

• Asynchronous Context Managers in Python

Tying this together, the complete example is listed below.

Running the example first starts the asyncio event loop and runs our main() coroutine as the entry point.

The main() coroutine runs and creates an asyncio.TaskGroup via the asynchronous context manager interface.

The asyncio.TaskGroup is then used to create a list of 20 coroutines, issued as independent tasks.

The block of the asyncio.TaskGroup context manager is exited. This suspends the main() coroutine, allowing all issued tasks to run.

Each task is given an opportunity to run in the event loop and sleeps immediately. Tasks finish their sleep, resume, report a message, and terminate.

Once all tasks are done, the main() coroutine resumes and reports its final message.

This highlights how we can execute many coroutines concurrently in asyncio by issuing them as independent tasks via the asyncio.TaskGroup and automatically waits for the tasks to be done.

Next, let’s explore how we may achieve a similar result using the asyncio.wait() function.

Example of Concurrent Tasks and asyncio.wait()

We can explore an example of how to issue coroutines as independent background tasks and wait for them to be done via the asyncio.wait() function.

In this case, we can update the above example to create and issue a task for each coroutine and store the results in a list as we did in the first example.

We can then pass the list of tasks to the asyncio.wait() function and await it directly.

This will be suspended until all tasks in the list are done.

Tying this together, the complete example is listed below.

Running the example first starts the asyncio event loop and runs our main() coroutine as the entry point.

The main() coroutine runs and creates a list of 20 coroutines, issued as independent tasks.

The list of tasks is then provided to the asyncio.wait() function and the main() coroutine suspends until all tasks are done.

Each task is given an opportunity to run in the event loop and sleeps immediately. Tasks finish their sleep, resume, report a message, and terminate.

Once all tasks are done, the main() coroutine resumes and reports its final message.

This highlights how we can execute many coroutines concurrently in asyncio by issuing them as independent tasks and waiting for the tasks to be done via the asyncio.wait() function.

Next, let’s explore how we can execute coroutines concurrently using asyncio.gather().

Example of Concurrent Coroutines With asyncio.gather()

We can explore an example of executing asyncio coroutines concurrently using the asyncio.gather() function.

In this case, we will update the above example to create a list of coroutines, not tasks, and pass the list of coroutines directly to the asyncio.gather() function.

This function takes awaitable expressions directly, it does not take a collection of awaitables.

This means, that if we create a list of coroutine objects, we can expand the list into expressions for the asyncio.gather() function using the star operator (*).

For example:

Tying this together, the complete example is listed below.

Running the example first starts the asyncio event loop and runs our main() coroutine as the entry point.

The main() coroutine runs and creates a list of 20 coroutines, not tasks. We could create tasks if we wanted, but is not required in this case.

The list of coroutines is then provided to the asyncio.gather() function and the main() coroutine suspends until all coroutines are done.

Each coroutine is given an opportunity to run in the event loop and sleeps immediately. Coroutines finish their sleep, resume, report a message, and terminate.

Once all coroutines are done, the main() coroutine resumes and reports its final message.

This highlights how we can execute many coroutines concurrently in asyncio by passing them to the asyncio.gather() function.

Further Reading

This section provides additional resources that you may find helpful.

Python Asyncio Books

• Python Asyncio Mastery, Jason Brownlee (my book!)
• Python Asyncio Jump-Start, Jason Brownlee.
• Python Asyncio Interview Questions, Jason Brownlee.
• Asyncio Module API Cheat Sheet

I also recommend the following books:

• Python Concurrency with asyncio, Matthew Fowler, 2022.
• Using Asyncio in Python, Caleb Hattingh, 2020.
• asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.

Guides

• Python Asyncio: The Complete Guide
• Python Asynchronous Programming

APIs

• asyncio — Asynchronous I/O
• Asyncio Coroutines and Tasks
• Asyncio Streams
• Asyncio Subprocesses
• Asyncio Queues
• Asyncio Synchronization Primitives

References

• Asynchronous I/O, Wikipedia.
• Coroutine, Wikipedia.

Takeaways

You now know how to execute asyncio tasks and coroutines concurrently in Python.

Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!

Do you have any additional tips?
I’d love to hear about them!

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by Jan Kopřiva on Unsplash