How to Get All Asyncio Tasks in Python

Last Updated on November 14, 2023

The asyncio event loop is a program for executing asyncio tasks.

It runs our asyncio programs, but it also provides tools for introspecting the tasks that are running.

One of these tools is the ability to access all of the tasks that are currently running and are not yet done.

In this tutorial, you will discover how to get all asyncio tasks.

After completing this tutorial, you will know:

How to access all tasks running in the asyncio event loop.
How we might develop a coroutine to wait for all currently running tasks to complete.
The dangers of a task that attempts to wait upon itself.

Let’s get started.

Table of Contents

What is an Asyncio Task

An asyncio Task is an object that schedules and independently runs an asyncio coroutine.

It provides a handle on a scheduled coroutine that an asyncio program can query and use to interact with the coroutine.

A Task is an object that manages an independently running coroutine.
— PEP 3156 – Asynchronous IO Support Rebooted: the “asyncio” Module

An asyncio task is represented via an instance of the asyncio.Task class.

A task is created from a coroutine. It requires a coroutine object, wraps the coroutine, schedules it for execution, and provides ways to interact with it.

A task is executed independently. This means it is scheduled in the asyncio event loop and will execute regardless of what else happens in the coroutine that created it. This is different from executing a coroutine directly, where the caller must wait for it to complete.

Tasks are used to schedule coroutines concurrently. When a coroutine is wrapped into a Task with functions like asyncio.create_task() the coroutine is automatically scheduled to run soon
— Coroutines and Tasks

We can create a task using the asyncio.create_task() function.

This function takes a coroutine instance and an optional name for the task and returns an asyncio.Task instance.

Wrap the coro coroutine into a Task and schedule its execution. Return the Task object.
— Coroutines and Tasks

For example:

...

# create and schedule a task

task = asyncio.create_task(coro)

You can learn more about asyncio tasks in the tutorial:

What is an Asyncio Task

Now that we know about asyncio tasks, let’s look at how we might get all tasks in the program.

Run loops using all CPUs, download your FREE book to learn how.

How to Get All Asyncio Tasks

We may need to get access to all tasks in an asyncio program.

This may be for many reasons, such as:

To introspect the current status or complexity of the program.
To log the details of all running tasks.
To find a task that can be queried or canceled.

We can get a set of all scheduled and running (not yet done) tasks in an asyncio program via the asyncio.all_tasks() function.

For example:

...

# get all tasks

tasks = asyncio.all_tasks()

This will return a set of all tasks in the asyncio program.

It is a set so that each task is only represented once.

A task will be included if:

The task has been scheduled but is not yet running.
The task is currently running (e.g. but is currently suspended)

The set will also include a task for the currently running task, e.g. the task that is executing the coroutine that calls the asyncio.all_tasks() function.

Also, recall that the asyncio.run() method that is used to start an asyncio program will wrap the provided coroutine in a task. This means that the set of all tasks will include the task for the entry point of the program.

Now that we know how to get all tasks in an asyncio program, let’s look at some worked examples.

Download Now: Free Asyncio PDF Cheat Sheet

Example of Getting All Tasks With Only One Task Running

We can explore how to get all asyncio tasks when there is only one task running.

This might be the simplest case of getting all tasks.

In this example, we create one coroutine and use it as the entry point to the program. The coroutine then gets a set of all tasks, which only includes the Task object for itself.

The complete example is listed below.

# SuperFastPython.com

# example of getting all tasks when only one task is running

import asyncio

# define a main coroutine

async def main():

# report a message

print('main coroutine started')

# get all tasks

tasks = asyncio.all_tasks()

# report all tasks

for task in tasks:

print(f'> {task.get_name()}, {task.get_coro()}')

# start the asyncio program

asyncio.run(main())

Running the example first creates the main coroutine and uses it to start the asyncio program.

The main() coroutine runs and first reports a message.

It then gets a set of all scheduled and running tasks.

It then traverses the set of all tasks and reports the name and coroutine of each.

The current task is the only task in the program and its details are reported.

This highlights the simplest case of getting a set of all tasks and reporting their details.

1 2	main coroutine started >Task-1, <coroutine object main at 0x102dc8740>

Next, let’s look at the case of creating many tasks and then getting access to all of them.

Free Python Asyncio Course

Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.

Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.

Learn more

Example of Getting All Tasks With Many Tasks Running

We can explore the case where we have many tasks within an asyncio program and then get a set of all tasks.

In this example, we first create 10 tasks, each wrapping and running the same coroutine.

The main coroutine then gets a set of all tasks scheduled or running in the program and reports their details.

The complete example is listed below.

# SuperFastPython.com

# example of starting many tasks and getting access to all tasks

import asyncio

# coroutine for a task

async def task_coroutine(value):

# report a message

print(f'task {value} is running')

# block for a moment

await asyncio.sleep(1)

# define a main coroutine

async def main():

# report a message

print('main coroutine started')

# start many tasks

started_tasks = [asyncio.create_task(task_coroutine(i)) for i in range(10)]

# allow some of the tasks time to start

await asyncio.sleep(0.1)

# get all tasks

tasks = asyncio.all_tasks()

# report all tasks

for task in tasks:

print(f'> {task.get_name()}, {task.get_coro()}')

# wait for all tasks to complete

for task in started_tasks:

await task

# start the asyncio program

asyncio.run(main())

Running the example first creates the main coroutine and uses it to start the asyncio program.

The main() coroutine runs and first reports a message.

It then creates and schedules 10 tasks that wrap the custom coroutine,

The main() coroutine then blocks for a moment to allow the tasks to begin running.

The tasks start running and each reports a message and then sleeps.

The main() coroutine resumes and gets a list of all tasks in the program.

It then reports the name and coroutine of each.

Finally, it enumerates the list of tasks that were created and awaits each, allowing them to be completed.

This highlights that we can get a set of all tasks in an asyncio program that includes both the tasks that were created as well as the task that represents the entry point into the program.

main coroutine started

task 0 is running

task 1 is running

task 2 is running

task 3 is running

task 4 is running

task 5 is running

task 6 is running

task 7 is running

task 8 is running

task 9 is running

> Task-9, <coroutine object task_coroutine at 0x10e186e30>

> Task-2, <coroutine object task_coroutine at 0x10e184e40>

> Task-11, <coroutine object task_coroutine at 0x10e186f10>

> Task-7, <coroutine object task_coroutine at 0x10e186d50>

> Task-4, <coroutine object task_coroutine at 0x10e185700>

> Task-10, <coroutine object task_coroutine at 0x10e186ea0>

> Task-8, <coroutine object task_coroutine at 0x10e186dc0>

> Task-5, <coroutine object task_coroutine at 0x10e186ab0>

> Task-1, <coroutine object main at 0x10e1847b0>

> Task-3, <coroutine object task_coroutine at 0x10e184f90>

> Task-6, <coroutine object task_coroutine at 0x10e186ce0>

Next, let’s look at how we might wait on all tasks in the program.

Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps

Example of Waiting on All Tasks

A common use case for getting all tasks in an asyncio program is to wait on all tasks.

This allows the caller to suspend and wait for all tasks to execute, such as before exiting the program.

The problem is, if we get all tasks and naively await each, it will result in an error. The reason is that the set of all tasks includes the current task and the current task cannot await itself.

We can demonstrate this with an example.

The example below first creates 10 tasks, it waits a moment for the tasks to start, then gets the set of all tasks and awaits each in turn.

The result is a RuntimeException.

The complete example is listed below.

# SuperFastPython.com

# example of staring many tasks and getting access to all tasks

import asyncio

# coroutine for a task

async def task_coroutine(value):

# report a message

print(f'task {value} is running')

# block for a moment

await asyncio.sleep(1)

# define a main coroutine

async def main():

# report a message

print('main coroutine started')

# start many tasks

for i in range(10):

asyncio.create_task(task_coroutine(i))

# allow some of the tasks time to start

await asyncio.sleep(0.1)

# get all tasks

tasks = asyncio.all_tasks()

# wait for all tasks to complete

for task in tasks:

await task

# start the asyncio program

asyncio.run(main())

Running the example first creates the main coroutine and uses it to start the asyncio program.

The main() coroutine runs and first reports a message.

It then creates and schedules 10 tasks that wrap the custom coroutine,

The main() coroutine then blocks for a moment to allow the tasks to begin running.

The tasks start running and each reports a message and then sleeps.

The main() coroutine resumes and gets a set of all tasks in the program.

It traverses the set of all tasks and awaits each.

This fails with a RuntimeException.

It fails because the set of all tasks contains a task for the currently running task and a task cannot await itself.

main coroutine started

task 0 is running

task 1 is running

task 2 is running

task 3 is running

task 4 is running

task 5 is running

task 6 is running

task 7 is running

task 8 is running

task 9 is running

Traceback (most recent call last):

...

RuntimeError: Task cannot await on itself: <Task pending name='Task-1' coro=<main() running at ...> cb=[_run_until_complete_cb() at ...]>

Instead, the example must be updated so that the running task does not await itself.

This can be achieved by first getting the Task object for the currently running task.

We then await each task as long as it does not represent the currently running task.

For example:

...

# get all tasks

tasks = asyncio.all_tasks()

# get the current task

current = asyncio.current_task()

# wait for all tasks to complete

for task in tasks:

# skip the current task

if task is current:

continue

# await the task

await task

We can simplify this by removing the current task from the set of all tasks, then traverse the set of the remaining tasks and await each in turn.

For example:

...

# get all tasks

tasks = asyncio.all_tasks()

# get the current task

current = asyncio.current_task()

# remove the current task from the set

tasks.remove(current)

# wait for all tasks to complete

for task in tasks:

# await the task

await task

Tying this together, the example of first getting all tasks in an asyncio program and awaiting each is listed below.

# SuperFastPython.com

# example of starting many tasks and getting access to all tasks

import asyncio

# coroutine for a task

async def task_coroutine(value):

# report a message

print(f'task {value} is running')

# block for a moment

await asyncio.sleep(1)

# define a main coroutine

async def main():

# report a message

print('main coroutine started')

# start many tasks

for i in range(10):

asyncio.create_task(task_coroutine(i))

# allow some of the tasks time to start

await asyncio.sleep(0.1)

# get all tasks

tasks = asyncio.all_tasks()

# get the current task

current = asyncio.current_task()

# remove the current task from the set

tasks.remove(current)

# wait for all tasks to complete

for task in tasks:

# await the task

await task

# start the asyncio program

asyncio.run(main())

Running the example first creates the main coroutine and uses it to start the asyncio program.

The main() coroutine runs and first reports a message.

It then creates and schedules 10 tasks that wrap the custom coroutine,

The main() coroutine then blocks for a moment to allow the tasks to begin running.

The tasks start running and each reports a message and then sleeps.

The main() coroutine resumes and gets a set of all tasks in the program. It also gets the Task object for the currently running task (e.g. the entry point) and removes it from the set of all tasks.

It then traverses the remaining set of all tasks and awaits each in turn.

This highlights how we can get a set of all tasks in an asyncio program and await each before exiting the program.