You cannot immediately terminate the asyncio event loop, or kill all running tasks.

Instead, we must enumerate all running tasks and request that each be canceled. This will cause all tasks to raise a CancelledError exception and unwind and match the way the asyncio event loop will automatically cancel all tasks when it is exited normally.

In this tutorial, you will discover how to shut down the asyncio event loop immediately by canceling all tasks.

Let’s get started.

Need to Shutdown Asyncio Immediately

Sometimes we need to shut down our asyncio program immediately.

This may be for many reasons, such as:

• A signal has been raised, like a SIGTERM, SIGKILL, or a keyboard interrupt.
• A fatal error has occurred in the program.
• A user or monitoring program has requested the program terminate.

The asyncio event loop does provide methods like stop() and close() but they cannot be used to immediately shut down the event loop.

Instead, they will cause an error. You can learn more about this in the tutorial:

• How to Exit the Asyncio Event Loop

Therefore, to terminate the event loop we must exit the main coroutine, e.g. the entry point of the program.

This may require killing all tasks in the event loop.

How can we kill all asyncio tasks?

How to Kill All Asyncio Tasks

We cannot kill all pending and running asyncio tasks.

Nevertheless, we can cancel them by calling the cancel() method.

This will cause each task to raise a CancelledError exception the next time they resume.

You can learn more about canceling asyncio tasks in the tutorial:

• How to Cancel an Asyncio Task

Note, that a task that is requested to cancel may not be canceled. It can choose to consume or swallow the request to cancel.

You can learn more about this in the tutorial:

• Asyncio CancelledError Can Be Consumed

As such, we may want to forcefully cancel each task.

You can learn how in the tutorial:

• How to Force an Asyncio Task to Cancel

Given that we must cancel all tasks instead of killing them, let’s look at how.

How to Cancel All Asyncio Tasks

We can cancel all tasks in the asyncio event loop by first getting a list of all tasks and then calling the cancel() method on each.

This can be achieved by first calling the asyncio.all_tasks() function to get a list of all tasks.

You can learn more about getting all tasks in the tutorial:

• How to Get All Asyncio Tasks in Python

We can then enumerate through the list and call the cancel() method on each.

Tying this together, we can define a function called cancel_all_tasks() that will cancel all tasks in the event loop.

It can be a function rather than a coroutine as it does not perform any awaiting of other coroutines or tasks.

We can then call this function from anywhere in our program at any time and all pending and running tasks will at least be requested to cancel.

For example:

How to Cancel All Asyncio Tasks And Exclude The Current Task

A limitation of blindly canceling all tasks is that the caller task is also canceled.

This is a problem if the caller tasks need to perform some logging or resource cleanup after all tasks have been canceled.

We may also trust the caller’s task to be done soon without complication, therefore it does not need to be canceled.

Therefore, we may want to cancel all tasks, except the current task.

This can be achieved by updating our cancel_all_tasks() coroutine to retrieve the current task and not cancel.

We can implement this with a boolean argument to “exclude_current“.

If this argument is set, the current task can be retrieved via the asyncio.current_task() function and removed from the list of tasks that are canceled.

For example:

You can learn more about how to get the current task in the tutorial:

• How to Get the Current Asyncio Task in Python

Note that this change will only work if the current task is not cancel_all_tasks(), but the calling asyncio task. If the cancel_all_tasks() function is changed into a coroutine, this approach will no longer work.

The updated cancel_all_tasks() function with this change is listed below.

It can then be called as a normal Python function and not cancel the calling asyncio Task.

For example:

How to Cancel All Asyncio Tasks With Exclusion List

A limitation of the previous approach is that only the current task is excluded.

We may have one or more background tasks running that we do not want to cancel.

This might include tasks for monitoring, logging, and more.

Therefore, we want a version of canceling all tasks that excludes a list of one or more tasks.

This can be achieved by updating the cancel_all_tasks() function to take a list of asyncio Tasks to exclude from the tasks that are canceled.

Then, while enumerating all tasks, we can skip any tasks in the provided exclusion list.

Tying this together, the updated cancel_all_tasks() function that supports a list of excluded tasks is listed below.

We can then call this function to cancel all tasks including itself by providing no arguments, e.g. a default empty exclusion list.

If we wanted to exclude the calling tasks, we could provide a list with one argument, the current task.

For example:

Otherwise, we can provide a list of all the tasks we want to exclude.

Another way we could implement this is to give each task a name and exclude tasks by name.

For example, we can call the set_name() method on an asyncio.Task after it is created to assign a useful name, then call the get_name() method on tasks in the cancel_all_tasks() function to check for tasks to exclude from cancellation.

You can learn more about managing task names in the tutorial:

• How to Get and Set Asyncio Task Names

How to Cancel All Asyncio Tasks And Wait

A common pattern when canceling all tasks is to wait for the tasks to be canceled.

Recall that calling the cancel() method only requests that a task be canceled. A target task is not canceled until it is given an opportunity to run and raise a CancelledError exception and unwind.

Therefore, some callers who cancel tasks may explicitly wait for the target to be done before moving on.

You can learn more about the cancel and await pattern in the tutorial:

• Asyncio Cancel Task and Wait

We may need to wait for all tasks to stop running before performing some clean-up of resources and logging.

This can be achieved by explicitly awaiting all tasks that were canceled.

Firstly, this requires changing the cancel_all_tasks() function into a coroutine that can await.

We then must keep track of all tasks that were requested to be canceled.

This can be achieved in a few ways. In this case, we will create a new list of all tasks with the excluded tasks filtered out.

For example:

We can then await all tasks to be canceled.

Again, this can be achieved in a few ways.

One way is to call asyncio.gather() and pass the list of canceled tasks. This approach might be preferred over other approaches as we can set the return_exceptions argument to return exceptions raised by the target tasks rather than have the exceptions propagate to the current task and unwind our intent to cancel and wait.

For example:

Awaiting all tasks will suspend the current task and allow all requested tasks to cancel, e.g. raise a CancelledError exception and unwind.

You can learn more about how to use asyncio.gather() in the tutorial:

• How to Use asyncio.gather() in Python

The updated cancel_all_tasks() coroutine with these changes is listed below.

This coroutine can then be awaited anywhere we wish to trigger all tasks to be canceled.

For example:

On Exiting The Main Coroutine

The main coroutine is the coroutine used as the entry point into the asyncio program.

For example, the coroutine we might pass to asyncio.run() to start our program.

When the main coroutine is done, e.g. returns or raises an exception, the asyncio event loop is terminated.

This will cause all asyncio tasks to be canceled.

We can look inside the run() function, actually the asyncio.Runner class and see that when it closes, it calls an internal function called _cancel_all_tasks().

The _cancel_all_tasks() function is listed below for reference.

We can see that it is doing a lot of the things that we are doing when we want to cancel all tasks.

It is first getting a list of all tasks, and bailing early if there are no tasks.

It then requests that all tasks be canceled by calling the cancel() method.

It then waits for all tasks to be canceled by calling asyncio.gather() configured to return rather than re-raise exceptions.

Finally, it logs any exceptions that might have been raised in the canceled tasks.

This is good confirmation that we are on the right track when we want to quickly cancel all tasks.

Now that we have reviewed how we might cancel all tasks in the asyncio event loop, let’s look at a worked example.

Example of Canceling All Asyncio Tasks

We can explore an example of canceling all tasks in asyncio.

In this example, we will define a task that sleeps for a fraction of 10 seconds. We will then issue 100 of these tasks and allow them to run for a while. We will then use our cancel_all_tasks() coroutine to cancel all tasks except the current task (the main task).

Firstly, we can define the task that will run for a random fraction of 10 seconds.

The body of the task will have a try-finally structure, and the finally block will simulate cleanup for a fraction of 2 seconds, to show that the tasks were canceled and are finishing up.

The work() coroutine below implements this.

Next, we can define the main() coroutine.

Firstly, a message is reported that the program has started, then 100 work() tasks are issued and allowed to run for a while.

The main() coroutine then awaits our cancel_all_tasks() coroutine defined above to cancel all running tasks except the current task, the main coroutine.

Finally, the main() coroutine reports a done message.

The main() coroutine below implements this.

We can then start the asyncio event loop and run the main() coroutine.

Tying this together, the complete example is listed below.

Running the example first starts the event loop and runs the main() coroutine.

The main() coroutine runs and reports a message.

It then creates and issues 100 work() tasks and then suspends for 4 seconds.

The work() tasks run and sleep for a fraction of 10 seconds. Some complete and report a done message.

The main() coroutine resumes and awaits the cancel_all_tasks() coroutine and excludes itself.

The cancel_all_tasks() coroutine runs and retrieves all tasks and removes the excluded task from the list. It then requests that all tasks in the list be canceled and then suspends and awaits the tasks to be done.

A CancelledError exception is raised in each of the remaining running tasks. Their finally blocks run and suspend each task for a random fraction of 2 seconds.

The finally blocks exit and the tasks are done.

The cancel_all_tasks() coroutine resumes and returns once all tasks exit their finally block.

The main() coroutine resumes and reports a final message before the program terminates.

This highlights how we can cancel all running tasks and exclude the main coroutine from the cancellation. It also highlights that although tasks can be requested to cancel, they may also delay the termination of the program with their own cleanup in a finally block.

Further Reading

This section provides additional resources that you may find helpful.

Python Asyncio Books

• Python Asyncio Mastery, Jason Brownlee (my book!)
• Python Asyncio Jump-Start, Jason Brownlee.
• Python Asyncio Interview Questions, Jason Brownlee.
• Asyncio Module API Cheat Sheet

I also recommend the following books:

• Python Concurrency with asyncio, Matthew Fowler, 2022.
• Using Asyncio in Python, Caleb Hattingh, 2020.
• asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.

Guides

• Python Asyncio: The Complete Guide
• Python Asynchronous Programming

APIs

• asyncio — Asynchronous I/O
• Asyncio Coroutines and Tasks
• Asyncio Streams
• Asyncio Subprocesses
• Asyncio Queues
• Asyncio Synchronization Primitives

References

• Asynchronous I/O, Wikipedia.
• Coroutine, Wikipedia.

Takeaways

You now know how to shut down the asyncio event loop immediately by canceling all tasks.

Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!

Do you have any additional tips?
I’d love to hear about them!

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by Tower Electric Bikes on Unsplash