You can cancel asyncio tasks and the request to cancel can propagate down a hierarchy of awaiting tasks.
The resulting CancelledError exceptions can then bubble back up the hierarchy.
In this tutorial, you will discover how CancelledError exceptions propagate when an asyncio task is cancelled.
Let’s get started.
What is Asyncio Task Cancellation?
Asyncio tasks can be canceled.
This can be achieved by calling the cancel() method on the asyncio.Task. This will request that the task cancel as soon as possible and returns True if the request was successful or False if it was not, e.g. the task is already done.
Request the Task to be cancelled. This arranges for a CancelledError exception to be thrown into the wrapped coroutine on the next cycle of the event loop.
— Coroutines and Tasks
For example:
|
1 2 3 4 |
... # cancel a task if task.cancel(): # request successful |
The next time that the target task that was cancelled resumes, a CancelledError exception will be raised.
This exception will bubble up to the top level of the task and cause it to stop running.
You can learn more about the mechanics of cancellation a task in the tutorial:
You can also learn more about task cancellation best practices in the tutorial:
Run loops using all CPUs, download your FREE book to learn how.
Asyncio Task Cancellation Propagates
When an asyncio task is cancelled, the CancelledError is propagated.
The CancelledError is propagated down to any task that the cancelled task is awaiting. The CancelledError is raised in the current task and is then bubbled up to the caller.
Any task in the hierarchy can intercept the CancelledError and choose not to propagate it up, although this is discouraged.
Task.cancel() does not guarantee that the Task will be cancelled, although suppressing cancellation completely is not common and is actively discouraged.
— Coroutines and Tasks
Generally, a CancelledError can be handled at the level that the cancellation request was made.
We can make the propagation of CancelledError exceptions clearer if we consider a hierarchy of tasks:
- Task 1 is created and awaited by the main program.
- Task 1 creates Task 2 and awaits it.
- Task 2 creates Task 3 and awaits it.
- Task 3 executes a long running task.
This hierarchy of tasks looks as follows:
- task1()
- -> task2()
- -> task3()
- -> task2()
Now, we can consider the cancellation of a task at each point in the hierarchy:
Cancel Top-Level Task
The main() coroutine may cancel the top-level task, Task 1.
For example:
- task1() <= cancel()
- -> task2()
- -> task3()
- -> task2()
Given that task1() is awaiting task2(), the CancelledError will propagate to task2().
Given that task2() is awaiting task3(), the CancelledError will propagate to task3().
Whatever task3() is doing, the CancelledError will propagate down to it as well.
This highlights that a cancellation of a top-level task in a task hierarchy will cause the propagation of the CancelledError from the target task down the hierarchy to all subtasks, assuming they are awaiting and assuming they do not consume the CancelledError.
Cancel Middle-Level Task
The task1() task may cancel the middle-level task, Task 2.
For example:
- task1()
- -> task2() <= cancel()
- -> task3()
- -> task2() <= cancel()
Given that task2() is awaiting task3(), the CancelledError will propagate to task3().
Whatever task3() is doing, the CancelledError will propagate down to it as well.
The CancelledError will then propagate back up to task1().
Given that task1() does not consume the CancelledError, it then propagates back up to the main() task driving the program.
This highlights that a cancellation request propagates down the task hierarchy as well as back up the hierarchy. A task in the middle of the hierarchy will have both subtasks and parent tasks cancelled, assuming they are awaiting and assuming they do not consume the CancelledError.
Cancel Bottom-Level Task
The task2() task may cancel the bottom-level task, Task 3.
For example:
- task1()
- -> task2()
- -> task3() <= cancel()
- -> task2()
Whatever task3() is doing, the CancelledError will propagate down to it.
The CancelledError will then propagate back up to task2().
Given that task2() does not consume the CancelledError, it then propagates back up to the task1().
Given that task1() does not consume the CancelledError, it then propagates back up to the main() task driving the program.
This highlights that a cancellation request at the bottom level of a task hierarchy can propagate all the way back up to the top level, assuming all tasks are waiting and do not consume the CancelledError exception.
This is probably the most expected and familiar case of CancelledError propagation, e.g. normal exception propagation.
When Is the CancelledError Propagated
A CancelledError is raised in a task the next time it runs.
Typically, the target task is suspended and another task is running which requests that the target cancel.
When the target next resumes, it raises a CancelledError.
But this is not always the case.
If the target task is running and is cancelled, e.g. it cancels itself, the task will not raise a CancelledError until the next time it resumes.
If the target task cancels itself and does not suspend, then the CancelledError will propagate when the task terminates, e.g. is done.
If a task cancels itself and has subtasks running in the background, they too will raise a CancelledError the next time they resume, or will propagate a CancelledError when they terminate.
Now that we are familiar with how cancellation requests can propagate through a task hierarchy, let’s look at some worked examples.
Example of Cancelling the Top-Level Task
We can explore asyncio task cancellation propagation when canceling a top-level task.
In this example, we will define a hierarchy of three tasks. Task1 awaits Task2, Task2 awaits Task3 and Task3 awaits a sleep.
Each task handles the CancelledError exception, so we can report a message, and then re-raise the exception.
The main() coroutine will run the Task1 task in the background, give it a moment to start the hierarchy, then cancel the top-level task.
We expect the cancellation to propagate down the hierarchy so that all three tasks are cancelled.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
# SuperFastPython.com # example of cancellation propagating down import asyncio # task that take a long time async def task3(): try: # sleep a long time await asyncio.sleep(5) print('Task3 completed normally') except asyncio.CancelledError: # report a message print('Task3 cancelled') # re-raise the cancellation raise # task that take a long time async def task2(): try: # await the next task await task3() print('Task2 completed normally') except asyncio.CancelledError: # report a message print('Task2 cancelled') # re-raise the cancellation raise # task that take a long time async def task1(): try: # await the next task await task2() print('Task1 completed normally') except asyncio.CancelledError: # report a message print('Task1 cancelled') # re-raise the cancellation raise # main coroutine async def main(): # create and schedule the task task = asyncio.create_task(task1()) # let the tasks run a moment await asyncio.sleep(1) # cancel the task task.cancel() # await the task try: await task except asyncio.CancelledError: print('Main saw the task cancel') print('Main is done') # start the event loop asyncio.run(main()) |
Running the example first starts the event loop and executes the main() coroutine.
The main() coroutine runs and schedules the task1() coroutine in the background. It then suspends for one second to allow the task1() coroutine to run.
The task1() coroutine runs and awaits task2().
The task2() coroutine runs and awaits task3().
The task3() coroutine runs and awaits a sleep.
The main() coroutine resumes and then cancels task1(). It then awaits the task.
The cancellation request propagates from task1 to task2 to task3 to sleep.
A CancelledError is raised in the sleep, then propagates to task3, which reports a message and re-raises the exception.
The CancelledError is propagated to task2 which is handled, reports a message, and re-raises the exception.
The CancelledError is propagated to task1 which is handled, reports a message, and re-raises the exception.
Finally, the CancelledError exception reaches the main() coroutine where it is handled and a message is reported.
The main() coroutine then reports a final message.
This highlights how a cancellation request can propagate down a hierarchy of awaiting tasks and the resulting CancelledError exception can bubble all the way back up.
|
1 2 3 4 5 |
Task3 cancelled Task2 cancelled Task1 cancelled Main saw the task cancel Main is done |
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Example of Cancelling the Middle-Level Task
We can explore asyncio task cancellation propagation when canceling a middle-level task.
In this case, we can update the above example so that the middle task, e.g. task2 is cancelled by task1.
This can be achieved by updating the task1() coroutine to first issue the task in the background, sleep a moment, then cancel task2 and await the cancellation to take effect.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
# task that take a long time async def task1(): # create and schedule the task task = asyncio.create_task(task2()) # let the tasks run a moment await asyncio.sleep(1) # cancel the task task.cancel() try: # await the next task await task print('Task1 completed normally') except asyncio.CancelledError: # report a message print('Task1 cancelled') # re-raise the cancellation raise |
This pattern is then removed from the main() coroutine so that the task1 coroutine is no longer cancelled directly.
|
1 2 3 4 5 6 7 8 9 10 11 |
# main coroutine async def main(): # create and schedule the task task = asyncio.create_task(task1()) # let the tasks run try: # await the task await task except asyncio.CancelledError: print('Main saw the task cancel') print('Main is done') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
# SuperFastPython.com # example of cancellation propagating down and up import asyncio # task that take a long time async def task3(): try: # sleep a long time await asyncio.sleep(5) print('Task3 completed normally') except asyncio.CancelledError: # report a message print('Task3 cancelled') # re-raise the cancellation raise # task that take a long time async def task2(): try: # await the next task await task3() print('Task2 completed normally') except asyncio.CancelledError: # report a message print('Task2 cancelled') # re-raise the cancellation raise # task that take a long time async def task1(): # create and schedule the task task = asyncio.create_task(task2()) # let the tasks run a moment await asyncio.sleep(1) # cancel the task task.cancel() try: # await the next task await task print('Task1 completed normally') except asyncio.CancelledError: # report a message print('Task1 cancelled') # re-raise the cancellation raise # main coroutine async def main(): # create and schedule the task task = asyncio.create_task(task1()) # let the tasks run try: # await the task await task except asyncio.CancelledError: print('Main saw the task cancel') print('Main is done') # start the event loop asyncio.run(main()) |
Running the example first starts the event loop and executes the main() coroutine.
The main() coroutine runs and schedules the task1() coroutine in the background. It then suspends and waits for the task to complete.
The task1() coroutine runs and schedules task2() to run in the background. It then sleeps for one second.
The task2() coroutine runs and awaits task3().
The task3() coroutine runs and awaits a sleep.
The task1() coroutine resumes and then cancels task2(). It then awaits the task.
The cancellation request propagates from task2 to task3 to sleep.
A CancelledError is raised in the sleep, then propagates to task3, which reports a message and re-raises the exception.
The CancelledError is propagated to task2 which is handled, reports a message, and re-raises the exception.
The CancelledError is propagated back to task1 which is handled, reports a message, and re-raises the exception.
Finally, the CancelledError exception reaches the main() coroutine where it is handled and a message is reported.
The main() coroutine then reports a final message.
This highlights how a cancellation request can propagate down a hierarchy of awaiting tasks and back up the hierarchy given the bubbling up of the CancelledError exception.
|
1 2 3 4 5 |
Task3 cancelled Task2 cancelled Task1 cancelled Main saw the task cancel Main is done |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Cancelling the Bottom-Level Task
We can explore asyncio task cancellation propagation when canceling a bottom-level task.
In this case, we can update the above example so that the middle task, e.g. task3 is cancelled by task2.
This can be achieved by updating the task2() coroutine to first issue the task in the background, sleep a moment, then cancel task3 and await the cancellation to take effect.
|
1 2 3 4 5 6 7 8 9 10 11 |
# task that take a long time async def task3(): try: # sleep a long time await asyncio.sleep(5) print('Task3 completed normally') except asyncio.CancelledError: # report a message print('Task3 cancelled') # re-raise the cancellation raise |
Task1 and main can then be simplified to create and await their subsequent tasks as before.
|
1 2 3 4 5 6 7 8 9 10 11 |
# task that take a long time async def task1(): try: # await the next task await task2() print('Task1 completed normally') except asyncio.CancelledError: # report a message print('Task1 cancelled') # re-raise the cancellation raise |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
# SuperFastPython.com # example of cancellation propagating up import asyncio # task that take a long time async def task3(): try: # sleep a long time await asyncio.sleep(5) print('Task3 completed normally') except asyncio.CancelledError: # report a message print('Task3 cancelled') # re-raise the cancellation raise # task that take a long time async def task2(): # create and schedule the task task = asyncio.create_task(task3()) # let the tasks run a moment await asyncio.sleep(1) # cancel the task task.cancel() try: # await the next task await task print('Task2 completed normally') except asyncio.CancelledError: # report a message print('Task2 cancelled') # re-raise the cancellation raise # task that take a long time async def task1(): try: # await the next task await task2() print('Task1 completed normally') except asyncio.CancelledError: # report a message print('Task1 cancelled') # re-raise the cancellation raise # main coroutine async def main(): # create and schedule the task task = asyncio.create_task(task1()) # let the tasks run try: # await the task await task except asyncio.CancelledError: print('Main saw the task cancel') print('Main is done') # start the event loop asyncio.run(main()) |
Running the example first starts the event loop and executes the main() coroutine.
The main() coroutine runs and schedules the task1() coroutine in the background. It then suspends and waits for the task to complete.
The task1() coroutine runs and awaits task2().
The task2() coroutine runs and schedules task3 to run in the background. It then sleeps for one second.
The task3() coroutine runs and awaits a sleep.
The task2() coroutine resumes and then cancels task3(). It then awaits the task.
The cancellation request propagates from task3 to sleep.
A CancelledError is raised in the sleep, then propagates to task3, which reports a message and re-raises the exception.
The CancelledError is propagated to task2 which is handled, reports a message, and re-raises the exception.
The CancelledError is propagated back to task1 which is handled, reports a message, and re-raises the exception.
Finally, the CancelledError exception reaches the main() coroutine where it is handled and a message is reported.
The main() coroutine then reports a final message.
This highlights how a cancellation request can propagate down into the asyncio infrastructure, e.g. asyncio.sleep(), and the resulting CancelledError can bubble back up to the top level of the program.
|
1 2 3 4 5 |
Task3 cancelled Task2 cancelled Task1 cancelled Main saw the task cancel Main is done |
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know how CancelledError exceptions propagate when an asyncio task is cancelled.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any additional tips?
I’d love to hear about them!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Simon Berger on Unsplash
Do you have any questions?