Last Updated on November 14, 2023
An asyncio task may fail with an unhandled exception.
The exception will unwind the task, although may not impact other tasks or the broader asyncio program.
As such, we need a way of checking if a task has failed and of retrieving any unhandled exceptions in the task, if they occurred.
In this tutorial, you will discover exception handling in asyncio tasks.
After completing this tutorial, you will know:
- How to check if a task failed due to an unhandled exception.
- How to retrieve the exception from a task and what happens if we get the exception while the task is running.
- How and when the exception in the task may be propagated to another coroutine or task.
Let’s get started.
What is an Asyncio Task
An asyncio Task is an object that schedules and independently runs an asyncio coroutine.
It provides a handle on a scheduled coroutine that an asyncio program can query and use to interact with the coroutine.
A Task is an object that manages an independently running coroutine.
— PEP 3156 – Asynchronous IO Support Rebooted: the “asyncio” Module
An asyncio task is represented via an instance of the asyncio.Task class.
A task is created from a coroutine. It requires a coroutine object, wraps the coroutine, schedules it for execution, and provides ways to interact with it.
A task is executed independently. This means it is scheduled in the asyncio event loop and will execute regardless of what else happens in the coroutine that created it. This is different from executing a coroutine directly, where the caller must wait for it to complete.
Tasks are used to schedule coroutines concurrently. When a coroutine is wrapped into a Task with functions like asyncio.create_task() the coroutine is automatically scheduled to run soon
— Coroutines and Tasks
We can create a task using the asyncio.create_task() function.
This function takes a coroutine instance and an optional name for the task and returns an asyncio.Task instance.
Wrap the coro coroutine into a Task and schedule its execution. Return the Task object.
— Coroutines and Tasks
For example:
|
1 2 3 |
... # create and schedule a task task = asyncio.create_task(coro) |
You can learn more about asyncio tasks in the tutorial:
Now that we know about asyncio tasks, let’s look at how we might handle and check for exceptions.
Run loops using all CPUs, download your FREE book to learn how.
How to Check for Exceptions in Tasks
A coroutine wrapped by a task may raise an exception that is not handled.
This will fail the task, in effect.
We can retrieve an unhandled exception in the coroutine wrapped by a task via the exception() method.
For example:
|
1 2 3 |
... # get the exception raised by a task exception = task.exception() |
If an unhandled exception was not raised in the wrapped coroutine, then a value of None is returned.
If the task was canceled, then a CancelledError exception is raised when calling the exception() method and may need to be handled.
For example:
|
1 2 3 4 5 6 |
... try: # get the exception raised by a task exception = task.exception() except asyncio.CancelledError: # task was canceled |
As such, it is a good idea to check if the task was canceled first.
For example:
|
1 2 3 4 5 6 7 |
... # check if the task was not canceled if not task.cancelled(): # get the exception raised by a task exception = task.exception() else: # task was canceled |
If the task is not yet done, then an InvalidStateError exception is raised when calling the exception() method and may need to be handled.
For example:
|
1 2 3 4 5 6 |
... try: # get the exception raised by a task exception = task.exception() except asyncio.InvalidStateError: # task is not yet done |
As such, it is a good idea to check if the task is done first.
For example:
|
1 2 3 4 5 6 |
... # check if the task is not done if not task.done(): await task # get the exception raised by a task exception = task.exception() |
Next, let’s look at when an unhandled exception in a task is propagated to the caller.
When Are Task Exceptions Propagated to the Caller
Exceptions that occur within a task can be propagated to the caller.
This can happen in two situations, they are:
- When the caller awaits the task.
- When the caller gets the result from the task.
When a coroutine awaits a task that raises an unhandled exception, the exception is propagated to the caller.
For example:
|
1 2 3 |
... # wait for the task to finish await task |
Therefore, if an unhandled exception is possible in a Task’s coroutine, it may need to be handled when awaiting the task.
For example:
|
1 2 3 4 5 6 |
... try: # wait for the task to finish await task except Exception as e: # ... |
Similarly, if the task is done and the caller tempts to retrieve the return value from the task via the result() method, any unhandled exceptions are propagated.
For example:
|
1 2 3 |
... # get the return value from the task value = task.result() |
Therefore, if an unhandled exception is possible in a Task‘s coroutine, it may need to be handled when awaiting the task.
For example:
|
1 2 3 4 5 6 |
... try: # get the return value from the task value = task.result() except Exception as e: # ... |
Now that we know when exceptions in tasks are propagated, let’s look at some worked examples of checking for and handling exceptions in tasks.
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Example of Checking for an Exception in a Done Task
We can explore how to check for and get an exception from a successfully done task.
That is, check for an exception in a task that does not raise an exception.
The expectation is that the exception() method will return None after the task is done.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
# SuperFastPython.com # example of getting an exception from a successful task import asyncio # define a coroutine for a task async def task_coroutine(): # report a message print('executing the task') # block for a moment await asyncio.sleep(1) # custom coroutine async def main(): # report a message print('main coroutine started') # create and schedule the task task = asyncio.create_task(task_coroutine()) # wait for the task to complete await task # get the exception ex = task.exception() print(f'exception: {ex}') # report a final message print('main coroutine done') # start the asyncio program asyncio.run(main()) |
Running the example starts the asyncio event loop and executes the main() coroutine.
The main() coroutine reports a message, then creates and schedules the task coroutine.
It then suspends and awaits the task to be completed.
The task runs, reports a message, and sleeps for a moment before terminating normally.
The main() coroutine resumes and retrieves an exception from the task.
The task did not raise an unhandled exception, so the exception() method returns None.
This example highlights that a successful task will return None if an unhandled exception was not raised. This could be checked for, e.g. checking to see if a task failed or not.
|
1 2 3 4 |
main coroutine started executing the task exception: None main coroutine done |
Next, we can look at an example of retrieving an exception from a failed task.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Checking for an Exception in a Failed Task
We can explore getting an exception from a task that failed with an unhandled exception.
This is the exact use case for the exception() method.
In this example, we can update the task coroutine to explicitly raise an exception that is not handled.
This will cause the task coroutine to fail.
The main coroutine will sleep to wait for the task to be completed. This is to avoid using the await expression which will propagate the exception back to the caller.
Once the task is done, the main coroutine will retrieve and report the exception raised in the task.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
# SuperFastPython.com # example of getting an exception from a failed task import asyncio # define a coroutine for a task async def task_coroutine(): # report a message print('executing the task') # block for a moment await asyncio.sleep(1) # raise an exception raise Exception('Something bad happened') # custom coroutine async def main(): # report a message print('main coroutine started') # create and schedule the task task = asyncio.create_task(task_coroutine()) # wait for the task to complete await asyncio.sleep(1.1) # get the exception ex = task.exception() print(f'exception: {ex}') # report a final message print('main coroutine done') # start the asyncio program asyncio.run(main()) |
Running the example starts the asyncio event loop and executes the main() coroutine.
The main() coroutine reports a message, then creates and schedules the task coroutine.
It then suspends and awaits the task to be completed.
The task runs, reports a message, and sleeps for a moment. The task resumes and raises an exception.
The exception does not terminate the application or the asyncio event loop.
Instead, the exception is captured by the asyncio event loop and stored in the task.
The main() coroutine resumes and then retrieves the exception from the task, which is reported.
|
1 2 3 4 |
main coroutine started executing the task exception: Something bad happened main coroutine done |
Next, let’s look at what happens if we try to retrieve an exception from a running task.
Example of Checking for an Exception in a Running Task
We cannot retrieve an exception from a running asyncio task.
Instead, we can only retrieve the exception from a task after it is done.
If we call the exception() method on a task that is scheduled or running, an InvalidStateError exception is raised in the caller.
The example below demonstrates this.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
# SuperFastPython.com # example of getting an exception from a running task import asyncio # define a coroutine for a task async def task_coroutine(): # report a message print('executing the task') # block for a moment await asyncio.sleep(1) # custom coroutine async def main(): # report a message print('main coroutine started') # create and schedule the task task = asyncio.create_task(task_coroutine()) # wait a moment await asyncio.sleep(0.2) # get the exception ex = task.exception() print(f'exception: {ex}') # report a final message print('main coroutine done') # start the asyncio program asyncio.run(main()) |
Running the example starts the asyncio event loop and executes the main() coroutine.
The main() coroutine reports a message, then creates and schedules the task coroutine.
It then suspends and sleeps for a moment.
The task runs, reports a message, and sleeps for a moment.
The main() coroutine resumes and attempts to retrieve the exception from the task while the task is running, even though the task is suspended.
This fails with an InvalidStateError that breaks the asyncio event loop in this case.
This example highlights that we must always retrieve a Task exception after the task is done.
|
1 2 3 4 5 |
main coroutine started executing the task Traceback (most recent call last): ... asyncio.exceptions.InvalidStateError: Exception is not set. |
We can check if a task is done before retrieving the exception via the done() method that will return True if the task is done, or False otherwise.
Next, we can look at the case of attempting to get a task exception for a canceled task.
Example of Checking for an Exception in a Canceled Task
We cannot retrieve an exception from a canceled task.
Although a canceled task is done, an exception will not be available and cannot be retrieved.
Instead, a CancelledError exception is raised when calling the exception() method if the task was canceled.
We can demonstrate this with a worked example.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
# SuperFastPython.com # example of getting an exception from a running task import asyncio # define a coroutine for a task async def task_coroutine(): # report a message print('executing the task') # block for a moment await asyncio.sleep(1) # custom coroutine async def main(): # report a message print('main coroutine started') # create and schedule the task task = asyncio.create_task(task_coroutine()) # wait a moment await asyncio.sleep(0.1) # cancel the task task.cancel() # wait a moment await asyncio.sleep(0.1) # get the exception ex = task.exception() print(f'exception: {ex}') # report a final message print('main coroutine done') # start the asyncio program asyncio.run(main()) |
Running the example starts the asyncio event loop and executes the main() coroutine.
The main() coroutine reports a message, then creates and schedules the task coroutine.
It then suspends and sleeps for a moment.
The main() coroutine resumes and cancels the task. It then suspends and waits a moment for the task to respond to the request for being canceled.
The task is canceled by raising a CancelledError within the wrapped coroutine.
The main() coroutine resumes and attempts to retrieve an exception.
This fails and the CancelledError exception is re-raised in the caller.
This breaks the event loop in this case.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
main coroutine started executing the task Traceback (most recent call last): ... asyncio.exceptions.CancelledError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "...", line 25, in main ex = task.exception() asyncio.exceptions.CancelledError During handling of the above exception, another exception occurred: Traceback (most recent call last): ... asyncio.exceptions.CancelledError |
Next, let’s look at how we might handle an exception propagated by awaiting a task.
Example of Handling a Task Exception with await
Awaiting a task that fails with an exception will cause the exception to be propagated to the caller.
As such, awaiting a task may require that the unhandled but possible exceptions be handled.
The example below demonstrates this with a task that fails with an exception that is awaited in a main coroutine that expects and then handles the exception
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
# SuperFastPython.com # example of handling a task exception when await import asyncio # define a coroutine for a task async def task_coroutine(): # report a message print('executing the task') # block for a moment await asyncio.sleep(1) # fail with an exception raise Exception('Something bad happened') # custom coroutine async def main(): # report a message print('main coroutine started') # create and schedule the task task = asyncio.create_task(task_coroutine()) try: # wait for the task to complete await task except Exception as e: print(f'Failed with: {e}') # report a final message print('main coroutine done') # start the asyncio program asyncio.run(main()) |
Running the example starts the asyncio event loop and executes the main() coroutine.
The main() coroutine reports a message, then creates and schedules the task coroutine.
It then suspends and awaits the task to be completed. Importantly, the main() coroutine awaits the task within a try-except block.
The task runs, reports a message and sleeps for a moment, and then fails with an exception.
The main() coroutine resumes and handles the exception that was raised in the wrapped coroutine. The exception is propagated to the caller, caught, and the details are reported.
This highlights that we may need to handle unhandled exceptions because they can be propagated back to any coroutines waiting on the task.
|
1 2 3 4 |
main coroutine started executing the task Failed with: Something bad happened main coroutine done |
Next, we will look at how to handle task exceptions propagated to the caller when getting task results.
Example of Handling a Task Exception with result()
We can get the return value from a task via the result() method.
Care must be taken with this method because any exception that was raised in the Task‘s coroutine that was not handled will be propagated back and re-raised in the caller.
We can demonstrate this with a worked example.
The task coroutine returns a value, but the line is never reached because it fails with an exception.
The main coroutine attempts to retrieve the result from the task and handles the exception that may be raised and propagated.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
# SuperFastPython.com # example of handling a task exception when getting the result import asyncio # define a coroutine for a task async def task_coroutine(): # report a message print('executing the task') # block for a moment await asyncio.sleep(1) # fail with an exception raise Exception('Something bad happened') # return a value (never reached) return 100 # custom coroutine async def main(): # report a message print('main coroutine started') # create and schedule the task task = asyncio.create_task(task_coroutine()) # wait for the task to complete await asyncio.sleep(1.1) try: # get the result value = task.result() except Exception as e: print(f'Failed with: {e}') # report a final message print('main coroutine done') # start the asyncio program asyncio.run(main()) |
Running the example starts the asyncio event loop and executes the main() coroutine.
The main() coroutine reports a message, then creates and schedules the task coroutine.
It then suspends and sleeps a moment to allow the task to be completed.
The task runs, reports a message and sleeps for a moment, and then fails with an exception.
The main() coroutine resumes and attempts to retrieve the return value from the task.
This fails and the unhandled exception raised in the task’s coroutine is re-raised in the caller.
The main() coroutine handles the exception, catching it and reporting the details.
This highlights that we may need to handle unhandled exceptions when getting task results because they can be propagated back to any coroutines waiting on the task.
|
1 2 3 4 |
main coroutine started executing the task Failed with: Something bad happened main coroutine done |
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know how to handle exceptions in asyncio tasks in Python.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by William Chiesurin on Unsplash
Do you have any questions?