You can get the details of silent never-retrieved exceptions in asyncio once the program is terminated.
A custom event loop exception handler can be defined that will be called for each never-retrieved exception, allowing these exceptions to be logged.
Nevertheless, we can avoid never-retrieved exceptions by retrieving and logging them. This can be achieved manually, or automated using a general done callback function.
In this tutorial, you will discover how to report and avoid silent never-retrieved exceptions in asyncio.
Let’s get started.
Asyncio Can Have Silent Exceptions
Exceptions in asyncio programs can be silent.
This can happen if an exception is raised within an asyncio task and never-checked-for or never-retrieved.
As such, these exceptions are referred to as: “never retrieved exceptions” in asyncio circles.
If a Future.set_exception() is called but the Future object is never awaited on, the exception would never be propagated to the user code. In this case, asyncio would emit a log message when the Future object is garbage collected.
— Developing with asyncio
This means that coroutines and tasks may be failing silently in a long-running asyncio program and we may have no visibility of these failures, perhaps until much later.
How can we get the details of silent exceptions in asyncio programs?
How can we avoid “never retrieved exceptions“?
Run loops using all CPUs, download your FREE book to learn how.
How to Expose Never-Retrieved Exceptions
Asyncio does provide means to get the details of never-retrieved exceptions in asyncio programs.
For example, they are reported automatically by the event loop once the program is terminated.
A full trace of never-retrieved exceptions is reported when the asyncio event loop is run in debug mode.
We can run the asyncio event loop in debug mode by passing debug=True to asyncio.run() when starting our program.
Therefore, the two default ways to access never-retrieved exceptions are as follows:
- Report exceptions at the end of the program.
- Report at the end with trace (debug mode)
You can learn more about running the asyncio event loop in debug mode in the tutorial:
How to Avoid Never-Retrieved Exceptions
We can use patterns and idioms to avoid never-retrieved exceptions in our asyncio programs.
This will ensure that exceptions that are raised are handled, or at the very least, logged.
There are three main methods we can use to avoid never-retrieved exceptions in asyncio, they are:
- Event loop exception handler
- Report exceptions via done callback
- Manually check for exceptions
Let’s take a closer look at each in turn.
Use Event Loop Exception Handler
We can configure the event loop to call a custom function for each exception raised in an asyncio program.
This will mean that any never-retrieved exceptions will at least be provided to this handler, allowing us to take action, such as logging them.
The event loop provides the set_exception_handler() method that takes the name of a custom function to call when an exception is raised.
… handler must be a callable with the signature matching (loop, context), where loop is a reference to the active event loop, and context is a dict object containing the details of the exception
— Event Loop API
The handler function provided must have two arguments, the event loop object, and a “context” dict containing a suite of details about the task and exception, including:
- ‘message’: Error message;
- ‘exception’ (optional): Exception object;
- ‘future’ (optional): asyncio.Future instance;
- ‘task’ (optional): asyncio.Task instance;
- ‘handle’ (optional): asyncio.Handle instance;
- ‘protocol’ (optional): Protocol instance;
- ‘transport’ (optional): Transport instance;
- ‘socket’ (optional): socket.socket instance;
The full details of the “context” dict are available in the documentation for the loop.call_exception_handler() function.
There are two important details about this approach:
- The handler is only called for unhandled exceptions, e.g. exceptions that bubble up to the top level causing the task to terminate.
- The handler is only called for exceptions that are never-retrieved via the Task.exception() method.
- The handler is only called after the program has terminated, not in real-time while the program is exiting, at least according to my testing.
For example, we can define a handler:
|
1 2 3 4 |
# define an exception handler def exception_handler(loop, context): # log exception print(context['exception']) |
We can then configure the event loop to use this handler when exceptions are raised.
This should be done shortly after starting the asyncio program.
|
1 2 3 4 5 |
... # get the event loop loop = asyncio.get_running_loop() # set the exception handler loop.set_exception_handler(exception_handler) |
Report Exceptions Via Done Callback Function
We can automatically check for and report exceptions in tasks using a generally done callback function.
This done callback function can be added to every task we create and schedule in our asyncio program.
We could do this manually, or define a helper function that creates tasks for us and adds the callback automatically.
You can learn more about done callback functions in the tutorial:
For example, we can define a done callback function that (safely) retrieves the exception and reports it if present.
Remember, a task may be canceled, in which case the CancelledError will be propagated if we try to get the exception.
|
1 2 3 4 5 6 7 8 9 10 11 |
# callback func called for all tasks def helper_done_callback(task): try: # get any exception raised ex = task.exception() # check task for exception if ex: # report the exception print(ex) except asyncio.exceptions.CancelledError: pass |
We can then define a helper function that will create a task for us and add the done callback.
For example:
|
1 2 3 4 5 6 7 8 |
# helper for creating all tasks def create_task_helper(coroutine): # wrap and schedule the task task = asyncio.create_task(coroutine) # add custom callback to task task.add_done_callback(helper_done_callback) # return the task that was created return task |
We can then use this helper function to create tasks in our program.
For example:
|
1 2 3 |
... # run the task task = create_task_helper(work()) |
Manually Check For Exceptions
A final solution to avoiding never-retrieved exceptions in tasks is to retrieve them.
We can manually retrieve and check exceptions in asyncio tasks before handling any results.
This can be achieved by calling the exception() function.
Again, it is good practice to handle task cancellation.
|
1 2 3 4 5 6 7 8 |
... try: # check for exception exp = task.exception() if exp is not None: print(f'Task failed with: {exp}') except asyncio.exceptions.CancelledError: pass |
You can learn more about checking for exceptions in the tutorial:
Now that we know how to report and avoid never-retrieved exceptions in asyncio, let’s look at some worked examples.
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Example of No Exceptions Until Program Ends
We can explore the case where never-retrieved exceptions are reported automatically after the program ends.
In this example, we will define a coroutine that sleeps one second and then raises an exception.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
# task that does work async def work(): # block for a moment await asyncio.sleep(1) # raise a problem raise Exception('Something Bad Happened') We will then run this task in the background, sleep, then report a done message. # main coroutine async def main(): # report a message print('Starting') # run the task _ = asyncio.create_task(work()) # block for a moment await asyncio.sleep(2) # report a message print('Done') |
The expectation is that the exception in the work() coroutine is never checked and will not interfere with the main() coroutine, and will be reported once the program terminates.
This is the default behavior in asyncio programs.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
# SuperFastPython.com # example of unhandled exception not reported until the end import asyncio # task that does work async def work(): # block for a moment await asyncio.sleep(1) # raise a problem raise Exception('Something Bad Happened') # main coroutine async def main(): # report a message print('Starting') # run the task _ = asyncio.create_task(work()) # block for a moment await asyncio.sleep(2) # report a message print('Done') # start asyncio.run(main()) |
Running the program executes the main() coroutine, then executes the work() coroutine.
The work() coroutine runs, sleeps, and raises an exception. This exception is never retrieved, remaining silent.
The main() coroutine sleeps, reports a final message, then the program terminates.
During the shutdown of the asyncio event loop, the details of the never-retrieved exception are reported.
This highlights the default behavior of the asyncio event loop to report never-retrieved exceptions once the program has terminated.
|
1 2 3 4 5 6 7 8 |
Starting Done Task exception was never retrieved future: <Task finished name='Task-2' coro=<work() done, defined at ...> exception=Exception('Something Bad Happened')> Traceback (most recent call last): File "...", line 10, in work raise Exception('Something Bad Happened') Exception: Something Bad Happened |
Next, let’s explore similar behavior when running the event loop in debug mode.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Exception Trace At End With Debug Mode
The asyncio event loop can be run in debug by setting debug=True in the call to asyncio.run() when starting the program.
This will run extra checks and perform extra logging when running the program.
Helpfully, asyncio will report the full stack trace of each never-retrieved exception after the program terminates, when the event loop is run in debug mode.
We can explore this in an example.
In this case, we can update the previous example to run the event loop in debug mode.
|
1 2 3 |
... # start asyncio.run(main(), debug=True) |
And that’s it.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
# SuperFastPython.com # example of trace of unhandled exception show in debug mode import asyncio # task that does work async def work(): # block for a moment await asyncio.sleep(1) # raise a problem raise Exception('Something Bad Happened') # main coroutine async def main(): # report a message print('Starting') # run the task _ = asyncio.create_task(work()) # block for a moment await asyncio.sleep(2) # report a message print('Done') # start asyncio.run(main(), debug=True) |
Running the program starts the event loop in debug mode.
The main() coroutine is run, then executes the work() coroutine.
The work() coroutine runs, sleeps, and raises an exception. This exception is never retrieved, remaining silent.
The main() coroutine sleeps, reports a final message, then the program terminates.
During the shutdown of the asyncio event loop, the details of the never-retrieved exception are reported, as in the previous example. In addition, the full stack trace of the exception is reported, showing the line that caused the failure.
This highlights the additional details provided about never-retrieved exceptions when running the asyncio event loop in debug mode.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
Starting Done Task exception was never retrieved future: <Task finished name='Task-2' coro=<work() done, defined at ...:6> exception=Exception('Something Bad Happened') created at .../asyncio/tasks.py:374> source_traceback: Object created at (most recent call last): File "...", line 24, in <module> asyncio.run(main(), debug=True) File ".../asyncio/runners.py", line 190, in run return runner.run(main) File ".../asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) File ".../asyncio/base_events.py", line 640, in run_until_complete self.run_forever() File ".../asyncio/base_events.py", line 607, in run_forever self._run_once() File ".../asyncio/base_events.py", line 1914, in _run_once handle._run() File ".../asyncio/events.py", line 80, in _run self._context.run(self._callback, *self._args) File "...", line 17, in main _ = asyncio.create_task(work()) File ".../asyncio/tasks.py", line 374, in create_task task = loop.create_task(coro) Traceback (most recent call last): File "...", line 10, in work raise Exception('Something Bad Happened') Exception: Something Bad Happened |
Next, let’s look at how we might always log exceptions, regardless of whether they were retrieved.
Example of Event Loop Exception Handler
The asyncio event loop provides the set_exception_handler() to register a handler that will be called for all exceptions raised in the program.
In this example, we will first define a handler function that will retrieve the exception and report its details.
|
1 2 3 4 5 6 |
# define an exception handler def exception_handler(loop, context): # get the exception ex = context['exception'] # log details print(f'Got exception {ex}') |
Then, early in our program before we start real work, we can retrieve the event loop and configure our exception handler function.
|
1 2 3 4 5 |
... # get the event loop loop = asyncio.get_running_loop() # set the exception handler loop.set_exception_handler(exception_handler) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
# SuperFastPython.com # example of using an event loop exception handler import asyncio # define an exception handler def exception_handler(loop, context): # get the exception ex = context['exception'] # log details print(f'Got exception {ex}') # task that does work async def work(): # block for a moment await asyncio.sleep(1) # raise a problem raise Exception('Something Bad Happened') # main coroutine async def main(): # get the event loop loop = asyncio.get_running_loop() # set the exception handler loop.set_exception_handler(exception_handler) # report a message print('Starting') # run the task _ = asyncio.create_task(work()) # block for a moment await asyncio.sleep(2) # report a message print('Done') # start asyncio.run(main()) |
Running the program executes the main() coroutine and configures our custom exception handler.
The main() coroutine then executes the work() coroutine. The work() coroutine runs, sleeps, and raises an unhandled exception. This exception is never retrieved, remaining silent.
The main() coroutine sleeps, reports a final message, then the program terminates.
During the shutdown of the asyncio event loop, our custom exception handler is called, reporting the details of the never-retrieved exception.
This highlights that we can control how never-retrieved exceptions are handled with a custom function that is called when the event loop is being shut down.
|
1 2 3 |
Starting Done Got exception Something Bad Happened |
Next, let’s look at how we might log exceptions automatically with a done callback function.
Example of Reporting Exceptions Via Done Callback Function
We can explore how to automatically log task exceptions using a done callback function.
We can define a done callback function that checks for an exception and logs it if present. It can also handle the case if the task was canceled.
|
1 2 3 4 5 6 7 8 9 10 11 |
# callback func called for all tasks def helper_done_callback(task): try: # get any exception raised ex = task.exception() # check task for exception if ex: # report the exception print(ex) except asyncio.exceptions.CancelledError: pass |
We can then define a helper function used to create tasks, that always adds the done callback function to the created tasks.
|
1 2 3 4 5 6 7 8 |
# helper for creating all tasks def create_task_helper(coroutine): # wrap and schedule the task task = asyncio.create_task(coroutine) # add custom callback to task task.add_done_callback(helper_done_callback) # return the task that was created return task |
We can then use this helper in our program to create tasks for execution.
|
1 2 3 |
... # run the task task = create_task_helper(work()) |
This means that any tasks created using this helper will have their exceptions logged automatically.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
# SuperFastPython.com # example of reporting exceptions via done callback import asyncio # callback func called for all tasks def helper_done_callback(task): try: # get any exception raised ex = task.exception() # check task for exception if ex: # report the exception print(ex) except asyncio.exceptions.CancelledError: pass # helper for creating all tasks def create_task_helper(coroutine): # wrap and schedule the task task = asyncio.create_task(coroutine) # add custom callback to task task.add_done_callback(helper_done_callback) # return the task that was created return task # task that does work async def work(): # block for a moment await asyncio.sleep(1) # raise a problem raise Exception('Something Bad Happened') # main coroutine async def main(): # report a message print('Starting') # run the task task = create_task_helper(work()) # block for a moment await asyncio.sleep(2) # report a message print('Done') # start asyncio.run(main()) |
Running the program executes the main() coroutine and reports a message.
The main() coroutine then creates and schedules the work() coroutine using the helper function. The helper function creates the task and adds the custom done callback to the task.
The work() coroutine runs, sleeps, and raises an unhandled exception. After the task is completed, the custom callback function is run, retrieving the exception and reporting its details immediately.
The main() coroutine sleeps, reports a final message, then the program terminates.
This highlights how we can report exceptions as soon as tasks are completed using a done callback function, rather than waiting for the program to terminate, in the case of an event loop exception handler.
|
1 2 3 |
Starting Something Bad Happened Done |
Next, we can explore an example of manually checking tasks for exceptions.
Example of Manually Checking for Exceptions
We can manually check a task for an exception.
This avoids never-retrieved exceptions by explicitly retrieving them in our program, perhaps the simplest solution.
The problem with this approach is that we may forget to check. This is why we may prefer one of the automated methods described above.
We can manually check for an exception after the task is completed.
Calling the Task.exception() method raises an InvalidStateError exception if the task is not done, and will raise a CancelledError exception if the task was canceled. It may be a good practice to check for these.
|
1 2 3 4 5 6 7 8 9 10 |
... try: # check for exception exp = task.exception() if exp is not None: print(f'Task failed with: {exp}') except asyncio.CancelledError: pass except asyncio.InvalidStateError: pass |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
# SuperFastPython.com # example of manually retrieving exception from task import asyncio # task that does work async def work(): # block for a moment await asyncio.sleep(1) # raise a problem raise Exception('Something Bad Happened') # main coroutine async def main(): # report a message print('Starting') # run the task task = asyncio.create_task(work()) # block for a moment await asyncio.sleep(2) try: # check for exception exp = task.exception() if exp is not None: print(f'Task failed with: {exp}') except asyncio.CancelledError: pass except asyncio.InvalidStateError: pass # report a message print('Done') # start asyncio.run(main()) |
Running the program executes the main() coroutine and reports a message.
The main() coroutine then creates and schedules the work() task. The work() coroutine runs, sleeps, and raises an unhandled exception.
The main() coroutine sleeps, then explicitly retrieves the exception from the task, reporting its details.
It then reports the final message and terminates.
This highlights how we can explicitly and safely retrieve an exception from a task.
|
1 2 3 |
Starting Task failed with: Something Bad Happened Done |
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know how to report and avoid silent never-retrieved exceptions in asyncio.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any additional tips?
I’d love to hear about them!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Claudio Schwarz on Unsplash
Do you have any questions?