You can get an InvalidStateError exception when attempting to retrieve an exception from an asyncio Task.
This will happen if we attempt to retrieve an exception from a task while the task is still running, e.g. the task is not yet done.
We can avoid the InvalidStateError exception by waiting for the task to be done before attempting to retrieve the exception either directly or via a function such as asyncio.gather() or asyncio.wait().
In this tutorial, you will discover why we get the InvalidStateError: “Exception is not set” and how to fix it.
Let’s get started.
InvalidStateError: Exception is not set
It is common to get an InvalidStateError exception when using asyncio.
An InvalidStateError exception is raised with the message:
- InvalidStateError: Exception is not set
Developers often get this exception when first getting started with asyncio.
What does this InvalidStateError exception mean?
How can we fix it and avoid it in the future?
Run loops using all CPUs, download your FREE book to learn how.
Why Do We Get An InvalidStateError
We get an InvalidStateError exception when we try to retrieve an exception from an asyncio.Task while the task is still running.
This will happen if we call the exception() method on an asyncio.Task instance and the task is not done.
For example:
|
1 2 3 |
... # attempt to get an exception from the task exc = task.exception() |
This is the regular way to get an unhandled exception from a task.
It will return None if the task is completed successfully or an exception instance if an exception was raised in the task.
The problem is, we must wait until the task has been completed, e.g., has the state “done“. This means that the done() method on the task returns True.
You can learn more about retrieving exceptions from tasks in the tutorial:
How to Avoid InvalidStateError
We can avoid an InvalidStateError when calling the exception() method by waiting for the task to be done.
There are many ways we can do this, three examples include:
- Await the task directly.
- Await the task via asyncio.gather()
- Await the task via asyncio.wait()
Let’s take a closer look at each approach in turn.
Await the Task Directly
The naive approach might be to attempt to await the task directly.
For example:
|
1 2 3 4 5 |
... # wait for the task to be done await task # attempt to get an exception from the task exc = task.exception() |
This is not helpful because if the task fails with an unhandled exception, it will be propagated to the caller and re-raised.
Therefore we would need to wrap the await in a try-except structure.
For example:
|
1 2 3 4 5 |
try: # wait for the task to be done await task except Exception as e: ... |
We would no longer need to call the exception() method on the task.
Await the Task Via asyncio.gather()
An approach that maintains the same code structure is to use the asyncio.gather() function.
This function takes one or more awaitables and returns once they are all done. We can set the “return_exceptions” argument to True so that any exceptions raised in awaited tasks are handled and returned directly in the list of return values, instead of being propagated to the caller.
For example:
|
1 2 3 4 5 |
... # wait for the task to be done _ = await asyncio.gather(task, return_exceptions=True) # attempt to get an exception from the task exc = task.exception() |
You can learn more about how to use the asyncio.gather() function in the tutorial:
Await the Task Via asyncio.wait()
Another similar approach is to use the asyncio.wait() function.
This function takes a collection of tasks and returns once a condition is met, where the default condition is that all provided tasks are done.
This means that we will have to enclose our single task in a list.
For example:
|
1 2 3 4 5 |
... # wait for the task to be done _ = await asyncio.wait([task]) # attempt to get an exception from the task exc = task.exception() |
You can learn more about the asyncio.wait() function in the tutorial:
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Example of InvalidStateError: Exception is not set
We can explore an example that raises an InvalidStateError.
In this case, we can define a custom coroutine that does some work and then raises an exception. Our main program will schedule the coroutine as a background task, sleep a moment then attempt to retrieve the exception and report a message if an exception was present.
Firstly, we can define a custom coroutine that reports a message, sleeps for one second, and then fails with an exception.
The work() coroutine below implements this.
|
1 2 3 4 5 6 7 8 |
# custom coroutine async def work(): # report a message print('work() is working!') # sleep a moment to simulate work await asyncio.sleep(1) # fails with an exception raise Exception('Something bad happened') |
Next, we can define the main() coroutine.
A message is printed, then a work() coroutine is created and scheduled as a background task. The main() coroutine then sleeps for half a second, not long enough for the task to be completed.
It then retrieves the exception and if present reports its details, otherwise reports that everything is fine before reporting a final message.
The main() coroutine below implements this.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
# main coroutine async def main(): # entry point of the program print('Main is running') # schedule as task task = asyncio.create_task(work()) # wait a moment await asyncio.sleep(0.5) # check for an exception if task.exception(): # report the exception print(f'Failed with: {task.exception()}') else: print('Everything is fine') # report final message print('Main is done') |
Finally, we can start the event loop.
|
1 2 3 |
... # start the event loop asyncio.run(main()) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# SuperFastPython.com # example of InvalidStateError: Exception is not set import asyncio # custom coroutine async def work(): # report a message print('work() is working!') # sleep a moment to simulate work await asyncio.sleep(1) # fails with an exception raise Exception('Something bad happened') # main coroutine async def main(): # entry point of the program print('Main is running') # schedule as task task = asyncio.create_task(work()) # wait a moment await asyncio.sleep(0.5) # check for an exception if task.exception(): # report the exception print(f'Failed with: {task.exception()}') else: print('Everything is fine') # report final message print('Main is done') # start the event loop asyncio.run(main()) |
Running the example first starts the asyncio event loop and runs the main() coroutine.
The main() coroutine runs and reports a message.
It then creates the work() coroutine and schedules it as a background task. It then suspends and sleeps for half a second.
The work() task runs and reports a message before suspending and sleeping for one second.
The main() coroutine resumes and attempts to retrieve the exception from the task.
This fails with an InvalidStateError exception and the message “Exception is not set“.
This highlights that if we attempt to retrieve an exception via the exception() method before the task is done an InvalidStateError exception will be raised.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
Main is running work() is working! Traceback (most recent call last): File "...", line 32, in <module> asyncio.run(main()) File ".../asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File ".../asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../asyncio/base_events.py", line 653, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "...", line 23, in main if task.exception(): ^^^^^^^^^^^^^^^^ asyncio.exceptions.InvalidStateError: Exception is not set. |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Avoiding InvalidStateError By Awaiting The Task
We can explore how to avoid the InvalidStateError exception by awaiting the target task directly.
In this case, we can update the above example to await the task. This requires that we wrap the await expression in a try-except structure. If an exception occurs, we can report the details directly, otherwise report an everything is fine message.
|
1 2 3 4 5 6 7 8 9 |
... try: # await the task directly await task except Exception as e: # report the exception print(f'Failed with: {e}') else: print('Everything is fine') |
The updated main() coroutine with this change is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
# main coroutine async def main(): # entry point of the program print('Main is running') # schedule as task task = asyncio.create_task(work()) try: # await the task directly await task except Exception as e: # report the exception print(f'Failed with: {e}') else: print('Everything is fine') # report final message print('Main is done') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# SuperFastPython.com # example of avoiding InvalidStateError by awaiting the task directly import asyncio # custom coroutine async def work(): # report a message print('work() is working!') # sleep a moment to simulate work await asyncio.sleep(1) # fails with an exception raise Exception('Something bad happened') # main coroutine async def main(): # entry point of the program print('Main is running') # schedule as task task = asyncio.create_task(work()) try: # await the task directly await task except Exception as e: # report the exception print(f'Failed with: {e}') else: print('Everything is fine') # report final message print('Main is done') # start the event loop asyncio.run(main()) |
Running the example first starts the asyncio event loop and runs the main() coroutine.
The main() coroutine runs and reports a message.
It then creates the work() coroutine and schedules it as a background task. It then suspends and awaits the task directly.
The work() task runs and reports a message before suspending and sleeping for one second.
The work() task then resumes and raises an exception, terminating the task.
The main() coroutine resumes and the exception is propagated. The exception is handled and reported directly.
This highlights how we can avoid an InvalidStateError exception by awaiting the task directly and handling any raised exception.
|
1 2 3 4 |
Main is running work() is working! Failed with: Something bad happened Main is done |
Example of Avoiding InvalidStateError By asyncio.gather()
We can explore how to avoid an InvalidStateError exception by awaiting the task with asyncio.gather().
In this case, we can update the example by awaiting a call to asyncio.gather() passing the task instance as an argument, and setting the return_exceptions argument to True.
|
1 2 3 |
... # wait for the task to be done _ = await asyncio.gather(task, return_exceptions=True) |
The benefit of this approach is that we do not have to handle the exception in the caller, it will not be propagated from the asyncio.gather() when the return_exceptions argument is set to True.
The updated main() coroutine with this change is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
# main coroutine async def main(): # entry point of the program print('Main is running') # schedule as task task = asyncio.create_task(work()) # wait for the task to be done _ = await asyncio.gather(task, return_exceptions=True) # check for an exception if task.exception(): # report the exception print(f'Failed with: {task.exception()}') else: print('Everything is fine') # report final message print('Main is done') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# SuperFastPython.com # example of avoiding InvalidStateError by asyncio.gather() import asyncio # custom coroutine async def work(): # report a message print('work() is working!') # sleep a moment to simulate work await asyncio.sleep(1) # fails with an exception raise Exception('Something bad happened') # main coroutine async def main(): # entry point of the program print('Main is running') # schedule as task task = asyncio.create_task(work()) # wait for the task to be done _ = await asyncio.gather(task, return_exceptions=True) # check for an exception if task.exception(): # report the exception print(f'Failed with: {task.exception()}') else: print('Everything is fine') # report final message print('Main is done') # start the event loop asyncio.run(main()) |
Running the example first starts the asyncio event loop and runs the main() coroutine.
The main() coroutine runs and reports a message.
It then creates the work() coroutine and schedules it as a background task.
It then suspends and awaits the task in a call to asyncio.gather().
The work() task runs and reports a message before suspending and sleeping for one second.
The work() task then resumes and raises an exception, terminating the task.
The main() coroutine resumes and checks if the task raised an exception. The task did raise an exception, so the exception details are retrieved and reported directly.
This highlights how we can avoid an InvalidStateError exception by awaiting the task within asyncio.gather().
|
1 2 3 4 |
Main is running work() is working! Failed with: Something bad happened Main is done |
Example of Avoiding InvalidStateError By asyncio.wait()
We can explore how to avoid an InvalidStateError exception by awaiting the task with asyncio.wait().
In this case, we can update the example by awaiting a call to asyncio.wait() and passing the task instance as an argument wrapped in a new list object.
|
1 2 3 |
... # wait for the task to be done _ = await asyncio.wait([task]) |
The benefit of this approach is that we do not have to handle the exception in the caller, it will not be propagated from the asyncio.wait() function.
The updated main() coroutine with this change is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
# main coroutine async def main(): # entry point of the program print('Main is running') # schedule as task task = asyncio.create_task(work()) # wait for the task to be done _ = await asyncio.wait([task]) # check for an exception if task.exception(): # report the exception print(f'Failed with: {task.exception()}') else: print('Everything is fine') # report final message print('Main is done') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# SuperFastPython.com # example of avoiding InvalidStateError by asyncio.wait() import asyncio # custom coroutine async def work(): # report a message print('work() is working!') # sleep a moment to simulate work await asyncio.sleep(1) # fails with an exception raise Exception('Something bad happened') # main coroutine async def main(): # entry point of the program print('Main is running') # schedule as task task = asyncio.create_task(work()) # wait for the task to be done _ = await asyncio.wait([task]) # check for an exception if task.exception(): # report the exception print(f'Failed with: {task.exception()}') else: print('Everything is fine') # report final message print('Main is done') # start the event loop asyncio.run(main()) |
Running the example first starts the asyncio event loop and runs the main() coroutine.
The main() coroutine runs and reports a message.
It then creates the work() coroutine and schedules it as a background task.
It then suspends and awaits the task in a call to asyncio.wait() as a one-item list.
The work() task runs and reports a message before suspending and sleeping for one second.
The work() task then resumes and raises an exception, terminating the task.
The main() coroutine resumes and checks if the task raised an exception. The task did raise an exception, so the exception details are retrieved and reported directly.
This highlights how we can avoid an InvalidStateError exception by awaiting the task within asyncio.wait().
|
1 2 3 4 |
Main is running work() is working! Failed with: Something bad happened Main is done |
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know why we get the InvalidStateError: “Exception is not set” and how to fix it.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any additional tips?
I’d love to hear about them!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Dylan Gillis on Unsplash
Do you have any questions?