SuperFastPython

7 Common Asyncio Exceptions and Warnings

March 10, 2024 Python Asyncio

There are a series of common exceptions and warnings in asyncio, and we see most of them when we first get started developing asyncio programs.

Knowing about them and how to fix them is important.

In this tutorial, you will discover the 7 most common asyncio warnings and exceptions and how to avoid them.

Let's get started.

Asyncio Common Exceptions and Warnings

The 7 most common asyncio warnings and exceptions are as follows:

  1. RuntimeWarning: Coroutine Was Never Awaited
  2. RuntimeWarning: Enable tracemalloc to get the object allocation traceback
  3. SyntaxError: 'await' outside function
  4. RuntimeError: cannot reuse already awaited coroutine
  5. InvalidStateError: Result is not set
  6. InvalidStateError: Exception is not set
  7. Warning: Asyncio Task exception was never retrieved

Let's take a closer look at each in turn and how to avoid them.

RuntimeWarning: Coroutine Was Never Awaited

It is common for Python developers to get a RuntimeWarning when getting started with asyncio.

The warning looks as follows:

Where the name of the coroutine function is between the quotes.

For example, if you tried to run a coroutine function with the name "custom_coro", then the RuntimeWarning message would look as follows:

Don't worry, you're not alone.

Many developers get this warning.

Why are you getting this warning and how can we fix it?

If you try to run a coroutine and get a RuntimeWarning, the reason is that the coroutine was not executed.

For example, you may have defined a coroutine function as follows:

# define a custom coroutine
async def custom_coro():
    print('Hello there')

You may then try to run the coroutine by calling the coroutine function.

For example:

...
# call the coroutine function
custom_coro()

This will cause the error.

It looks like you are calling the function, but you are not.

The reason is that calling the coroutine function does not run the coroutine.

Instead, it creates a coroutine object.

For example:

...
# create a coroutine object:
coro = custom_coro()

If this coroutine object is not given an opportunity to run, then Python will report the RuntimeWarning.

To run a coroutine object from a regular Python program, you must pass it to the asyncio.run() function.

This will start the asyncio event loop and execute your provided coroutine.

For example:

...
# run the coroutine
asyncio.run(coro)

This is often performed on one line using a compound statement.

For example:

...
# run the coroutine
asyncio.run(custom_coro())

If you are trying to run a coroutine from within a coroutine, then this can be achieved using the "await" expression.

This will suspend the caller coroutine and run the called coroutine.

For example:

...
# suspend and run the other coroutine
await coro

This too is typically performed on one line using a compound statement.

For example:

...
# suspend and run the other coroutine
await custom_coro()

You can learn more about this RuntimeWarning and how to avoid it in the tutorial:

RuntimeWarning: Enable tracemalloc to get the object allocation traceback

It is common to get a RuntimeWarning warning when developing asyncio programs.

This is especially common when you are first getting started. You may see RuntimeWarning messages reported in your standard output or standard error.

A common RuntimeWarning is as follows:

The "RuntimeWarning: Enable tracemalloc to get the object allocation traceback" is emitted by asyncio programs.

Most commonly it is emitted when an asyncio program creates a coroutine and does not await it.

This means that the coroutine is never run in the asyncio event loop.

Creating and not running a coroutine is a common error in an asyncio program and typically raises a RuntimeWarning with the message:

Where '...' is replaced with the name of the coroutine that was not awaited.

Along with this warning, the following warning is emitted:

It is emitted because the Python interpreter is giving us advice on how to track down the cause of the previous RuntimeWarning.

It is suggesting that we use the tools in the tracemalloc module to find the coroutine that was never awaited.

The tracemalloc module provides tools for debugging Python programs.

Specifically, provides tools for tracing memory allocation in the programs.

The tracemalloc module is a debug tool to trace memory blocks allocated by Python.

-- tracemalloc — Trace memory allocations

It is a reasonably sophisticated module and although we may use it directly, we are more likely to use it indirectly via a memory profiling tool applied to our program.

We could instrument our asyncio program with the tracemalloc module to discover the cause of the "RuntimeWarning: coroutine '...' was never awaited" warning, but this is typically not required.

We can fix both RuntimeWarning warnings by awaiting the coroutine that was not awaited.

When the "RuntimeWarning: coroutine '...' was never awaited" is emitted, it will report the file and line on which the coroutine was created that was never awaited.

We can go directly to this line and await the coroutine or schedule the coroutine in a way that is appropriate for our program.

For example, typically we might call a coroutine like a function, causing both RuntimeWarning warnings to be emitted:

...
# call the work coroutine
work()

This will create a coroutine object, but not run it in the event loop.

It is equivalent to the following:

...
# call the work coroutine
coro = work()

We can fix the RuntimeWarning warnings by explicitly awaiting the coroutine.

For example:

...
# await the work coroutine
await work()

You can learn more about this RuntimeWarning and how to avoid it in the tutorial:

SyntaxError: 'await' outside function

It is common to get a SyntaxError referring to "await" when getting started with asyncio.

Specifically, we may be developing an asyncio program and get the error:

The program does not even run.

Instead, the SyntaxError is reported by the Python interpreter before it even begins executing instructions.

Why are we getting this error?

Await refers to the "await" expression used with coroutines.

It can only be used within a coroutine and is used to yield execution to an awaitable.

Await expression: Suspend the execution of coroutine on an awaitable object. Can only be used inside a coroutine function.

-- Python Expressions

An awaitable may be another coroutine or a coroutine wrapped in a Task object for independent execution.

An object that can be used in an await expression. Can be a coroutine or an object with an __await__() method.

-- Python Glossary

Put another way, await will cause the caller coroutine to suspend execution at that point and wait for the given awaitable to be done.

The await expression can be used by using the "await" keyword followed by an awaitable.

For example:

...
# await a coroutine
await custom_coroutine()

This line does a few things.

Firstly, it creates a coroutine object.

It then schedules the coroutine for execution in the asyncio event loop.

The caller then suspends execution and waits for the new coroutine to be done.

The awaitable coroutine may return a value that we may want after the awaitable is done.

This can be assigned as part of the await expression.

For example:

...
# await a coroutine and store the return value
value = await custom_coroutine()

We will get a SyntaxError when we use an "await" expression outside of a coroutine.

For example:

And so on.

We are probably trying to await a coroutine or a task within a function.

This is the most common case.

For example:

# our function
def somefunction():
	# await a coroutine
	await asyncio.sleep(1)

This will result in a SyntaxError.

An await expression can only ever be used within a coroutine.

We can fix the SyntaxError by ensuring that all of our await expressions are within a coroutine.

Recall that we can define a coroutine using the "async def" expression.

For example:

# our function
async def somefunction():
	# await a coroutine
	await asyncio.sleep(1)

This will not result in a SyntaxError.

This means that when developing our asyncio programs all of our functions and methods should be defined using an "async def" expression. They must be coroutines in order to use the await expression.

You can learn more about this SyntaxError and how to avoid it in the tutorial:

RuntimeError: cannot reuse already awaited coroutine

It is common to get a RuntimeError exception with the message:

Developers often get this exception when first getting started with asyncio.

What does this RuntimeError exception mean?

The RuntimeError exception with the message "cannot reuse already awaited coroutine" is caused because we attempt to run the same coroutine object more than once.

Recall that when we call a coroutine, e.g. custom_coro(), that it creates a coroutine object.

We can then run the coroutine object by awaiting it or by scheduling it as a task.

For example:

...
# create a coroutine
coro = custom_coro()
# run the coroutine
await coro

Or:

...
# create a coroutine
coro = custom_coro()
# schedule coroutine for execution later
asyncio.create_task(coro)

If we attempt to run the same coroutine object more than once it will raise an exception:

For example:

...
# create a coroutine
coro = custom_coro()
# run coroutine
await coro
# run coroutine again
await coro

Or:

...
# create a coroutine
coro = custom_coro()
# schedule coroutine for execution later
asyncio.create_task(coro)
# schedule coroutine again for execution later
asyncio.create_task(coro)

We cannot execute the same coroutine object more than once.

We can avoid the RuntimeError by creating coroutines and running them directly in one line.

For example:

...
# create and await coroutine
await custom_coro()

Or

...
# create and schedule coroutine for execution later
asyncio.create_task(custom_coro())

You can learn more about this RuntimeError and how to avoid it in the tutorial:

InvalidStateError: Result is not set

It is common to get an InvalidStateError exception when using asyncio.

An InvalidStateError exception is raised with the message:

Developers often get this exception when first getting started with asyncio.

What does this InvalidStateError exception mean?

We get an InvalidStateError exception when we try to retrieve a return value result from an asyncio.Task while the task is still running.

This will happen if we call the result() method on an asyncio.Task instance and the task is not done.

For example:

...
# attempt to get a result from the task
value = task.result()

This is the regular way to get a return value result from a task.

It will return the return value from the Task's coroutine or re-raise an exception if the coroutine fails with an uncancelled exception.

The problem is, that we must wait until the task is completed, e.g., has the state "done". This means that the done() method on the task returns True.

We can avoid an InvalidStateError when calling the result() method by waiting for the task to be done.

There are many ways we can do this.

The simplest approach is to await the task directly.

For example:

...
# wait for the task to be done and get the result
result = await task

This does two things:

  1. Waits for the task to be done.
  2. Retrieves the return value from the task.

We would no longer need to call the result() method on the task. Nevertheless, we can call the result() method if we prefer.

For example:

...
# wait for the task to be done
await task
# attempt to get a result from the task
value = task.result()

You can learn more about this InvalidStateError and how to avoid it in the tutorial:

InvalidStateError: Exception is not set

It is common to get an InvalidStateError exception when using asyncio.

An InvalidStateError exception is raised with the message:

Developers often get this exception when first getting started with asyncio.

What does this InvalidStateError exception mean?

We get an InvalidStateError exception when we try to retrieve an exception from an asyncio.Task while the task is still running.

This will happen if we call the exception() method on an asyncio.Task instance and the task is not done.

For example:

...
# attempt to get an exception from the task
exc = task.exception()

This is the regular way to get an unhandled exception from a task.

It will return None if the task is completed successfully or an exception instance if an exception was raised in the task.

The problem is, we must wait until the task has been completed, e.g., has the state "done". This means that the done() method on the task returns True.

We can avoid an InvalidStateError when calling the exception() method by waiting for the task to be done.

There are many ways we can do this.

The simplest approach might be to attempt to await the task directly.

For example:

...
# wait for the task to be done
await task
# attempt to get an exception from the task
exc = task.exception()

This is not helpful because if the task fails with an unhandled exception, it will be propagated to the caller and re-raised.

Therefore we would need to wrap the await in a try-except structure.

For example:

try:
	# wait for the task to be done
	await task
except Exception as e:
	...

We would no longer need to call the exception() method on the task.

You can learn more about this InvalidStateError and how to avoid it in the tutorial:

Warning: Asyncio Task exception was never retrieved

A never-retrieved exception is an exception raised in a coroutine or task that is not explicitly retrieved.

If a Future.set_exception() is called but the Future object is never awaited on, the exception would never be propagated to the user code. In this case, asyncio would emit a log message when the Future object is garbage collected.

-- Developing with asyncio

If an exception is not retrieved, then a warning message is reported automatically by the asyncio event loop with the message:

This message is reported along with the details of the task for each never-retrieved exception.

The message is not reported until the event loop is terminated, e.g. the program has ended.

We can avoid this message in our programs by retrieving the exception.

An exception can be retrieved from an asyncio task in one of three ways:

  1. Awaiting the coroutine or task object, the exception is re-raised.
  2. Calling the result() method to get the return value, the exception is re-raised
  3. Calling the exception() method to get the exception.

For example:

# retrieve the exception from the task
exc = task.exception()
# check if the exception is present
if exc:
	# report the exception
	print(exc)

You can learn more about this warning in the tutorial:

Takeaways

You now know about the 7 most common asyncio warnings and exceptions and how to avoid them.