Python Async Keywords

Python asyncio introduced new async keywords to the language to support coroutines.

This includes async expressions such as "async def", "async for", and "async with", as well as the "await" expression. Together, these expressions are referred to as the "async/await" syntax.

In this tutorial, you will discover how to use the new expressions and keywords introduced with asyncio in Python.

After completing this tutorial, you will know:

• How to use "async def" to define coroutines in asyncio programs.
• How to suspend and wait for a target coroutine to complete with the "await" expression.
• How to run asynchronous iterators and context managers with the "async for" and "async with" expressions.

Let's get started.

Python Async Keywords
(Async Expressions)

Asyncio introduces new expressions or keywords into the language specifically for working with coroutines.

The new asyncio expressions are as follows:

1. "async def" for defining coroutines.
2. "async for" for using asynchronous iterators (and generators).
3. "async with" for using asynchronous context managers.
4. "await" for suspending the current coroutine until the target coroutine is done.

Together, the async expressions ("async def", "async for", "async with") and the "await" expression are referred to as "async/await" syntax.

In computer programming, the async/await pattern is a syntactic feature of many programming languages that allows an asynchronous, non-blocking function to be structured in a way similar to an ordinary synchronous function.

-- Async/await, Wikipedia.

Let's take a closer look at each of these asyncio expressions in Python.

"async def" Expression

The "async def" expression defines a coroutine.

Functions defined with async def syntax are always coroutine functions, even if they do not contain await or async keywords.

-- Python Compound statements

A coroutine is a function or routine that can be suspended and resumed. As such it has more than one entry point.

Coroutines are a more generalized form of subroutines. Subroutines are entered at one point and exited at another point. Coroutines can be entered, exited, and resumed at many different points.

-- Python Glossary

The async def defines a coroutine expression that returns a coroutine object.

A coroutine function can be used to create a coroutine that may be used as the entry point for an asyncio program by passing it to the asyncio.run() function.

A coroutine may also be scheduled and awaited by another coroutine or wrapped in a Task object and scheduled independently.

We can use the "async def" to define a coroutine function.

For example:

# define a coroutine
async def custom_coroutine():
	# ..

This is just like defining a function using the def expression.

A defined coroutine may take arguments and return values, just like a function.

For example:

# define a coroutine with arguments and a return value
async def custom_coroutine(arg1, arg2, arg3):
	# ...
	return 100

Calling the defined coroutine will create a coroutine object.

It does not call the coroutine.

For example:

...
# create a coroutine
coro = custom_coroutine()

The coroutine can be executed by using it as the entry point to an asyncio program via the asyncio.run() function.

For example:

...
# create a coroutine
coro = custom_coroutine()
# execute a coroutine program
asyncio.run(coro)

You can learn more about how to use the async def expression in the tutorial:

• How to Use the “async def” Expression in Python

You can learn more about coroutines in the tutorial:

• What is a Coroutine in Python

"async for" Expression

The "async for" expression is used to traverse an asynchronous iterator.

It is an asynchronous for loop statement.

An asynchronous iterator is an iterator that yields awaitables.

asynchronous iterator: An object that implements the __aiter__() and __anext__() methods. __anext__ must return an awaitable object. async for resolves the awaitables returned by an asynchronous iterator's __anext__() method until it raises a StopAsyncIteration exception.

-- Python Glossary

You may recall that an awaitable is an object that can be waited for, such as a coroutine or a task.

awaitable: An object that can be used in an await expression.

-- Python Glossary

An asynchronous generator will automatically implement the asynchronous iterator methods, allowing it to be iterated like an asynchronous iterator.

The await for expression allows the caller to traverse an asynchronous iterator of awaitables and retrieve the result from each.

This is not the same as traversing a collection or list of awaitables (e.g. coroutine objects), instead, the awaitables returned must be provided using the expected asynchronous iterator methods.

Internally, the async for loop will automatically resolve or await each awaitable, scheduling coroutines as needed.

Because it is a for-loop, it assumes, although does not require, that each awaitable being traversed yields a return value.

The async for loop must be used within a coroutine because internally it will use the await expression, which can only be used within coroutines.

The async for expression can be used to traverse an asynchronous iterator within a coroutine.

For example:

...
# traverse an asynchronous iterator
async for item in async_iterator:
	print(item)

This does not execute the for-loop in parallel. The asyncio is unable to execute more than one coroutine at a time within a Python thread.

Instead, this is an asynchronous for-loop.

The difference is that the coroutine that executes the for loop will suspend and internally await for each awaitable.

Behind the scenes, this may require coroutines to be scheduled and awaited, or tasks to be awaited.

We may also use the async for expression in a list comprehension.

For example:

...
# build a list of results
results = 

This would construct a list of return values from the asynchronous iterator.

You can learn more about the "async for" expression in the tutorial:

• How to Use the “async for” Expression in Python

You can learn more about asynchronous iterators in the tutorial:

• Asynchronous Iterators in Python

You can learn more about how to use asynchronous generators in the tutorial:

• Asynchronous Generators in Python

"async with" Expression

The "async with" expression is for creating and using asynchronous context managers.

An asynchronous context manager is a context manager that is able to suspend execution in its enter and exit methods.

-- The async with statement

It is an extension of the "with" expression for use in coroutines within asyncio programs.

The "async with" expression is just like the "with" expression used for context managers, except it allows asynchronous context managers to be used within coroutines.

In order to better understand "async with", let's take a closer look at asynchronous context managers.

An asynchronous context manager must implement the __aenter__() and __aexit__() methods.

It must also only be used within a coroutine.

An asynchronous context manager is a context manager that is able to suspend execution in its enter and exit methods.

-- Asynchronous Context Managers and "async with"

Unlike a traditional context manager, the asynchronous context manager may await when entering and exiting the context manager's block.

This allows the caller to suspend execution and schedule and wait upon coroutines to perform tasks, such as preparing resources required within the block.

This might include opening a socket or preparing a file for use.

An asynchronous context manager must be used via the "async with" expression.

The async with expression allows a coroutine to create and use an asynchronous version of a context manager.

For example:

...
# create and use an asynchronous context manager
async with AsyncContextManager() as manager:
	# ...

This is equivalent to something like:

...
# create or enter the async context manager
manager = await AsyncContextManager()
try:
	# ...
finally:
	# close or exit the context manager
	await manager.close()

Notice that we are implementing much the same pattern as a traditional context manager, except that creating and closing the context manager involves awaiting coroutines.

This suspends the execution of the current coroutine, schedules a new coroutine, and waits for it to complete.

As such an asynchronous context manager must implement the __aenter__() and __aexit__() methods that must be defined via the async def expression. This makes them coroutines themselves which may also await.

You can learn more about asynchronous context managers in the tutorial:

• Asynchronous Context Managers in Python

"await" Expression

In asyncio, "await" is a keyword and expression.

It is used within a coroutine to yield execution to an awaitable.

Await expression: Suspend the execution of coroutine on an awaitable object. Can only be used inside a coroutine function.

-- Python Expressions

An awaitable may be another coroutine or a coroutine wrapped in a Task object for independent execution.

An object that can be used in an await expression. Can be a coroutine or an object with an __await__() method.

-- Python Glossary

Put another way, await will cause the caller coroutine to suspend execution at that point and wait for the given awaitable to be done.

An awaitable is done if it finishes normally, returns explicitly, is canceled or fails with an error or exception.

The await expression can be used by using the "await" keyword followed by an awaitable.

For example:

...
# await a coroutine
await custom_coroutine()

This line does a few things.

Firstly, it creates a coroutine object.

It then schedules the coroutine for execution in the asyncio event loop.

The caller then suspends execution and waits for the new coroutine to be done.

The awaitable coroutine may return a value that we may want after the awaitable is done.

This can be assigned as part of the await expression.

For example:

...
# await a coroutine and store the return value
value = await custom_coroutine()

Another awaitable is a task, that allows the independent execution of a coroutine and provides a handle on its execution.

We can create a task via the asyncio.create_task() function that takes a coroutine, then await the task object directly.

For example:

...
# await a task
await asyncio.create_task(custom_coroutine())

You can learn more about the "await" expression in the tutorial:

• What is an Asyncio Awaitable in Python

Together, async expressions and await expressions are referred to as "async/await" syntax.

You can learn more about async/await syntax in the tutorial:

• What is Async/Await in Python

Takeaways

You now know how to use the new expressions and keywords introduced with asyncio in Python.

If you enjoyed this tutorial, you will love my book: Python Asyncio Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.