Python asyncio introduced new async keywords to the language to support coroutines.
This includes async expressions such as “async def“, “async for“, and “async with“, as well as the “await” expression. Together, these expressions are referred to as the “async/await” syntax.
In this tutorial, you will discover how to use the new expressions and keywords introduced with asyncio in Python.
After completing this tutorial, you will know:
- How to use “async def” to define coroutines in asyncio programs.
- How to suspend and wait for a target coroutine to complete with the “await” expression.
- How to run asynchronous iterators and context managers with the “async for” and “async with” expressions.
Let’s get started.
Python Async Keywords
(Async Expressions)
Asyncio introduces new expressions or keywords into the language specifically for working with coroutines.
The new asyncio expressions are as follows:
- “async def” for defining coroutines.
- “async for” for using asynchronous iterators (and generators).
- “async with” for using asynchronous context managers.
- “await” for suspending the current coroutine until the target coroutine is done.
Together, the async expressions (“async def“, “async for“, “async with“) and the “await” expression are referred to as “async/await” syntax.
In computer programming, the async/await pattern is a syntactic feature of many programming languages that allows an asynchronous, non-blocking function to be structured in a way similar to an ordinary synchronous function.
— Async/await, Wikipedia.
Let’s take a closer look at each of these asyncio expressions in Python.
Run loops using all CPUs, download your FREE book to learn how.
“async def” Expression
The “async def” expression defines a coroutine.
Functions defined with async def syntax are always coroutine functions, even if they do not contain await or async keywords.
— Python Compound statements
A coroutine is a function or routine that can be suspended and resumed. As such it has more than one entry point.
Coroutines are a more generalized form of subroutines. Subroutines are entered at one point and exited at another point. Coroutines can be entered, exited, and resumed at many different points.
— Python Glossary
The async def defines a coroutine expression that returns a coroutine object.
A coroutine function can be used to create a coroutine that may be used as the entry point for an asyncio program by passing it to the asyncio.run() function.
A coroutine may also be scheduled and awaited by another coroutine or wrapped in a Task object and scheduled independently.
We can use the “async def” to define a coroutine function.
For example:
|
1 2 3 |
# define a coroutine async def custom_coroutine(): # .. |
This is just like defining a function using the def expression.
A defined coroutine may take arguments and return values, just like a function.
For example:
|
1 2 3 4 |
# define a coroutine with arguments and a return value async def custom_coroutine(arg1, arg2, arg3): # ... return 100 |
Calling the defined coroutine will create a coroutine object.
It does not call the coroutine.
For example:
|
1 2 3 |
... # create a coroutine coro = custom_coroutine() |
The coroutine can be executed by using it as the entry point to an asyncio program via the asyncio.run() function.
For example:
|
1 2 3 4 5 |
... # create a coroutine coro = custom_coroutine() # execute a coroutine program asyncio.run(coro) |
You can learn more about how to use the async def expression in the tutorial:
You can learn more about coroutines in the tutorial:
“async for” Expression
The “async for” expression is used to traverse an asynchronous iterator.
It is an asynchronous for loop statement.
An asynchronous iterator is an iterator that yields awaitables.
asynchronous iterator: An object that implements the __aiter__() and __anext__() methods. __anext__ must return an awaitable object. async for resolves the awaitables returned by an asynchronous iterator’s __anext__() method until it raises a StopAsyncIteration exception.
— Python Glossary
You may recall that an awaitable is an object that can be waited for, such as a coroutine or a task.
awaitable: An object that can be used in an await expression.
— Python Glossary
An asynchronous generator will automatically implement the asynchronous iterator methods, allowing it to be iterated like an asynchronous iterator.
The await for expression allows the caller to traverse an asynchronous iterator of awaitables and retrieve the result from each.
This is not the same as traversing a collection or list of awaitables (e.g. coroutine objects), instead, the awaitables returned must be provided using the expected asynchronous iterator methods.
Internally, the async for loop will automatically resolve or await each awaitable, scheduling coroutines as needed.
Because it is a for-loop, it assumes, although does not require, that each awaitable being traversed yields a return value.
The async for loop must be used within a coroutine because internally it will use the await expression, which can only be used within coroutines.
The async for expression can be used to traverse an asynchronous iterator within a coroutine.
For example:
|
1 2 3 4 |
... # traverse an asynchronous iterator async for item in async_iterator: print(item) |
This does not execute the for-loop in parallel. The asyncio is unable to execute more than one coroutine at a time within a Python thread.
Instead, this is an asynchronous for-loop.
The difference is that the coroutine that executes the for loop will suspend and internally await for each awaitable.
Behind the scenes, this may require coroutines to be scheduled and awaited, or tasks to be awaited.
We may also use the async for expression in a list comprehension.
For example:
|
1 2 3 |
... # build a list of results results = [item async for item async_iterator] |
This would construct a list of return values from the asynchronous iterator.
You can learn more about the “async for” expression in the tutorial:
You can learn more about asynchronous iterators in the tutorial:
You can learn more about how to use asynchronous generators in the tutorial:
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
“async with” Expression
The “async with” expression is for creating and using asynchronous context managers.
An asynchronous context manager is a context manager that is able to suspend execution in its enter and exit methods.
— The async with statement
It is an extension of the “with” expression for use in coroutines within asyncio programs.
The “async with” expression is just like the “with” expression used for context managers, except it allows asynchronous context managers to be used within coroutines.
In order to better understand “async with“, let’s take a closer look at asynchronous context managers.
An asynchronous context manager must implement the __aenter__() and __aexit__() methods.
It must also only be used within a coroutine.
An asynchronous context manager is a context manager that is able to suspend execution in its enter and exit methods.
— Asynchronous Context Managers and “async with”
Unlike a traditional context manager, the asynchronous context manager may await when entering and exiting the context manager’s block.
This allows the caller to suspend execution and schedule and wait upon coroutines to perform tasks, such as preparing resources required within the block.
This might include opening a socket or preparing a file for use.
An asynchronous context manager must be used via the “async with” expression.
The async with expression allows a coroutine to create and use an asynchronous version of a context manager.
For example:
|
1 2 3 4 |
... # create and use an asynchronous context manager async with AsyncContextManager() as manager: # ... |
This is equivalent to something like:
|
1 2 3 4 5 6 7 8 |
... # create or enter the async context manager manager = await AsyncContextManager() try: # ... finally: # close or exit the context manager await manager.close() |
Notice that we are implementing much the same pattern as a traditional context manager, except that creating and closing the context manager involves awaiting coroutines.
This suspends the execution of the current coroutine, schedules a new coroutine, and waits for it to complete.
As such an asynchronous context manager must implement the __aenter__() and __aexit__() methods that must be defined via the async def expression. This makes them coroutines themselves which may also await.
You can learn more about asynchronous context managers in the tutorial:
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
“await” Expression
In asyncio, “await” is a keyword and expression.
It is used within a coroutine to yield execution to an awaitable.
Await expression: Suspend the execution of coroutine on an awaitable object. Can only be used inside a coroutine function.
— Python Expressions
An awaitable may be another coroutine or a coroutine wrapped in a Task object for independent execution.
An object that can be used in an await expression. Can be a coroutine or an object with an __await__() method.
— Python Glossary
Put another way, await will cause the caller coroutine to suspend execution at that point and wait for the given awaitable to be done.
An awaitable is done if it finishes normally, returns explicitly, is canceled or fails with an error or exception.
The await expression can be used by using the “await” keyword followed by an awaitable.
For example:
|
1 2 3 |
... # await a coroutine await custom_coroutine() |
This line does a few things.
Firstly, it creates a coroutine object.
It then schedules the coroutine for execution in the asyncio event loop.
The caller then suspends execution and waits for the new coroutine to be done.
The awaitable coroutine may return a value that we may want after the awaitable is done.
This can be assigned as part of the await expression.
For example:
|
1 2 3 |
... # await a coroutine and store the return value value = await custom_coroutine() |
Another awaitable is a task, that allows the independent execution of a coroutine and provides a handle on its execution.
We can create a task via the asyncio.create_task() function that takes a coroutine, then await the task object directly.
For example:
|
1 2 3 |
... # await a task await asyncio.create_task(custom_coroutine()) |
You can learn more about the “await” expression in the tutorial:
Together, async expressions and await expressions are referred to as “async/await” syntax.
You can learn more about async/await syntax in the tutorial:
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know how to use the new expressions and keywords introduced with asyncio in Python.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any additional tips?
I’d love to hear about them!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Charles DeLoye on Unsplash
Do you have any questions?