Last Updated on November 14, 2023

Asyncio is new and challenging to understand for beginners.

The reason is because it requires a different way of thinking.

Asyncio programs are not like regular Python programs where we might have a linear sequence of steps where we call functions and object methods. Instead, asyncio programs use an asynchronous programming paradigm.

Asyncio programs are also different from other types of concurrency in Python, such as threading and multiprocessing. Instead, concurrency is achieved using coroutines that are baked into the language itself.

In this tutorial, you will discover what exactly asyncio is all about.

After completing this tutorial, you will know:

• That asyncio really refers to two things, support for coroutines and the asyncio module in the standard library.
• What is asynchronous programming and how asyncio provides this capability in Python.
• When you should consider using asyncio in your Python programs.

Let’s get started.

What is Asyncio

Broadly, asyncio refers to the ability to implement asynchronous programming in Python.

Specifically, it refers to two elements:

1. The addition of the “asyncio” module to the Python standard library in Python 3.4.
2. The addition of async/await expressions to the Python language in Python 3.5.

Together, the module and changes to the language facilitate the development of Python programs that support coroutine-based concurrency, non-blocking I/O, and asynchronous programming.

Python 3.4 introduced the asyncio library, and Python 3.5 produced the async and await keywords to use it palatably. These new additions allow so-called asynchronous programming.

— Page vii, Using Asyncio in Python, 2020.

Let’s take a closer look at these two aspects of asyncio, starting with the changes to the language.

Changes to Python to add Support for Coroutines

The Python language was changed to accommodate asyncio with the addition of expressions and types.

More specifically, it was changed to support coroutines as first-class concepts. In turn, coroutines are the unit of concurrency used in asyncio programs.

A coroutine is a function that can be suspended and resumed.

coroutine: Coroutines are a more generalized form of subroutines. Subroutines are entered at one point and exited at another point. Coroutines can be entered, exited, and resumed at many different points.

— Python Glossary

A coroutine may suspend for many reasons, such as executing another coroutine, e.g. awaiting another task, or waiting for some external resources, such as a socket connection or process to return data.

Many coroutines can be created and executed at the same time. They have control over when they will suspend and resume, allowing them to cooperate as to when concurrent tasks are executed.

This is called cooperative multitasking and is different from the multitasking typically used with threads called preemptive multitasking tasking.

A coroutine can be defined via the “async def” expression. It can take arguments and return a value, just like a function.

For example:

Calling a coroutine function will create a coroutine object, this is a new class. It does not execute the coroutine function.

A coroutine can execute another coroutine via the await expression.

This suspends the caller and schedules the target for execution.

An asynchronous iterator is an iterator that yields awaitables.

asynchronous iterator: An object that implements the __aiter__() and __anext__() methods. __anext__ must return an awaitable object. async for resolves the awaitables returned by an asynchronous iterator’s __anext__() method until it raises a StopAsyncIteration exception.

— Python Glossary

You can learn more about coroutines in the tutorial:

• What is a Coroutine in Python

An asynchronous iterator can be traversed using the “async for” expression.

This does not execute the for-loop in parallel.

Instead, the calling coroutine that executes the for loop will suspend and internally await each awaitable yielded from the iterator.

You can learn more about asynchronous iterators in the tutorial:

• Asynchronous Iterators in Python

You can learn more about the “async for” expression in the tutorial:

• Asyncio async for loop

An asynchronous context manager is a context manager that can await the enter and exit methods.

An asynchronous context manager is a context manager that is able to suspend execution in its enter and exit methods.

— Asynchronous Context Managers and “async with”

The “async with” expression is for creating and using asynchronous context managers.

The calling coroutine will suspend and await the context manager before entering the block for the context manager, and similarly when leaving the context manager block.

You can learn more about asynchronous context managers in the tutorial:

• Asynchronous Context Managers in Python

You can learn more about the “async with” expression in the tutorial:

• What is Asyncio async with

These are the sum of the major changes to Python language to support coroutines.

Next, let’s look at the asyncio module.

The asyncio Module

The “asyncio” module provides functions and objects for developing coroutine-based programs using the asynchronous programming paradigm.

Specifically, it supports non-blocking I/O with subprocesses (for executing commands) and with streams (for TCP socket programming).

asyncio is a library to write concurrent code using the async/await syntax.

— asyncio — Asynchronous I/O

Central to the asyncio module is the event loop.

This is the mechanism that runs a coroutine-based program and implements cooperative multitasking between coroutines.

The event loop is the core of every asyncio application. Event loops run asynchronous tasks and callbacks, perform network IO operations, and run subprocesses.

— Asyncio Event Loop

The module provides both a high-level and low-level API.

The high-level API is for us Python application developers. The low-level API is for framework developers, not us, in most cases.

Most use cases are satisfied using the high-level API that provides utilities for working with coroutines, streams, synchronization primitives, subprocesses, and queues for sharing data between coroutines.

The lower-level API provides the foundation for the high-level API and includes the internals of the event loop, transport protocols, policies, and more.

… there are low-level APIs for library and framework developers

— asyncio — Asynchronous I/O

Now that we know what asyncio is, broadly, and that it is for Asynchronous programming.

Next, let’s better understand what we mean by asynchronous programming.

What is Asynchronous Programming

Asynchronous means not at the same time, as opposed to synchronous or at the same time.

When programming, asynchronous means that the action is requested, although not performed at the time of the request. It is performed later.

For example, we can make an asynchronous function call. This will issue the request to make the function call and will not wait around for the call to complete. We can choose to check on the status or result of the function call later.

Asynchronous programming is a programming paradigm that does not block.

Instead, requests and function calls are issued and executed somehow in the background at some future time. This frees the caller to perform other activities and handle the results of issued calls at a later time when results are available or when the caller is interested.

This may require specific programming patterns.

For example, issuing an asynchronous function call often results in some handle on the request that the caller can use to check on the status of the call or get results. This is often called a future.

Developing programs focused on asynchronous function calls and asynchronous tasks that make use of facilitating programming patterns, like Futures, is referred to as asynchronous programming.

• Asynchronous Programming: The use of asynchronous techniques, such as issuing asynchronous tasks or function calls.

Broadly, asynchronous programming in Python refers to making requests and not blocking to wait for them to complete.

Although there are other ways to achieve elements of asynchronous programming, full asynchronous programming in Python requires the use of coroutines and the asyncio module.

asyncio is short for asynchronous I/O. It is a Python library that allows us to run code using an asynchronous programming model. This lets us handle multiple I/O operations at once, while still allowing our application to remain responsive.

— Page 3, Python Concurrency with asyncio, 2022.

Asynchronous programming is commonly used with non-blocking I/O. In fact, they are so tightly coupled, that their usage is often referred to as asynchronous I/O.

Input/Output or I/O for short means reading or writing from a resource.

A read or write is requested and the caller waits for the request to complete. As such, these operations are commonly referred to as blocking I/O tasks.

Non-blocking I/O allows read and write calls to be made as asynchronous requests.

The operating system will handle the request and notify the calling program when the results are available.

You can learn more about asynchronous programming in Python in the guide:

• Asynchronous Programming in Python

We can develop Python programs using asynchronous programming using asyncio, broadly defined. Specifically, they will use coroutines and the “asyncio” module.

Why would we want to do this?

When Should We Use Asyncio

There are perhaps three reasons to use asyncio in a Python project.

They are:

1. Use asyncio in order to adopt coroutines in your program.
2. Use asyncio in order to use the asynchronous programming paradigm.
3. Use asyncio in order to use non-blocking I/O.

These are the most common defensible reasons you may choose to use asyncio in your program.

We might need to use asyncio.

A key reason we may need to use asyncio is to meet the scalability requirements of a project.

Asyncio allows us to develop programs that easily support many thousands of concurrency socket connections. This can be achieved with threads but requires more overhead.

Asyncio offers a simple way to support many thousands of simultaneous socket connections, including being able to handle many long-lived connections for newer technologies like WebSockets, or MQTT for Internet of Things (IoT) applications.

— Page 6, Using Asyncio in Python, 2020.

There are additional reasons, that may be less robust and more emotional, and the real reasons a developer may need to use asyncio on a project.

1. Use asyncio because someone else made the decision for you.
2. Use asyncio because the project you have joined is already using it.
3. Use asyncio because you want to learn more about it.

Ideally, we would choose a reason that is defended in the context of the requirements of the project.

More decisions are made based on emotion that we credit, including the choice of libraries.

When Should We Not Use Asyncio

There are also cases to be made for not using asyncio.

One reason to not use asyncio is that you cannot defend its use using one of the reasons above.

I think the major reason to not use asyncio is that it does not deliver the benefit that you think it does.

Common misconceptions include:

• Asyncio will work around the global interpreter lock.
• Asyncio is faster than threads.
• Asyncio avoids the need for mutex locks and other synchronization primitives.
• Asyncio is easier to use than threads.

These are all false.

Choosing asyncio for one of these reasons is a bad idea.

Another reason to not use asyncio is that you don’t like asynchronous programming.

Asynchronous programming has been popular for some time now in a number of different programming communities, most notably the JavaScript community.

It is different from procedural, object-oriented, and functional programming, and some developers just don’t like it.

No problem. If you don’t like it, don’t use it. It’s a fair reason.

You can learn more about when to use and when to not use asyncio in the tutorial:

• When to Use Asyncio in Python

Further Reading

This section provides additional resources that you may find helpful.

Python Asyncio Books

• Python Asyncio Mastery, Jason Brownlee (my book!)
• Python Asyncio Jump-Start, Jason Brownlee.
• Python Asyncio Interview Questions, Jason Brownlee.
• Asyncio Module API Cheat Sheet

I also recommend the following books:

• Python Concurrency with asyncio, Matthew Fowler, 2022.
• Using Asyncio in Python, Caleb Hattingh, 2020.
• asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.

Guides

• Python Asyncio: The Complete Guide
• Python Asynchronous Programming

APIs

• asyncio — Asynchronous I/O
• Asyncio Coroutines and Tasks
• Asyncio Streams
• Asyncio Subprocesses
• Asyncio Queues
• Asyncio Synchronization Primitives

References

• Asynchronous I/O, Wikipedia.
• Coroutine, Wikipedia.

Takeaways

You now know what asyncio is in Python.

Specifically, you know:

• That asyncio really refers to two things, support for coroutines and the asyncio module in the standard library.
• What is asynchronous programming and how asyncio provides this capability in Python.
• When you should consider using asyncio in your Python programs.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by Arifin Salleh on Unsplash