Asyncio is strongly disliked, perhaps hated by many Python developers.

This can be seen in the comments on social media when asyncio in Python is discussed. I believe this mostly stems from Python developers misunderstanding the promise of asyncio (e.g. not speed or ease of use) and not taking asyncio seriously as an alternative programming paradigm rather than a garnish to an existing program or design.

In this tutorial, you will discover why Python developers hate asyncio.

Let’s get started.

Python Developers Hate Asyncio

Python asyncio is hated by many Python developers.

As a first step, I want to share some recent examples I’ve cherry-picked.

After 2 years of using asyncio in production, I recommend to avoid it if you can. With async programming, you take the complexity of concurrent programming, which is way harder than you can imagine.

— kissgyorgy, hacker news comment.

There is very little in everyday Python usage that benefits from Asyncio. Two in webdev, are long running request (Websockets, SSE, long polling), and processing multiple backend IO processes in parallel. However the later is very rare, you may think you have multiple DB request that could use asyncio, but most of the time they are dependent on each other. Almost all of the time a normal multithreaded Python server is perfectly sufficient and much easer to code. My recommendation is only use it where it is really REALY required.

— samwillis, hacker news comment.

Ok. This is what has been a huge hangup for me. It really seems that if you’re doing asyncio, you must do EVERYTHING async, it’s like asyncio takes over (infects?) the entire program.

— bjt2n3904, hacker news comment.

Asyncio is annoying, and often unexpectedly slow. You think things are parallel, but then one misbehaving coroutine can hog your cpu bringing everything to a halt. GIL makes it useless for anything other than _heavily_ IO bound tasks. And yeah, Python’s documentation is useless. Never how to use stuff, only listing of everything that’s possible to do / the API. Unfortunately that style is being mimicked by most other Python projects as well.

— matsemann, hacker news comment.

I’ve used python since 1995 and I can say that async is one of the worst things I’ve seen put into python since then. I’ve used a wide range of frameworks (twisted, gevent, etc) as well as threads and even if async is a good solution (I don’t think it is) it broke awscli for quite some time (through aiobotocore and related package dependencies). It’s too late in the game for long-term breaks like that or any backward-incompatible changes impacting users.

— dekhn, hacker news comment.

Python asyncio is pretty awful. The libraries are of extremely poor quality, and the slightest mistake can lead to blocking the event loop. After a few years of dealing with it I refuse to continue and am just using threads.

— ltbarcly3, hacker news comment.

Man that thing is complex and it keeps getting more complex. I do not have the mental capacity to casually work with asyncio. It requires constantly updating the knowledge with all language changes and it has tremendously complicated the language. It’s impressive that an ecosystem is evolving around it but I can’t help but get the impression that it will take quite a few more years for it to become a particularly enjoyable and stable development experience.

— Armin Ronacher, blog.

I think I’ve made the point a few times that I don’t like this style of programming at all, because the coroutine layer turns into an Inner Platform [1] replicating all the control-flow structures the original language has, which then has to integrate with the original language which causes more than twice the complexity to emerge.

— jerf, hacker news comment.

So I’m not the only one. I wrote larger-ish project while using asyncio and it is pain. The syntax is very unfamiliar (especially in 3.5 with async/await), the documentation is confusing and it’s very hard to debug it in general. Also it’s very hard to combine/stack multiple IO heavy events (make 5 calls to these URLs and whichever is done and returns these task run them in parallel).

— doh, hacker news comment.

Now that we have seen some strong dislike for asyncio, let’s take a closer look at what asyncio is exactly.

What is Asyncio

Broadly, asyncio refers to the ability to implement asynchronous programming in Python.

Specifically, it refers to two elements:

• The addition of the “asyncio” module to the Python standard library in Python 3.4.
• The addition of async/await expressions to the Python language in Python 3.5.

Together, the module and changes to the language facilitate the development of Python programs that support coroutine-based concurrency, non-blocking I/O, and asynchronous programming.

Python 3.4 introduced the asyncio library, and Python 3.5 produced the async and await keywords to use it palatably. These new additions allow so-called asynchronous programming.

— Page vii, Using Asyncio in Python, 2020.

Let’s take a closer look at these two aspects of asyncio, starting with the changes to the language.

Changes to Python to add Support for Coroutines

The Python language was changed to accommodate asyncio with the addition of expressions and types.

More specifically, it was changed to support coroutines as first-class concepts. In turn, coroutines are the unit of concurrency used in asyncio programs.

A coroutine is a function that can be suspended and resumed.

coroutine: Coroutines are a more generalized form of subroutines. Subroutines are entered at one point and exited at another point. Coroutines can be entered, exited, and resumed at many different points.

— Python Glossary

A coroutine may suspend for many reasons, such as executing another coroutine, e.g. awaiting another task, or waiting for some external resources, such as a socket connection or process to return data.

Many coroutines can be created and executed at the same time. They have control over when they will suspend and resume, allowing them to cooperate as to when concurrent tasks are executed.

This is called cooperative multitasking and is different from the multitasking typically used with threads called preemptive multitasking tasking.
https://en.wikipedia.org/wiki/Cooperative_multitasking

A coroutine can be defined via the “async def” expression. It can take arguments and return a value, just like a function.

For example:

Calling a coroutine function will create a coroutine object, this is a new class. It does not execute the coroutine function.

A coroutine can execute another coroutine via the await expression.

This suspends the caller and schedules the target for execution.

Next, let’s look at the asyncio module.

The asyncio Module

The “asyncio” module provides functions and objects for developing coroutine-based programs using the asynchronous programming paradigm.

Specifically, it supports non-blocking I/O with subprocesses (for executing commands) and with streams (for TCP socket programming).

asyncio is a library to write concurrent code using the async/await syntax.

— asyncio — Asynchronous I/O

Central to the asyncio module is the event loop.

This is the mechanism that runs a coroutine-based program and implements cooperative multitasking between coroutines.

The event loop is the core of every asyncio application. Event loops run asynchronous tasks and callbacks, perform network IO operations, and run subprocesses.

— Asyncio Event Loop

The module provides both a high-level and low-level API.

The high-level API is for us Python application developers. The low-level API is for framework developers, not us, in most cases.

Most use cases are satisfied using the high-level API that provides utilities for working with coroutines, streams, synchronization primitives, subprocesses, and queues for sharing data between coroutines.

The lower-level API provides the foundation for the high-level API and includes the internals of the event loop, transport protocols, policies, and more.

… there are low-level APIs for library and framework developers

— asyncio — Asynchronous I/O

Now that we know what asyncio is, broadly, and that it is for Asynchronous programming.

Why Python Developers Hate Asyncio

We have seen that there is a strong dislike for asyncio by practicing Python developers.

This may be a vocal minority.

This may be because the documentation is terse (it is). I think most complaints come down to the “inherent complexity” of asyncio. I think this complexity is perceived because it’s not treated seriously. It’s treated like a bolt-on feature to a project.

We have also seen that asyncio is not just concurrent Python, it is a new programming paradigm that involves coroutines and changes to the language itself and additional modules to support it.

I think this is the crux of the issue. It’s a different programming paradigm. Not a revelation perhaps, but bear with me.

Asyncio does not promise speed improvements over threads because it uses non-blocking I/O.

Asyncio will make my code blazing fast. Unfortunately, no. In fact, most benchmarks seem to show that threading solutions are slightly faster than their comparable Asyncio solutions.

— Page 7, Using Asyncio in Python, 2020.

It also does not promise easy use over regular concurrent programming.

Asyncio makes concurrent programming easy. Ahem, where do I even begin? […] Dealing with concurrency is always complex, regardless of whether you’re using threading or Asyncio.

— Page 8, Using Asyncio in Python, 2020.

Nevertheless, many Python developers wander into asyncio expecting these benefits. Even after all these years.

Asyncio does require that you start with asynchronous programming in mind before you start.

It requires that you develop using the asynchronous programming paradigm.

This does not preclude imperative, functional, object-oriented programming, we can still use functions, lambdas, and objects. Nevertheless, it does require that you know about coroutines suspending, resuming, and cooperating toward the goals of the program.

And this is hard because it’s different. Like generators are different (and suffer a similar level of under-adoption)

Asynchronous programming takes practice. It is a different way of problem-solving and therefore a different way of thinking.

It is also not appropriate for all problem types.

In many cases, a thread or process pool will work just fine. It can be tacked on and used to make tasks executed in a for-loop concurrently.

What I fear is that newer developers will see this dislike of asyncio spread far and wide, and avoid it. Modules in the standard library that go unused for too long will be neglected and fall away, perhaps being deprecated and removed over the longer arc. I don’t think this is a desirable outcome.

Would things be different if async programming was taught early on, alongside functions and objects?

Yeah, I think so.

I don’t think asynchronous programming is going away. It’s not a fad like aspect-oriented programming was in the early 2000s. JavaScript has made the case. A subset of developers, perhaps just web developers, embrace this approach to developing software.

Perhaps that is truly the sweet spot for asyncio.

A way of programming for a sub-community focused on a specific class of problems that were already indoctrinated over in JavaScriptland. Easy adoption and little education are needed.

I’m sure there will be developers who put their hand up and say, “I’m a die-hard async programmer from way back, but the async baked into the Python stdlib is bad in these specific ways“.

Sure, but I would bet that a pool of Python developers that love vs hate asyncio will break down by heavy js vs light/no javascript background. A testable hypothesis with a big enough survey.

Further Reading

This section provides additional resources that you may find helpful.

Python Asyncio Books

• Python Asyncio Mastery, Jason Brownlee (my book!)
• Python Asyncio Jump-Start, Jason Brownlee.
• Python Asyncio Interview Questions, Jason Brownlee.
• Asyncio Module API Cheat Sheet

I also recommend the following books:

• Python Concurrency with asyncio, Matthew Fowler, 2022.
• Using Asyncio in Python, Caleb Hattingh, 2020.
• asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.

Guides

• Python Asyncio: The Complete Guide
• Python Asynchronous Programming

APIs

• asyncio — Asynchronous I/O
• Asyncio Coroutines and Tasks
• Asyncio Streams
• Asyncio Subprocesses
• Asyncio Queues
• Asyncio Synchronization Primitives

References

• Asynchronous I/O, Wikipedia.
• Coroutine, Wikipedia.

Takeaways

You now know why many Python developers dislike asyncio.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by Matias Malka on Unsplash