Asyncio Coroutine-Safe in Python

Coroutine-safe is the concept of thread-safety for concurrency with coroutines, such as with asyncio.

In this tutorial, you will discover the concept of coroutine safety in Python.

Let’s get started.

Table of Contents

What is Thread-Safe

Thread-safe refers to program code that can be executed free of concurrency errors by multiple threads concurrently.

Primarily, it refers to the fact that the code is free of race conditions.

A race condition is a bug in concurrency programming.

It is a failure case where the behavior of the program is dependent upon the order of execution by two or more threads. This means the behavior of the program will not be predictable, possibly changing each time it is run.

There are many types of race conditions, although a common type of race condition is when two or more threads attempt to change the same data variable.

You can see examples of race conditions in threads and how to fix them in these tutorials:

We can develop code to be thread-safe using concurrency primitives like mutex locks.

For example:

Thread-Safe Counter in Python

Next, let’s take a look at process-safe.

Run loops using all CPUs, download your FREE book to learn how.

What is Process-Safe

Process-safe refers to program code that can be executed free of concurrency errors by multiple processes concurrently.

It is the concept of “thread-safe” applied to processes, where processes are the unit of concurrency instead of threads.

Thread safety is a major concern of concurrent programming using threads. This is because threads have shared memory within the process, meaning that concurrent access of the same data or variables can lead to race conditions.

Processes do not have direct shared memory and therefore are not subject to the same concerns of race conditions.

Nevertheless, processes do simulate shared memory using socket connections and files. They may need to protect simulated shared program state or data from race conditions due to timing and concurrent modification.

Happily, some tools used for inter-process communication provide some process safety, such as queues.

You can learn more about process safety in the tutorial:

Process-Safe in Python

Let’s take a closer look at how we can ensure process safety in our multiprocessing programs.

Download Now: Free Asyncio PDF Cheat Sheet

What is Coroutine-Safe

Coroutine-safe is the idea of “thread-safe” and “process-safe” applied to coroutines.

Recall that a coroutine is like a function (routine) except that it can be suspended and resumed.

It is a unit of concurrency lower than the level of an operating system thread and is used in the asyncio module.

Although two or more coroutines cannot execute at the same time within a thread, it is possible for program state and resources to be corrupted or made inconsistent via concurrent execution.

It is possible to have race conditions with coroutines in asyncio.

It is also possible to have deadlocks with coroutines in asyncio.

Coroutine-safe means that code or a program can be executed concurrently with multiple coroutines and will not result in concurrency failure modes such as race conditions and deadlocks.

Coroutine safety is achieved using common concurrency primitives, such as queues, locks, and more.

Next, let’s take a closer look at how we might implement coroutine safety in our asyncio programs.

Free Python Asyncio Course

Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.

Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.

Learn more

How to Ensure Coroutine-Safety

The asyncio module provides a number of concurrency primitives that are coroutine-safe.

Nevertheless, they are not thread-safe (or process-safe), and this is clearly stated in the API documentation.

Additionally, if we attempt to use thread-safe concurrency primitives to protect coroutines from race conditions, it can result in unexpected behavior, such as deadlocks.

The asyncio module provides a suite of concurrency primitives to help us make our programs coroutine-safe.

asyncio synchronization primitives are designed to be similar to those of the threading module […] asyncio primitives are not thread-safe, therefore they should not be used for OS thread synchronization
— Asyncio Synchronization Primitives

These primitives mirror the primitives provided in the threading and multiprocessing modules, including the classes:

asyncio.Lock
asyncio.Event
asyncio.Condition
asyncio.Semaphore
asyncio.BoundedSemaphore

We can use the primitives in our asyncio programs to ensure that our code is coronet-safe. That is, to protect critical sections, synchronize and coordinate behavior across multiple coroutines within a single thread (event loop).

In addition to these primitives, the asyncio module provides a number of coroutine-safe queue classes.

These too mirror the thread-safe queues in the “queue” module, but unlike those classes, these classes are not thread-safe.

asyncio queues are designed to be similar to classes of the queue module. Although asyncio queues are not thread-safe, they are designed to be used specifically in async/await code.
— Asyncio Queues

They include: