Asyncio has found a home in Python web development and many asyncio web development projects require database access.

This means there is a need for Python database drivers that support asyncio. This support may be naive via an async-first driver, or provided via a wrapper around a blocking I/O database driver that simulates async/await syntax using a thread pool.

In this tutorial, you will discover asyncio-compatible database drivers for Python.

Let’s get started.

Need Asyncio Database Drivers

Asyncio provides asynchronous programming in Python via coroutines and the asyncio module in the standard library.

One of the primary benefits of asyncio is non-blocking I/O, making asyncio ideal for network programming. As such, asyncio has found a home in modern web development for web applications and APIs.

The vast majority of these asyncio applications make use of a database.

Database access is just another type of network I/O where the Python application connects, queries, and receives results from the database via a socket connection.

Typically, these are blocking I/O socket connections, which is antithetical to the asynchronous programming paradigm used in asyncio programs.

This means that when we are using asyncio in our web applications, we require asyncio-compatible database drivers to avoid blocking when querying the database.

What are the asyncio database drivers available for asyncio?

Asyncio Database Drivers

There are asyncio-compatible database drivers for most popular databases used in Python

This includes popular relational databases, such as:

• PostgreSQL
• MySQL
• SQLite
• ODBC (generic)

It also includes popular in-memory and NoSQL databases, such as:

• Redis
• Memcache
• MongoDB
• CouchDB

And more.

Let’s take a closer look at the popular drivers for these databases in turn.

Asyncio Relational Database Drivers

In this section, we will review popular asyncio database drivers for relational databases.

This includes PostgreSQL, MySQL, SQLite, and the generic ODBC driver.

PostgreSQL

PostgreSQL is probably one of the most popular relational databases used in Python.

PostgreSQL is a powerful, open source object-relational database system with over 35 years of active development that has earned it a strong reputation for reliability, feature robustness, and performance.

— PostgreSQL Homepage

Three PostgreSQL database drivers for asyncio are as follows:

• aiopg: aiopg is a library for accessing a PostgreSQL database from the asyncio
• asyncpg: A fast PostgreSQL Database Client Library for Python/asyncio.
• asyncpgsa: A wrapper around asyncpg for use with sqlalchemy

We can review the GitHub star rating histories to get some idea of the relative popularity and usage of each of these libraries.

We can see that aiopg has a longer history and that asyncpg is by far the more popular library.

The aiopg library may have been one of the first Postgres database drivers.

It is now a part of the aio-libs project for maintaining high-quality asyncio libraries.

aiopg is a library for accessing a PostgreSQL database from the asyncio (PEP-3156/tulip) framework. It wraps asynchronous features of the Psycopg database driver.

— Welcome to AIOPG

Internally, it is built on top of the popular psycopg2 library, and does support SQLAlchemy.

The library uses psycopg2-binary connections in asynchronous mode internally.

— Welcome to AIOPG

Unlike aiopg, asyncpg is a clean implementation of an asyncio-first Postgres database driver.

asyncpg is a database interface library designed specifically for PostgreSQL and Python/asyncio. asyncpg is an efficient, clean implementation of PostgreSQL server binary protocol for use with Python’s asyncio framework.

— asyncpg GitHub Project.

This clean-room implementation may mean it is significantly faster than other libraries. The project lists benchmark results suggesting it is significantly faster than aiopg and 5x faster than psycopg3 (the popular Python Postgres database driver).

It was developed and is maintained by magicstack, who are also responsible for uvloop the plug-in faster asyncio loop replacement.

The asyncpg library does not offer support for the popular SQLAlchemy framework.

As such, the asyncpgsa is a wrapper for asyncpg that provides this ability directly.

A python library wrapper around asyncpg for use with sqlalchemy

— asyncpgsa GitHub Project.

MySQL

MySQL is another very popular open-source database used by the Python community.

The popular asyncio-compatible database drivers for MySQL include the following:

• aiomysql: aiomysql is a library for accessing a MySQL database from the asyncio
• asyncmy: A fast asyncio MySQL/MariaDB driver with replication protocol support

We can review the GitHub star rating histories to get some idea of the relative popularity and usage of each of these libraries.

The plot shows that asyncmy is a newer library and that aiomysql is perhaps more popular and widely regarded.

Like the aiopg library, the aiomysql library is also maintained by the aio-libs project.

And like that project, the aiomysql library is based on a non-asyncio Python database library, in this case the PyMySQL library.

Inspired by aiopg, it seeks to preserve a similar interface.

aiomysql is a “driver” for accessing a MySQL database from the asyncio (PEP-3156/tulip) framework. It depends on and reuses most parts of PyMySQL . aiomysql tries to be like awesome aiopg library and preserve same api, look and feel.

— aiomysql GitHub Project.

The aiomysql library provides support for the SQLAlchemy ORM framework.

Internally aiomysql is copy of PyMySQL, underlying io calls switched to async, basically await and async def coroutine added in proper places. sqlalchemy support ported from aiopg.

— Welcome to aiomysql’s documentation!

The asyncmy library is newer and perhaps less popular and focuses on asyncio support for both MySQL and MariaDB (a fork of MySQL) with a focus on performance.

asyncmy is a fast asyncio MySQL/MariaDB driver, which reuse most of pymysql and aiomysql but rewrite core protocol with cython to speedup.

— asyncmy GitHub Project.

The project provides benchmark results showing that asyncmy is faster than other Python MySQL libraries.

SQLite

SQLite is a very popular tiny and fast database, used perhaps more widely than any other database.

SQLite is a C-language library that implements a small, fast, self-contained, high-reliability, full-featured, SQL database engine. SQLite is the most used database engine in the world. SQLite is built into all mobile phones and most computers and comes bundled inside countless other applications that people use every day.

— SQLite Homepage

The asyncio provides an asyncio-compatible library for SQLite.

• aiosqlite: asyncio bridge to the standard sqlite3 module

It replicates the popular sqlite3 module in the Python standard library and includes support for the async/await syntax.

aiosqlite provides a friendly, async interface to sqlite databases. It replicates the standard sqlite3 module, but with async versions of all the standard connection and cursor methods, plus context managers for automatically closing connections and cursors

— aiosqlite GitHub Project.

Internally, the library makes use of threads to simulate asynchronous programming.

aiosqlite allows interaction with SQLite databases on the main AsyncIO event loop without blocking execution of other coroutines while waiting for queries or data fetches. It does this by using a single, shared thread per connection. This thread executes all actions within a shared request queue to prevent overlapping actions.

— aiosqlite GitHub Project.

ODBC

ODBC or Open Database Connectivity is a generic way to connect to relational databases.

It provides a way to connect to many databases that do not have a Python-specific database driver.

In computing, Open Database Connectivity (ODBC) is a standard application programming interface (API) for accessing database management systems (DBMS).

— Open Database Connectivity, Wikipedia.

The aioodbc library provides an asyncio-compatible ODBC database driver.

• aioodbc: aioodbc – is a library for accessing a ODBC databases from the asyncio

It is built upon the pyodbc Python ODBC driver and uses threads to simulate non-blocking I/O.

aioodbc is a Python 3.7+ module that makes it possible to access ODBC databases with asyncio. It relies on the awesome pyodbc library and preserves the same look and feel. Internally aioodbc employs threads to avoid blocking the event loop, threads are not that as bad as you think!. Other drivers like motor use the same approach.

— aioodbc GitHub Project.

Asyncio NoSQL Database Drivers

This section reviews popular asyncio-compatible nosql database drivers.

This includes projects that support Redis, Memcache, MongoDB, and CouchDB.

The plot below shows the star rating histories for the projects in this section. Ignoring the generic redis-py driver project, we can see that the MongoDB and (now archived) Redis asyncio-specific drivers share similar histories.

Redis

Redis is perhaps the most popular in-memory key-value NoSQL database.

The open source, in-memory data store used by millions of developers as a database, cache, streaming engine, and message broker.

— Redis Homepage.

Asyncio database drivers for Redis include the following projects:

• redis-py: Redis Python Client
• aioredis: asyncio (PEP 3156) Redis support

Asyncio support for Redis used to be provided via the aioredis-py library.

This has since been shut down and archived as of early 2023.

The preferred asyncio-compatible Redis library is the redis-py library. It is the official library maintained by Redis Inc.

The Python interface to the Redis key-value store.

— redis-py GitHub Project.

It is not asyncio specific, but rather the general Redis Python library that has added asyncio support.

This means that asyncio is an afterthought rather than the library offering an async-first design. Performance and/or elegance of the API may suffer as a result.

Utilizing asyncio Redis requires an explicit disconnect of the connection since there is no asyncio deconstructor magic method. By default, a connection pool is created on redis.Redis() and attached to this Redis instance. The connection pool closes automatically on the call to Redis.aclose which disconnects all connections.

— Redis Asyncio Examples

MongoDB

Like Redis, MongoDB is a very popular NoSQL database although focused on structured documents more than a general key-value store.

MongoDB is a source-available cross-platform document-oriented database program. Classified as a NoSQL database program, MongoDB uses JSON-like documents with optional schemas.

— MongoDB, Wikipedia.

Asyncio support for MongoDB is provided by the Motor library.

• motor: Motor – the async Python driver for MongoDB and Tornado or asyncio

The Motor library is a general async Python library that supports both the async Tornado webserver and asyncio.

Motor presents a coroutine-based API for non-blocking access to MongoDB from Tornado or asyncio.

— Motor: Asynchronous Python driver for MongoDB.

Memcache

Memcache is another popular NoSQL database

Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load. Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

— Memcache Homepage.

Memcache support in asyncio is provided by the aiomcache library.

• aiomcache: Minimal asyncio memcached client

This is a simple Memcache API and the library is maintained by the aio-libs project.

CouchDB

CouchDB is a popular open-source NoSQL database.

Apache CouchDB is an open-source document-oriented NoSQL database, implemented in Erlang.

— Apache CouchDB, Wikipedia.

The aiocouchdb library provides asyncio support for CouchDB.

• aiocouchdb: CouchDB client built on top of aiohttp (asyncio)

The libary is mainained by aio-libs and appears to be limited to beta support. The reposiory suggests that it is not activelly developed.

CouchDB client built on top of aiohttp and made for asyncio. Current status: beta. aiocouchdb has all CouchDB API implements up to 1.6.1 release. However, it may lack of some usability and stability bits, but work is in progress.

— aiocouchdb GitHub Project.

Asyncio Unified Database Drivers

A common solution when using an asyncio web framework is to use a unified database driver.

This is a database driver that has asyncio support and supports many different types of databases behind the scenes Allowing the same web framework and application to be agnostic to the specific database implementation.

Popular unified asyncio database driver libraries include the following:

• databases: Async database support for Python
• asyncdb: Asynchronous generic database connectors for project Navigator-API

The plot below shows the star rating histories for these projects. We can see that the databases project may be the community-preferred approach.

The Databases project is popular and widely used by web frameworks.

Databases gives you simple asyncio support for a range of databases.

— databases GitHub Project.

It includes support for popular relational databases such as PostgreSQL, MySQL, and SQLite and supports use in popular webframeworks such as Starlette, Sanic, Responder, Quart, aiohttp, Tornado, or FastAPI.

This is achieved by using the l SQLAlchemy ORM framework.

It allows you to make queries using the powerful SQLAlchemy Core expression language, and provides support for PostgreSQL, MySQL, and SQLite.

— databases GitHub Project.

It can be installed with specific support for a given relational database, leveraging many of the libraries listed above such as: asyncpg, aiopg, aiomysql, asyncmy, and aiosqlite.

This is probably the preferred approach if you are using a popular web framework.

The asyncdb provides another less widely used unified asyncio database framework.

It supports a large number of databases and mostly simulates asynchronous programming by internally using a thread pool and a wrapper around blocking database drivers.

AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.

— asyncdb GitHub Project.

It seems fully featured and I’m surprised it’s not more widely used than it appears to be.

Further Reading

This section provides additional resources that you may find helpful.

Python Asyncio Books

• Python Asyncio Mastery, Jason Brownlee (my book!)
• Python Asyncio Jump-Start, Jason Brownlee.
• Python Asyncio Interview Questions, Jason Brownlee.
• Asyncio Module API Cheat Sheet

I also recommend the following books:

• Python Concurrency with asyncio, Matthew Fowler, 2022.
• Using Asyncio in Python, Caleb Hattingh, 2020.
• asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.

Guides

• Python Asyncio: The Complete Guide
• Python Asynchronous Programming

APIs

• asyncio — Asynchronous I/O
• Asyncio Coroutines and Tasks
• Asyncio Streams
• Asyncio Subprocesses
• Asyncio Queues
• Asyncio Synchronization Primitives

References

• Asynchronous I/O, Wikipedia.
• Coroutine, Wikipedia.

Takeaways

You now know about asyncio-compatible database drivers for Python.

Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!

Do you have any additional tips?
I’d love to hear about them!

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by Soroush Karimi on Unsplash