Asyncio Dunder Methods (Magic Methods)

Asyncio brings asynchronous programming to Python.

This includes a number of dunder methods (magic methods) that define behaviors expected of asynchronous objects in Python, intended to be used via specific asynchronous expressions.

In this tutorial, you will discover asyncio dunder methods (magic methods) in Python.

After completing this tutorial, you will know:

• Awaitables implement the __await__() and are used with the "await" expression.
• Asynchronous context managers implement the __aenter__() and __aexit__() methods and are used with the "async with" expression.
• Asynchronous iterators implement the __aiter__() and __anext__() methods and are used with the "async for" expression.

Let's get started.

Python Dunder Methods (Magic Methods)

Methods on objects that start and end with double underscore characters (e.g. '__') are called "dunder" methods.

They are also sometimes called "magic methods" or "special method names".

An example of a dunder method that most Python developers are familiar with is __init__() which is the constructor for a Python object.

object.__init__(self[, ...]) Called after the instance has been created (by __new__()), but before it is returned to the caller.

-- Python Data Model

For example:

# define a class
class MyClass:
	# define class constructor method
	def __init__(self):
		pass

Dunder methods are different from regular Python methods because they are expected and used by Python itself to handle default object behaviors.

Python also provides methods, often called dunder methods (due to their names beginning and ending with double-underscores), to allow user-defined classes to modify how they are handled by native operations including length, comparison, in arithmetic operations and type conversion.

-- Python (programming language), Wikipedia.

Now that we know about dunder methods, let's take a look at dunder methods specific to asyncio.

Asyncio Dunder Methods

There are 5 dunder methods that are specific to asyncio and asynchronous programming, they are:

1. __await__() for Awaitables
2. __aenter__() for Asynchronous Context Managers
3. __aexit__() for Asynchronous Context Managers
4. __aiter__() for Asynchronous Iterators
5. __anext__() for Asynchronous Iterators

Put another way, there are three classes of dunder methods for asyncio.

1. Dunder methods for awaitables, e.g. __await__().
2. Dunder methods for asynchronous context managers, e.g. __aenter__() and __aexit__().
3. Dunder methods for asynchronous iterators, e.g. __aiter__() and __anext__().

Let's take a closer look at each in turn.

Awaitables and __await__()

A Python object is "awaitable" if it implements the __await__() method.

awaitable: An object that can be used in an await expression. Can be a coroutine or an object with an __await__() method.

-- Python Glossary

An awaitable object means that it can be used with the await expression in an asyncio program.

For example:

...
# await a coroutine
await coro()

Coroutine objects implement the __await__() method and are awaitable.

Another example is the asyncio.Future object which provides the base class for the asyncio.Task. This means that instances both the asyncio.Future and asyncio.Task classes are awaitable and can be used in await expressions.

object.__await__(self): Must return an iterator. Should be used to implement awaitable objects. For instance, asyncio.Future implements this method to be compatible with the await expression.

-- Python Data Model

The await expression is used in asyncio programs within coroutines to suspend execution and wait for the target awaitable object to return.

You can learn more about the await expression in the tutorial:

• What is an Asyncio Awaitable in Python

The "await" expression and __await__() method were described in PEP 492 and added to the language in Python version 3.5.

Inside a coroutine function, the new await expression can be used to suspend coroutine execution until the result is available. Any object can be awaited, as long as it implements the awaitable protocol by defining the __await__() method.

-- What’s New In Python 3.5

Asynchronous Context Managers

An asynchronous context manager is like a regular context manager, except the enter and exit methods return awaitables that can be awaited.

An asynchronous context manager is a context manager that is able to suspend execution in its enter and exit methods. To make this possible, a new protocol for asynchronous context managers is proposed. Two new magic methods are added: __aenter__ and __aexit__. Both must return an awaitable.

-- PEP 492 – Coroutines with async and await syntax

Asynchronous context managers implement two methods, the __aenter__() and __aexit__() methods that are asynchronous versions of the __enter__() and __exit__() methods on regular context managers.

They are used via the "async with" expression, an asynchronous version of the "with" expression.

For example:

...
# create and use an async context manager
async with AsyncContextmMnager() as mgr:
	# do things

You can learn more about the "async with" expression in the tutorial:

• How to Use the "async with" Expression in Python

These asynchronous versions of context managers were defined in PEP 492 and added to Python in version 3.5.

You can learn more about how to define and use asynchronous context managers in the tutorial:

• Asynchronous Context Managers in Python

Next, let's take a closer look at each method in turn.

__aenter__() Method

The __aenter__() method is called when an asynchronous context manager is entered and returns an awaitable.

object.__aenter__(self): Semantically similar to __enter__(), the only difference being that it must return an awaitable.

-- Python Data Model

This means it is defined using the "async def" expression.

For example:

# define enter method for async context manager
async def __aenter__(self):
	# do things...

It does not "return" an awaitable, instead when it is "called" an awaitable coroutine object is constructed.

For example:

...
# create a coroutine object
coro = object.__aenter__()

This awaitable is then awaited automatically as part of the asynchronous context manager interface.

It could be used to open or prepare a resource used within the body of a context manager.

__aexit__() Method

The __aexit__() method is called when an asynchronous context manager is exited and returns an awaitable.

object.__aexit__(self, exc_type, exc_value, traceback): Semantically similar to __exit__(), the only difference being that it must return an awaitable.

-- Python Data Model

This means it is also defined using the "async def" expression.

For example:

# define exit method for async context manager
async def __aexit__(self, exc_type, exc_value, traceback):
	# do things...

Like __aenter__(), it does not "return" an awaitable, instead when it is "called" an awaitable coroutine object is constructed.

For example:

...
# create a coroutine object
coro = object.__aenter__()

This awaitable is then awaited automatically as part of the asynchronous context manager interface.

It could be used to close or terminate a resource used within the body of a context manager.

A common example is the asyncio.Server that implements the asynchronous context manager interface and will close all client connections and the server when the context manager is exited.

For example:

...
# open the async context manager for the server
async with server:
	# use the server
	...
# all connections closed when the block is exited

Asynchronous Iterators (and Generators)

An asynchronous iterator is like a regular iterator, except the next method returns an awaitable that can be awaited.

An asynchronous iterable is able to call asynchronous code in its iter implementation, and asynchronous iterator can call asynchronous code in its next method. An object must implement an __aiter__ method [...] An asynchronous iterator object must implement an __anext__ method ...

-- PEP 492 – Coroutines with async and await syntax

An asynchronous iterator is created via the __aiter__() method and implements the __anext__() method.

They are used via the "async for" expression, an asynchronous version of the "for" expression.

For example:

...
# traverse an asynchronous iterator
async for value in async_it:
	# ...

You can learn more about the async for expression in the tutorial:

• How to Use the "async for" Expression in Python

Asynchronous iterators were described in PEP 492 and introduced in Python version 3.5.

Changed in version 3.5.2: Starting with CPython 3.5.2, __aiter__ can directly return asynchronous iterators. Returning an awaitable object will result in a PendingDeprecationWarning.

-- What's New In Python 3.5

You can learn more about how to create and use asynchronous iterators in the tutorial:

• Asynchronous Iterators in Python

Importantly, asynchronous generators are examples of asynchronous iterators.

An asynchronous generator object implements the asynchronous iterator methods, e.g. __anext__().

The result of calling an asynchronous generator function is an asynchronous generator object, which implements the asynchronous iteration protocol

Asynchronous generators were described in PEP 525 and introduced in Python version 3.6.

You can learn more about asynchronous generators in the tutorial:

• Asynchronous Generators in Python

Next, let's take a closer look at each method in turn.

__aiter__() Method

The __aiter__() method must return an object that implements the asynchronous interface, e.g. the __aiter__() and __next__() methods.

object.__aiter__(self): Must return an asynchronous iterator object.

-- Python Data Model

The __aiter__() does not return an awaitable, this means it is defined with the "def" expression, and not the "async def "expression.

For example, the __aiter__() can be implemented on an asynchronous iterator object to return itself.

# get an instance of the async iterator
def __aiter__(self):
    return self

__anext__() Method

The __anext__() method must return an awaitable that itself will return the next value of the iterator.

object.__anext__(self): Must return an awaitable resulting in a next value of the iterator. Should raise a StopAsyncIteration error when the iteration is over.

-- Python Data Model

This means that an implementation of the __anext__() method must be defined using the "async def" expression.

For example:

# return the next awaitable
async def __anext__(self):
	# ...

It does not "return" an awaitable, instead when it is "called" an awaitable coroutine object is constructed.

For example:

...
# create a coroutine object
coro = object.__anext__()

The awaitable can then be awaited automatically as part of the "async for" expression to acquire the next value in the iteration.

This could be within a loop, for example:

...
# traverse an asynchronous iterator
async for result in AsyncIterator():
	print(result)

This can also be within a comprehension, such as an asynchronous list comprehension.

For example:

...
# async list comprehension with async iterator
results = 

You can learn more about asynchronous comprehensions in the tutorial:

• Asynchronous Comprehensions in Python

Takeaways

You now know about the asyncio dunder methods (magic methods) in Python.

Specifically, you know:

• Awaitables implement the __await__() and are used with the "await" expression.
• Asynchronous context managers implement the __aenter__() and __aexit__() methods and are used with the "async with" expression.
• Asynchronous iterators implement the __aiter__() and __anext__() methods and are used with the "async for" expression.

If you enjoyed this tutorial, you will love my book: Python Asyncio Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.