We can define a custom awaitable for use in asyncio programs.
This can be achieved by defining a Python object that implements the __await__() method that either yields execution or delegates or returns the result of another awaitables __await__() method.
In this tutorial, you will discover how to define Python objects with the __await__() method.
Let’s get started.
What is an Awaitable
An awaitable is a Python object that can be used in an await expression.
An awaitable must implement the __await__() special method (dunder method).
An awaitable object generally implements an __await__() method. Coroutine objects returned from async def functions are awaitable.
— Python Data Model.
Examples of Python objects that are awaitable include:
- Coroutines returned from async def expressions.
- asyncio.Task objects.
- asyncio.Future objects.
You can learn more about awaitable objects in the tutorial:
Run loops using all CPUs, download your FREE book to learn how.
What is __await__()
The __await__() method is a special method (dunder method) implemented by awaitable Python objects.
It must return an iterator and allow the object to be used as an await expression within coroutines.
object.__await__(self): Must return an iterator. Should be used to implement awaitable objects. For instance, asyncio.Future implements this method to be compatible with the await expression.
— Python Data Model.
This does not mean that we can return a regular iterator, as an iterator cannot be used directly in an await expression. It would result in an error.
Instead, it must be an iterator that is comfortable with the await expression, such as a yield statement that suspends execution, or delegation to another iterator. This may be a generator as a generator is an iterator of sorts.
Recall that a generator can delegate to a sub-generator or an iterator can delegate to a sub-iterator via the yield from expression.
When yield from is used, the supplied expression must be an iterable. The values produced by iterating that iterable are passed directly to the caller of the current generator’s methods.
— Python Expressions
You can learn more about Python dunder methods in the tutorial:
Examples of __await__()
We can learn more about the __await__() method by looking at some examples from the Python standard library.
Below is an example of the __await__() method from the asyncio.Future class.
|
1 2 3 4 5 6 7 |
def __await__(self): if not self.done(): self._asyncio_future_blocking = True yield self # This tells Task to wait for completion. if not self.done(): raise RuntimeError("await wasn't used with future") return self.result() # May raise too. |
Notice that the main return is either a yield to self or a return of the result.
The Coroutine class extends the Awaitable class and the __await__() method from this parent class is listed below.
|
1 2 3 |
@abstractmethod def __await__(self): yield |
Notice that it is simply a yield statement.
Generally, any chain of awaitables will at some point end with a yield statement.
Any yield from chain of calls ends with a yield. This is a fundamental mechanism of how Futures are implemented. Since, internally, coroutines are a special kind of generators, every await is suspended by a yield somewhere down the chain of await calls
— PEP 492 – Coroutines with async and await syntax
Now that we have seen some examples, let’s explore how we might implement the __await__() method ourselves.
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
How to Define a Custom Awaitable
Generally, we don’t define objects that implement the __await__() method in our asyncio programs.
Instead, we define coroutines.
For example:
|
1 2 3 |
# custom coroutine asyncio def custom_coroutine(): await asyncio.sleep() |
You can learn more about Python coroutines in the tutorial:
Nevertheless, this is an interesting exercise in order to learn more about how asyncio works internally.
We can define a custom awaitable object in Python by adding the __await__() method.
As we have seen, the __await__() method must have a yield statement, or delegate to other awaitables that eventually end in a yield statement.
The __await__() method is defined as a regular Python function, not a coroutine. This means it is defined using the “def” expression, not the “async def” expression.
For example:
|
1 2 3 |
# awaitable dunder method def __await__(self): # ... |
The content of the method can be regular Python statements.
As we have seen, the method must return an iterator. This may be a generator, as a generator is a type of iterator.
The example below shows this in the use of a yield expression.
Therefore the simplest implementation of the __await__() method is a yield statement.
For example:
|
1 2 3 4 |
# awaitable dunder method def __await__(self): # suspend execution yield |
We can also delegate the yield to another awaitable
This can be achieved using the “yield from” expression used by a generator to delegate to another generator.
PEP 380 adds the yield from expression, allowing a generator to delegate part of its operations to another generator. This allows a section of code containing yield to be factored out and placed in another generator. Additionally, the subgenerator is allowed to return with a value, and the value is made available to the delegating generator.
— What’s New In Python 3.3
If __await__() always returns an iterator, then we yield from another asyncio coroutine or task’s __await__() method.
This has the effect of delegating the call.
For example:
|
1 2 3 4 |
# awaitable dunder method def __await__(self): # suspend execution yield from asyncio.sleep(2).__await__() |
This does not have to be a yield or yield from expression, we might also return the result directly.
For example:
|
1 2 3 4 |
# awaitable dunder method def __await__(self): # return the generator return asyncio.sleep(2).__await__() |
Now that we know how to define an __await__(), let’s look at some worked examples.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of A Custom Awaitable That Yields
We can define a custom awaitable with an __await__() method that has nothing more than a yield statement.
This suspends the execution of the caller, and then returns it again.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
# SuperFastPython.com # example of a custom awaitable with yield import asyncio # define a custom awaitable class CustomAwaitable(object): # awaitable dunder method def __await__(self): # report a message print('Await is running') # suspend execution yield # report a message print('Await is done') # main coroutine async def main(): # create and await custom awaitable await CustomAwaitable() # start the asyncio event loop asyncio.run(main()) |
Running the example first creates the main() coroutine then starts the asyncio event loop.
The main() coroutine runs and creates an instance of our CustomAwaitable class and awaits it.
The __await__() method runs, reports a message, then yields execution.
If other coroutines were scheduled in the event loop, they would have an opportunity to run at this point
Control is then returned as there are no other tasks running in the event loop. A final message is reported and the method returns.
The main() coroutine then exits and the event loop is terminated.
This highlights how we can define a custom awaitable that yields execution.
|
1 2 |
Await is running Await is done |
We can make this example clearer by scheduling another task before our custom awaitable, but not allowing it to run.
Instead, it must wait until execution is yielded by our custom awaitable before it can run.
For example, our new coroutine looks as follows:
|
1 2 3 4 |
# test coroutine that prints 10 numbers async def test(): for i in range(10): print(i) |
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
# SuperFastPython.com # example of a custom awaitable with yield import asyncio # define a custom awaitable class CustomAwaitable(object): # awaitable dunder method def __await__(self): # report a message print('Await is running') # suspend execution yield # report a message print('Await is done') # test coroutine that prints 10 numbers async def test(): for i in range(10): print(i) # main coroutine async def main(): asyncio.create_task(test()) # create and await custom awaitable await CustomAwaitable() # start the asyncio event loop asyncio.run(main()) |
Running the example first creates the main() coroutine then starts the asyncio event loop.
The main() coroutine runs and creates the test() coroutine and schedules it for execution.
The main() coroutine then creates an instance of our CustomAwaitable class and awaits it.
The __await__() method runs, reports a message, then yields execution.
This allows our background test() task to run, running its loop and reporting all 10 integers.
Control is then returned to the custom awaitable and a final message is reported.
The main() coroutine then exits and the event loop is terminated.
This highlights how a custom awaitable can yield control and allow other tasks running in the event loop an opportunity to run.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
Await is running 0 1 2 3 4 5 6 7 8 9 Await is done |
Example of A Custom Awaitable That Delegates
We can explore an example of delegating the __await__() method to another __await__() method.
In this case, we can create an asyncio.sleep() coroutine object and delegate the __await__() method of our custom awaitable to the coroutines __await__() method.
For example:
|
1 2 3 4 5 6 7 8 |
# awaitable dunder method def __await__(self): # report a message print('Await is running') # suspend execution yield from asyncio.sleep(2).__await__() # report a message print('Await is done') |
This will have the effect of running the target coroutine in the event loop.
It provides a workaround to allow the __await__() method of our custom awaitables to execute other awaitables without having to await them directly, which we cannot do because the __await__() method is not a coroutine.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
# SuperFastPython.com # example of a custom awaitable with yield from import asyncio # define a custom awaitable class CustomAwaitable(object): # awaitable dunder method def __await__(self): # report a message print('Await is running') # suspend execution yield from asyncio.sleep(2).__await__() # report a message print('Await is done') # main coroutine async def main(): # create and await custom awaitable await CustomAwaitable() # start the asyncio event loop asyncio.run(main()) |
Running the example first creates the main() coroutine then starts the asyncio event loop.
The main() coroutine runs and creates an instance of our CustomAwaitable class and awaits it.
The __await__() method runs, reports a message, then creates the asyncio.sleep() coroutine object and delegates to it.
The asyncio.sleep() coroutine’s __await__() method runs, suspending the caller for two seconds.
Control is then returned and a final message is reported.
The main() coroutine then exits and the event loop is terminated.
This highlights how we can delegate to another awaitable from without a custom awaitable.
|
1 2 |
Await is running Await is done |
We don’t have to explicitly delegate to another awaitable.
Instead, we could return another awaitable.
This is different as when the control of execution is returned, it will be returned to the caller (that has an await expression), rather than to our custom awaitable.
For example:
|
1 2 3 4 5 6 |
# awaitable dunder method def __await__(self): # report a message print('Await is running') # suspend execution return asyncio.sleep(2).__await__() |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
# SuperFastPython.com # example of a custom awaitable that returns an awaitable import asyncio # define a custom awaitable class CustomAwaitable(object): # awaitable dunder method def __await__(self): # report a message print('Await is running') # suspend execution return asyncio.sleep(2).__await__() # main coroutine async def main(): # create and await custom awaitable await CustomAwaitable() # start the asyncio event loop asyncio.run(main()) |
Running the example first creates the main() coroutine then starts the asyncio event loop.
The main() coroutine runs and creates an instance of our CustomAwaitable class and awaits it.
The __await__() method runs, reports a message, then creates the asyncio.sleep() coroutine object and returns the result of its __await__() method.
The result of the asyncio.sleep() coroutine’s __await__() method is returned to the main() coroutine and awaited. This suspends the caller for two seconds.
Control is then returned to the main() coroutine and the event loop is terminated.
This highlights how we can return the result of another awaitables __await__() method from a custom awaitable.
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know how to define objects with the __await__() method.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any additional tips?
I’d love to hear about them!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Robson Hatsukami Morgan on Unsplash
Do you have any questions?