Asyncio Suspend Forever

April 9, 2024 Python Asyncio

The asyncio.Server in the asyncio module provides a way to suspend the main coroutine forever and accept client connections.

Reviewing the code in the standard library, we can see that this is achieved by creating a new and empty asyncio.Future and await it. We can use this approach in our own asyncio server applications to suspend the main coroutine forever.

In this tutorial, you will discover how to suspend the main coroutine forever, such as in a custom asyncio server.

Let's get started.

Asyncio Suspend Forever

The asyncio server provides a way to open a TCP or Unix socket server and accept client connections forever.

It offers this ability via the serve_forever() method.

Start accepting connections until the coroutine is cancelled. Cancellation of serve_forever task causes the server to be closed. This method can be called if the server is already accepting connections. Only one serve_forever task can exist per one Server object.
-- Event Loop

You can learn more about the asyncio server in the tutorial:

Asyncio Socket Servers

Calling this method never returns, unless the task is canceled or the program is terminated via a SIGINT (control-c).

Reviewing the code for this method in the standard library, we can see that it calls the loop.create_future() to get an empty asyncio.Future object then awaits this object.

For example:

...
self._serving_forever_fut = self._loop.create_future()

try:
    await self._serving_forever_fut
except exceptions.CancelledError:
    try:
        self.close()
        await self.wait_closed()
    finally:
        raise
finally:
    self._serving_forever_fut = None

This provides insight into a different way that we can suspend forever in our programs, such as when developing our own asyncio-based servers.

For more conventional ways to run the event loop forever, see the tutorial:

Asyncio Run Event Loop Forever

How to Suspend Forever

We can suspend our asyncio program forever by awaiting an empty asyncio.Future object in the main coroutine.

An empty future refers to an asyncio.Future object that is not associated with any specific asyncio.Task or coroutine. It does not run anything.

The body of the asyncio.Future objects __await__() method is a call to yield to itself, meaning that it will wait forever.

The main coroutine refers to the entry point to the asyncio program passed to the asyncio.run() module function.

You can learn more about the main coroutine in the tutorial:

What is the Main Coroutine

A way that we can suspend our program forever is to create a new asyncio.Future object and to await it directly.

For example:

...
# await a future forever
await asyncio.Future()

Another way to create an asyncio.Future is to get access to the asyncio event loop and call the create_future() method.

For example:

...
# get the event loop
loop = asyncio.get_running_loop()
# await a future forever
await loop.create_future()

This is the preferred way to create asyncio.Future objects in the low-level API.

The rule of thumb is to never expose Future objects in user-facing APIs, and the recommended way to create a Future object is to call loop.create_future(). This way alternative event loop implementations can inject their own optimized implementations of a Future object.
-- Futures

Either of these approaches provides a way to suspend our program forever.

Suspend Forever With Custom Awaitable

There is not a lot to an asyncio.Future object.

It implements the "future protocol" as described in the internal documentation, and provides a base class for asyncio.Task objects used on high-level asyncio programs.

We can create a new custom awaitable class that implements the __await__() method that can be awaited forever.

It can implement the minimum set of functionality required for the class to be awaited forever.

This involves just a few elements (determined after some trial and error), they are:

An implementation of the add_done_callback() method.
An implementation of the __await__() method.
Assigning a _loop attribute to the current event loop.
Assigning a _asyncio_future_blocking attribute to True.
Yielding self in the body of the __await__() method.

We can declare and define the required attributes as part of the class constructor or as part of the __await__() method itself. I prefer the latter to reduce the overall amount of code.

The complete example of a custom awaitable that can be awaited forever is listed below.

# custom awaitable
class CustomAwaitable:
    # add done callbacks
    def add_done_callback(self, fn, *, context=None):
        pass

    # await
    def __await__(self):
        # reference the event loop
        self._loop = asyncio.get_running_loop()
        # indicate that the awaitable is blocking
        self._asyncio_future_blocking = True
        # await self forever
        yield self

You can learn more about implementing a custom awaitable in the tutorial:

Asyncio Custom Awaitable With __await__()

This may be helpful if we require the capability to suspend our main coroutine forever for a custom server and have more control over how and when it may resume.

Really, it's just an experiment to learn more about the asyncio.Future class.

If more control is required in a custom server, then it would be preferable to extend the asyncio.Future or asyncio.Task classes and add the required functionality.

Now that we know how to suspend forever, let's look at some worked examples.

Example of Suspending Forever With An Empty Future

We can explore how to suspend the main coroutine forever using an empty future.

In this case, we will define a main coroutine that reports a message and creates and awaits a new asyncio.Future object, then reports a second message.

The second print statement is never reached as waiting for a new asyncio.Future object never returns.

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of suspending forever with an empty future
import asyncio

# main coroutine
async def main():
    # report a message
    print('Main running')
    # await a future forever
    await asyncio.Future()
    # report a message
    print('Main done')

# start the event loop
asyncio.run(main())

Running the example first creates the main() coroutine and starts the asyncio event loop.

The main() coroutine runs and reports messages.

It then creates a new asyncio.Future object and awaits it.

This suspends the main coroutine forever.

The program must be terminated by sending it a signal interrupt (SIGINT) via the Control-C key combination.

This highlights how we can await a new and empty asyncio.Future object to suspend our asyncio program forever.

This capability is helpful when developing custom asyncio servers that are required to run for an extended duration.

Main running

Next, let's explore how we can suspend forever using a custom awaitable.

Example of Suspending Forever With A Custom Awaitable

We can explore an example of suspending forever using a custom awaitable.

In this case, we will use the custom awaitable developed above based on a cut-down version of the asyncio.Future class to do one thing, suspend the main coroutine forever.

The complete example is listed below.

# SuperFastPython.com
# example of a custom awaitable that suspends forever
import asyncio

# custom awaitable
class CustomAwaitable:
    # add done callbacks
    def add_done_callback(self, fn, *, context=None):
        pass

    # await
    def __await__(self):
        # reference the event loop
        self._loop = asyncio.get_running_loop()
        # indicate that the awaitable is blocking
        self._asyncio_future_blocking = True
        # await self forever
        yield self

# main coroutine
async def main():
    # report a message
    print('Main running')
    # await a task forever
    await CustomAwaitable()
    # report a message
    print('Main done')

# start the asyncio event loop
asyncio.run(main())

Running the example first creates the main() coroutine and starts the asyncio event loop.

The main() coroutine runs and reports a message.

It then creates a new instance of our CustomAwaitable class and awaits it.

This suspends the main coroutine forever.

The program must be terminated by sending it a signal interrupt (SIGINT) via the Control-C key combination.

This highlights how we can await a custom awaitable object to suspend our asyncio program forever.

This approach may be helpful in those cases where we require specific capabilities related to suspending the main coroutine forever. Such as resuming occasionally.

Main running

Takeaways

You now know how to suspend the main coroutine forever.

If you enjoyed this tutorial, you will love my book: Python Asyncio Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.