SuperFastPython

Asyncio Run Event Loop Forever

April 2, 2024 Python Asyncio

We can run an asyncio event loop "forever", until explicitly stopped or killed.

This is required in many applications that offer a service, accept client connections, or perform long-term event handling.

In this tutorial, you will discover how to run an asyncio program forever.

After completing this tutorial, you will know:

Let's get started.

Need to Run Asyncio Event Loop Forever

It is common to need to run an asyncio program forever.

Not literally "forever", but to have the program start and not stop until it is explicitly terminated.

This means that internally, the asyncio program runs in some type of endless loop performing the desired activity.

This may be for many reasons, such as:

  1. Continuous Service: Many asyncio applications, especially servers, are designed to run indefinitely, continuously servicing incoming requests or handling various asynchronous tasks. Looping forever allows the application to stay active and responsive, consistently handling new events or connections.
  2. Event Handling: In an event-driven paradigm like asyncio, the event loop continuously monitors and processes events or coroutines. Looping indefinitely ensures that the application remains responsive to incoming events, such as user interactions, I/O operations, or timers, without exiting prematurely.
  3. Persistent Connections: For network-based applications or services (like chat servers or streaming services), maintaining persistent connections with clients or external systems often requires an infinite loop to listen for incoming data, manage connections, and respond accordingly. This enables continuous communication between parties without terminating the application.

Looping forever in asyncio allows applications to remain active, responsive, and available for handling asynchronous events or tasks without exiting or shutting down.

How can we run an asyncio program "forever"?

What are the general approaches that are available?

How to Run the Asyncio Event Loop Forever

There are three ways that we can run an asyncio loop forever, they are:

  1. While Loop
  2. asyncio.Server.serve_forever()
  3. loop.run_forever()

Let's take a closer look at each approach in turn.

Asyncio Run Forever With While Loop

We can run forever in asyncio by defining a coroutine with a while loop that never ends.

For example:

# custom coroutine
async def custom_coroutine():
	while True:
		# do things

We can then start the asyncio event loop and run our coroutine.

For example:

...
# start the event loop and run our coroutine
asyncio.run(custom_coroutine())

This will run our coroutine forever, or until we terminate the program.

You can learn more about running an asyncio coroutine as a program in the tutorial:

Asyncio Run Forever With serve_forever()

An asyncio server can run forever accepting client connections.

This can be achieved by creating a TCP or Unix socket server, and then calling the serve_forever() method on the server.

For example:

...
# create a socket server
server = asyncio.start_server(...)
# ensure connections are closed safely
async with server:
	# accept client connections forever
	server.serve_forever()

This will allow the server to accept new client connections forever, or until we terminate the program.

You can learn more about the asyncio server in the tutorial:

Asyncio Run Forever With run_forever()

We can run an asyncio program forever using the low-level asyncio API (intended for library developers, not application programmers).

This requires first creating and starting an asyncio event loop via the asyncio.new_event_loop() function.

For example:

...
# create a new event loop (low-level api)
loop = asyncio.new_event_loop()

Next, we can schedule our custom coroutine to run in the event loop via the create_task() method.

For example:

...
# schedule our coroutine
loop.create_task(custom_coroutine())

Finally, we can run the event loop forever, until it is explicitly stopped via the run_forever() method.

For example:

...
# run the event loop until stopped
loop.run_forever()

You can learn more about the difference between the high-level and low-level asyncio APIs in the tutorial:

Now that we know how to run an asyncio program "forever", let's look at some worked examples.

Example Run Asyncio Forever With While Loop

We can explore how to run an asyncio event loop forever with a while loop in a custom coroutine.

In this example, we will define a custom coroutine that has a while loop that never ends.

Within the loop, we will sleep for one second and report a message, then repeat.

The main() coroutine below implements this.

# entry point
async def main():
    # loop forever
    while True:
        # run activities
        await asyncio.sleep(1)
        # report progress
        print('Running...')

We can then start the asyncio event loop and run this coroutine.

...
# start and run the event loop
asyncio.run(main())

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of running the event loop forever with a while loop
import asyncio

# entry point
async def main():
    # loop forever
    while True:
        # run activities
        await asyncio.sleep(1)
        # report progress
        print('Running...')

# start and run the event loop
asyncio.run(main())

Running the example first starts the asyncio event loop and runs our main() coroutine.

The main() coroutine runs and starts the while loop. Each iteration it sleeps for one second and reports a message.

The program must be terminated manually, such as via the Control-C key combination that sends a signal interrupt (SIGINT) signal to the program and forces it to exit.

This highlights how we can run an asyncio program forever using a while loop.

Running...
Running...
Running...
Running...
...

Next, let's look at how we can configure an asyncio server to accept client connections forever.

Example Run Asyncio Server Forever With serve_forever()

We can explore an example of running an asyncio server forever and accepting client connections.

In this example we will create a TCP socket server and then configure the server to run forever, accepting client connections.

Firstly, we must define the client callback coroutine, called to run each new client connection. In this case, we don't expect any client connections, so we can leave it empty.

# client callback handler
async def handler():
    pass

Next, we can define the main coroutine that first creates a new TCP socket server instance, configured to run on the local host on port 8888.

...
# create and start tcp socket server
server = await asyncio.start_server(handler, '127.0.0.1', 8888)
# report the details of the server
print(server)

Next, we can use the context manager interface on the server to ensure that the client connections are closed correctly when the server is terminated.

Within the context manager we can call the serve_forever() method on the asyncio server. This will suspend the main() coroutine forever and allow the server to accept client connections as they come in.

...
# use context manager interface to handle closing safely
async with server:
    # accept client connections forever (kill via control-c)
    await server.serve_forever()

Tying this together, the main() coroutine below implements this.

# entry point
async def main():
    # create and start tcp socket server
    server = await asyncio.start_server(handler, '127.0.0.1', 8888)
    # report the details of the server
    print(server)
    # use context manager interface to handle closing safely
    async with server:
        # accept client connections forever (kill via control-c)
        await server.serve_forever()

Finally, we can start the asyncio event loop and run our main() coroutine.

...
# start and run the event loop
asyncio.run(main())

Tying this together, the complete example of the asyncio server accepting client connections forever is listed below.

# SuperFastPython.com
# example of running the event loop forever a server
import asyncio

# client callback handler
async def handler():
    pass

# entry point
async def main():
    # create and start tcp socket server
    server = await asyncio.start_server(handler, '127.0.0.1', 8888)
    # report the details of the server
    print(server)
    # use context manager interface to handle closing safely
    async with server:
        # accept client connections forever (kill via control-c)
        await server.serve_forever()

# start and run the event loop
asyncio.run(main())

Running the example first starts the asyncio event loop and runs our main() coroutine.

The main() coroutine runs and creates and starts the asyncio server. The details of the server are reported.

Next, the context manager is opened and entered, then the asyncio server accepts client socket connections forever.

The program must be terminated manually, such as via the Control-C key combination that sends a signal interrupt (SIGINT) signal to the program and forces it to exit.

This highlights how we can run an asyncio program forever by accepting client connections.

<Server sockets=(<asyncio.TransportSocket fd=6, family=2, type=1, proto=6, laddr=('127.0.0.1', 8888)>,)>

Next, let's look at how we can run an asyncio program forever with the low-level asyncio API.

Example Run Asyncio Forever With run_forever()

We can explore an example of running an asyncio program forever using the low-level asyncio API.

In this case, we can define a custom coroutine that performs some activity, in this case, a loop that iterates 5 times, sleeps, and reports a message.

The coroutine can perform any long-running or never-ending task we require.

# entry point
async def main():
    # loop a little
    for i in range(5):
        # run activities
        await asyncio.sleep(1)
        # report progress
        print(f'Running {i}...')
    # this guy is done
    print('main() is done, but the loop carries on...')

Next, we can create a new asyncio event loop, manually schedule our main() coroutine to run in the event loop, and allow the event loop to run forever until stopped by calling the run_forever() method.

...
# create a new event loop (low-level api)
loop = asyncio.new_event_loop()
# schedule our coroutine
loop.create_task(main())
# run the event loop until stopped
loop.run_forever()

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of running the event loop forever with loop_forever()
import asyncio

# entry point
async def main():
    # loop a little
    for i in range(5):
        # run activities
        await asyncio.sleep(1)
        # report progress
        print(f'Running {i}...')
    # this guy is done
    print('main() is done, but the loop carries on...')

# create a new event loop (low-level api)
loop = asyncio.new_event_loop()
# schedule our coroutine
loop.create_task(main())
# run the event loop until stopped
loop.run_forever()

Running the example first creates a new asyncio event loop.

Next, our main() coroutine is created and scheduled in the event loop.

The event loop is then run until it is manually stopped.

The main() coroutine runs and executes the loop, reporting messages and sleeping each iteration, before reporting a final message.

The main() coroutine ends, although the event loop continues to run. Additional coroutines and tasks could be scheduled in the event loop.

The program must be terminated manually, such as via the Control-C key combination that sends a signal interrupt (SIGINT) signal to the program and forces it to exit.

This highlights how we can run an asyncio program forever using the low-level API.

Running 0...
Running 1...
Running 2...
Running 3...
Running 4...
main() is done, but the loop carries on...

Takeaways

You now know how to run an asyncio program forever.

Specifically, you know:



If you enjoyed this tutorial, you will love my book: Python Asyncio Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.