How to Execute Multiple Coroutines with asyncio.Runner

June 27, 2023 Python Asyncio

You can execute multiple coroutines in the same event loop using the asyncio.Runner class in the high-level asyncio API.

This is a new feature provided in Python 3.11. Before the addition of the asyncio.Runner class, we would have to execute each coroutine in a separate event loop, restructure our program to use a wrapper coroutine or delve into the low-level asyncio API.

In this tutorial, you will discover how to execute multiple coroutines in the same event loop from Python using the asyncio.Runner class.

Let's get started.

Need to Run Multiple Coroutines from Python

The typical way to run an asyncio program is to call asyncio.run() and pass in a coroutine object to execute.

For example:

...
asyncio.run(coro())

The single coroutine is provided to the asyncio.run() represents the entry point into the asyncio event loop.

There may be cases where we want to execute multiple separate coroutines from our Python program.

This could be achieved with multiple calls to asyncio.run(), but this would start and run a new asyncio event loop for each call.

For example:

...
asyncio.run(coro1())
asyncio.run(coro2())

Another solution is to restructure the program and create a new entry point coroutine that calls the multiple coroutines for us.

For example:

async def main():
	await coro1()
	await coro2()

asyncio.run(main())

Can we run multiple coroutines in the same event loop without having to restructure our program via the high-level asyncio API?

How can we call multiple coroutines directly from a Python program?

How to Run Multiple Coroutines with asyncio.Runner

Python 3.11 introduced a new feature to allow multiple coroutines to be run directly from a Python program.

This capability is provided via the asyncio.Runner class, provided in the high-level asyncio API.

Added the Runner class, which exposes the machinery used by run().
-- What’s New In Python 3.11

The asyncio.Runner can be created and used to execute multiple coroutines within the same event loop.

An instance of the class can be created and coroutines can be executed directly via the run() method. Once finished, the close() method can be called.

For example:

...
# create asyncio runner
runner = asyncio.Runner()
# execute first coroutine
runner.run(coro1())
# execute second coroutine
runner.run(coro2())
# close runner
runner.close()

Alternatively, we can use the context manager interface on the class which will close the event loop for us when we're finished with it.

For example:

...
# create asyncio runner
with asyncio.Runner() as runner:
	# execute first coroutine
	runner.run(coro1())
	# execute second coroutine
	runner.run(coro2())

Now that we know how to execute multiple coroutines in the same event loop from our Python program, let's look at some worked examples.

Example of Running Multiple Coroutines with asyncio.run()

Before we look at an example of using asyncio.Runner, let's look at how we would run multiple coroutines before the asyncio.Runner was provided, e.g. Python 3.10 and lower.

Consider the case where we have two coroutines to run from our Python program.

# example coroutine
async def task_coro1():
    print('Hello from first coro')
    await asyncio.sleep(1)

# another example coroutine
async def task_coro2():
    print('Hello from second coro')
    await asyncio.sleep(1)

There are two methods we could use:

Separate calls to asyncio.run()
Create a wrapper coroutine

Method 1: Separate Calls to asyncio.run()

One approach would be to run each coroutine separately.

For example:

# run first coroutine
asyncio.run(task_coro1())
# run second coroutine
asyncio.run(task_coro2())

A complete example using this method is listed below.

# example of using separate asyncio.run() calls
import asyncio

# example coroutine
async def task_coro1():
    print('Hello from first coro')
    await asyncio.sleep(1)

# another example coroutine
async def task_coro2():
    print('Hello from second coro')
    await asyncio.sleep(1)

# run first coroutine
asyncio.run(task_coro1())
# run second coroutine
asyncio.run(task_coro2())

Running the example first creates an asyncio event loop and uses the loop to execute the first coroutine.

The event loop is then closed and cleaned up.

A second asyncio event loop is created and used to execute the second coroutine before being closed and cleaned up.

Hello from first coro
Hello from second coro

This approach is expensive as a new event loop must be created and closed for each coroutine. This can become a problem if we have thousands or millions of coroutines to execute from Python.

Method 2: Wrapper Coroutine

Another approach is to define a new wrapper coroutine that in turn calls the coroutines for us using the single event loop.

For example:

# asyncio entry point
async def main():
    # execute the first coroutine
    await task_coro1()
    # execute the second coroutine
    await task_coro2()

The complete example of this change is listed below.

# example of using asyncio.run() with wrapper coroutine
from asyncio import run
from asyncio import sleep

# example coroutine
async def task_coro1():
    print('Hello from first coro')
    await sleep(1)

# another example coroutine
async def task_coro2():
    print('Hello from second coro')
    await sleep(1)

# asyncio entry point
async def main():
    # execute the first coroutine
    await task_coro1()
    # execute the second coroutine
    await task_coro2()

# entry point of the program
run(main())

Running the example creates a single asyncio event loop and executes the main() wrapper coroutine.

The main() coroutine then executes and awaits each task coroutine in turn before closing.

Hello from first coro
Hello from second coro

The problem with this approach is that the program must be restructured with a new wrapper coroutine used as the entry point for executing all coroutines.

This may mean that all data and coroutines needed must be collected and set up before executing the wrapper coroutine. It also does not allow the Python program to conditionally create and start coroutines based on the results of prior coroutines.

Next, let's look at an alternative using the asyncio.Runner class.

Example of Running Multiple Coroutines with asyncio.Runner

We can develop an example of executing multiple coroutines from a Python program using the asyncio.Runner class.

This can be achieved using the context manager interface for the asyncio.Runner and call the run() method for each coroutine to execute.

For example:

# entry point of the program
with Runner() as runner:
    # execute the first coroutine
    runner.run(task_coro1())
    # execute the second coroutine
    runner.run(task_coro2())

A complete example of this approach is listed below.

# example of using asyncio.Runner
from asyncio import Runner
from asyncio import sleep

# example coroutine
async def task_coro1():
    print('Hello from first coro')
    await sleep(1)

# another example coroutine
async def task_coro2():
    print('Hello from second coro')
    await sleep(1)

# entry point of the program
with Runner() as runner:
    # execute the first coroutine
    runner.run(task_coro1())
    # execute the second coroutine
    runner.run(task_coro2())

Running the example creates the asyncio.Runner using the context manager interface.

The first coroutine is run, then the second.

Each coroutine is executed in the same event loop and the loop used is not created until the first call to run().

Hello from first coro
Hello from second coro

This approach provides flexibility to the Python program, allowing it to execute coroutines in an ad hoc and even conditional manner within the same event loop.

Takeaways

You now know how to execute multiple coroutines in the same event loop from Python using the asyncio.Runner class.

If you enjoyed this tutorial, you will love my book: Python Asyncio Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.