How to Run a Follow-Up Task in Asyncio

January 12, 2023 Python Asyncio

You can schedule follow-up tasks in asyncio either directly from the primary task, from the caller of the primary task, or automatically from a done callback function.

In this tutorial, you will discover how to schedule and run follow-up asyncio tasks in Python.

Let's get started.

Need to Run a Follow-Up Task in Asyncio

We can execute coroutines as independent tasks by scheduling them with the asyncio.create_task() function.

For example:

...
# schedule a coroutine to execute independently
task = asyncio.create_task(coro())

Depending on the result of the task, we may need to execute a follow-up task.

This may be for many reasons, such as to gather more data, handle the result, or progress a broader process where the task is only one piece.

This is a common situation and we may issue many tasks and have their follow-up tasks handled automatically.

How can we issue follow-up tasks in asyncio?

How to Run a Follow-Up Task

There are three main ways to issue follow-up tasks in asyncio.

They are:

Schedule the follow-up task from the completed task itself.
Schedule the follow-up task from the caller.
Schedule the follow-up task automatically using done callback.

Let's take a closer look at each approach.

Schedule Follow-up Task From The Task Itself

The task that is completed can issue its own follow-up task.

This may require checking some state in order to determine whether the follow-up task should be issued or not.

The task can then be scheduled via a call to asyncio.create_task().

For example:

...
# schedule a follow-up task
task = asyncio.create_task(followup_task())

The task itself may choose to await the follow-up task or let it complete in the background independently.

For example:

...
# wait for the follow-up task to complete
await task

Schedule Follow-up Task From The Caller

The caller that issued the task can choose to issue a follow-up task.

For example, when the caller issues the first task, it may keep the asyncio.Task object.

It can then check the result of the task or whether the task was completed successfully or not.

The caller can then decide to issue a follow-up task.

It may or may not await the follow-up task directly.

For example:

...
# issue and await the first task
task = await asyncio.create_task(task())
# check the result of the task
if task.result():
	# issue the follow-up task
	followup = await asyncio.create_task(followup_task())

Schedule Follow-up Task Using a Callback Function

We can execute a follow-up task automatically using a done callback function.

For example, the caller that issues the task can register a done callback function on the task itself.

The done callback function must take the asyncio.Task object as an argument and will be called only after the task is done. It can then choose to issue a follow-up task.

The done callback function is a regular Python function, not a coroutine, so it cannot await the follow-up task

For example, the callback function may look as follows:

# callback function
def callback(task):
    # schedule and await the follow-up task
    _ = asyncio.create_task(followup())

The caller can issue the first task and register the done callback function.

For example:

...
# schedule and the task
task = asyncio.create_task(work())
# add the done callback function
task.add_done_callback(callback)

You can learn more about done callback functions for asyncio tasks in the tutorial:

How to Use Asyncio Task Done Callback Functions

Now that we know how to execute a follow-up task, let's look at some worked examples.

Example of a Follow-Up Task from the Task Itself

We can explore how to schedule a follow-up task from the primary task itself.

In this example, we will schedule and await the main task and then have the main task schedule and await its own follow-up task.

The complete example is listed below.

# SuperFastPython.com
# example of a follow-up task from the task itself
import asyncio

# follow-up task coroutine
async def followup():
    # report a message
    print('Follow-up task starting')
    # simulate work
    await asyncio.sleep(1)
    # report a message
    print('Follow-up task done')

# task coroutine
async def work():
    # report a message
    print('Task starting')
    # simulate work
    await asyncio.sleep(1)
    # report a message
    print('Task done')
    # schedule and await the follow-up task
    _ = await asyncio.create_task(followup())

# main coroutine
async def main():
    # schedule and await the task
    _ = await asyncio.create_task(work())

# run the asyncio program
asyncio.run(main())

Running the example first creates the main() coroutine and runs it as the entry point to the asyncio program.

The main() coroutine runs and schedules the primary task and suspends, waiting for it to complete.

The work() coroutine runs, reports a message, and sleeps for one second. It then reports a final message and then schedules the follow-up task, suspending it until it is complete.

The followup() coroutine runs, reports a message, sleeps, and reports its final message.

This highlights how a follow-up task may be scheduled directly from the primary task.

Task starting
Task done
Follow-up task starting
Follow-up task done

Example of a Follow-Up Task from the Caller

We can explore how to schedule a follow-up task from the caller of the main task.

In this example, the main coroutine will schedule and await the first task. It will then schedule and await the follow-up task.

The complete example is listed below.

# SuperFastPython.com
# example of a followup task from the caller
import asyncio

# follow-up task coroutine
async def followup():
    # report a message
    print('Follow-up task starting')
    # simulate work
    await asyncio.sleep(1)
    # report a message
    print('Follow-up task done')

# task coroutine
async def work():
    # report a message
    print('Task starting')
    # simulate work
    await asyncio.sleep(1)
    # report a message
    print('Task done')

# main coroutine
async def main():
    # schedule and await the task
    _ = await asyncio.create_task(work())
    # schedule and await the follow-up task
    _ = await asyncio.create_task(followup())

# run the asyncio program
asyncio.run(main())

Running the example first creates the main() coroutine and runs it as the entry point to the asyncio program.

The main() coroutine runs and schedules the primary task and suspends, waiting for it to complete.

The work() coroutine runs, reports a message and sleeps for one second, then reports a final message.

The main() coroutine resumes and schedules the follow-up coroutine and suspends, waiting for it to complete.

The followup() coroutine runs, reports a message, sleeps, and reports its final message.

This highlights how a follow-up task can be scheduled from the caller of the primary task.

Task starting
Task done
Follow-up task starting
Follow-up task done

Example of a Follow-Up Task Automatically With a Callback

We can explore how to automatically schedule a follow-up task using a done callback function.

In this example, we will schedule and await the first task. The follow-up task will be scheduled automatically via the done callback function.

The main coroutine will get a list of all running tasks and await them, ensuring the asyncio program does not exit before the follow-up task has had a chance to complete.

The complete example is listed below.

# SuperFastPython.com
# example of running a follow-up task automatically from a callback
import asyncio

# callback function
def callback(task):
    # schedule and await the follow-up task
    _ = asyncio.create_task(followup())

# follow-up task coroutine
async def followup():
    # report a message
    print('Follow-up task starting')
    # simulate work
    await asyncio.sleep(1)
    # report a message
    print('Follow-up task done')

# task coroutine
async def work():
    # report a message
    print('Task starting')
    # simulate work
    await asyncio.sleep(1)
    # report a message
    print('Task done')

# main coroutine
async def main():
    # schedule the task
    task = asyncio.create_task(work())
    # add the done callback
    task.add_done_callback(callback)
    # wait for task to complete
    await task
    # get a set of all tasks
    tasks = asyncio.all_tasks()
    # get the current task
    current = asyncio.current_task()
    # remove the current task from the set of all tasks
    tasks.remove(current)
    # wait for running tasks, e.g. the follow-up task
    _ = await asyncio.wait(tasks)

# run the asyncio program
asyncio.run(main())

Running the example first creates the main() coroutine and runs it as the entry point to the asyncio program.

The main() coroutine runs and schedules the primary task.

It then adds the done callback function to ensure the follow-up task will be scheduled automatically.

The main() coroutine then suspends and waits for the primary task to complete.

The work() coroutine runs, reports a message and sleeps for one second, then reports a final message.

The callback() done callback function runs and schedules the follow-up task.

The main() coroutine resumes and gets a set of all running tasks. This includes itself and the follow-up task.

It removes itself from the set and then waits for all running tasks to complete, which in this case is just the follow-up tasks.

This is required otherwise the main() coroutine will exit and terminate the asyncio program, not allowing the follow-up task to be complete.

The followup() coroutine runs, reports a message, sleeps, and reports its final message.

The program then ends.

This highlights how we can schedule and run a follow-up task automatically using a done callback function and to have the asyncio program wait for the background task to complete without prematurely terminating.

Task starting
Task done
Follow-up task starting
Follow-up task done

Takeaways

You now know how to schedule and run follow-up asyncio tasks in Python.

If you enjoyed this tutorial, you will love my book: Python Asyncio Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.