Asyncio Periodic Task

February 27, 2024 Python Asyncio

You can run an asyncio task periodically in the background.

This requires developing a new periodic() coroutine that runs in a loop forever, each iteration sleeping for a given number of seconds and awaiting a target coroutine.

In this tutorial, you will discover how to develop a periodic task in asyncio programs.

Let's get started.

Need to Run a Periodic Task

Generally, a periodic task is a task that runs in the background repeatedly in a loop.

These tasks are designed to repeat automatically after a certain predefined fixed interval of time, often to perform routine or recurring actions within a software application. They provide a way to automate essential operations, ensuring that they are carried out consistently without manual intervention.

The repeated nature of the periodic task makes them different from other types of background tasks, like delayed tasks.

You can learn more about delayed asyncio tasks in the tutorial:

How to Run a Coroutine After a Delay in Python

Periodic tasks are valuable in various domains, from system maintenance and data processing to application monitoring and notification scheduling. They play a crucial role in maintaining the reliability and functionality of many software systems.

We often need to perform a periodic task in an asyncio program.

This may be for many reasons, such as:

Scheduled Maintenance: Perform routine maintenance tasks, such as database cleanup, cache clearing, or log rotation, on a regular schedule to keep the application running smoothly.
Data Synchronization: Periodically synchronize data between different systems, databases, or services to ensure data consistency and accuracy.
Background Processing: Run background tasks at predefined intervals to process data, generate reports, or perform calculations without blocking the main event loop.
Monitoring and Alerting: Periodically check the health and performance of the application and its dependencies. Send alerts or notifications when predefined conditions or thresholds are met.
Resource Management: Manage and monitor system resources, such as memory usage, CPU load, or network bandwidth, to prevent resource exhaustion and optimize resource allocation.
Caching and Invalidation: Refresh or invalidate cached data or results at regular intervals to ensure that users receive up-to-date information.
Scheduled Jobs: Schedule jobs for specific times or intervals, such as sending email newsletters, generating backups, or processing batch uploads.
Rate Limiting: Implement rate limiting or throttling mechanisms to control the frequency at which certain actions or requests can be performed, reducing the risk of abuse.
Data Aggregation: Aggregate data from multiple sources over time, such as collecting statistics, logs, or metrics for reporting and analysis.
Security Scanning: Perform periodic security scans and vulnerability assessments to identify and address potential security issues.

Given that periodic tasks are an important part of application development, how can we develop coroutines and tasks to run periodically in the background?

How can we develop a periodic task in asyncio?

How to Run a Periodic Task

We can develop a periodic task in asyncio as a coroutine that runs in a loop.

The coroutine takes the name of the target coroutine to run and any arguments to the coroutine, as well as the interval of time in seconds between running the target coroutine.

It then loops forever first sleeping for the provided fixed interval and then running and awaiting the target coroutine.

# helper function for running a target periodically
async def periodic(interval_sec, coro_name, *args, **kwargs):
    # loop forever
    while True:
        # wait an interval
        await asyncio.sleep(interval_sec)
        # await the target
        await coro_name(*args, **kwargs)

The periodic() coroutine can then be run as a background task in an asyncio program.

This can be achieved by creating and scheduling the configured periodic() coroutine as an asyncio task via the asyncio.create_task() function.

For example:

...
# create and schedule the periodic task
task = asyncio.create_task(periodic(1.0, work))

The asyncio program can then proceed with other tasks while the periodic coroutine runs in the background.

You can learn more about running asyncio coroutines in the background in the tutorial:

How to Run Asyncio Coroutine in the Background

Why Not Provide a Task to periodic()?

We cannot provide an asyncio.Task instance to the periodic() coroutine because asyncio.Task instances cannot be run more than once.

Attempting to run the same asyncio.Task task more than once results in an exception.

This is why we provide the name of a target coroutine and a new coroutine is created each time the loop runs within the periodic() coroutine.

Why Not Provide a Coroutine Object to periodic()?

We cannot provide a coroutine object to the periodic() coroutine.

The reason is that we cannot run the same coroutine more than once as it results in a RuntimeError exception:

RuntimeError: cannot reuse already awaited coroutine

This is why we provide the name of a target coroutine and a new coroutine is created each time the loop runs within the periodic() coroutine.

Don't We Need to Explicitly Stop the periodic() Task?

We do not need to explicitly stop or cancel the periodic() background task.

This is because the periodic() will be canceled automatically by the asyncio event loop when it is shut down.

We can choose to cancel the periodic task ourselves before closing our program. This can be helpful if the target coroutine needs to perform some cleanup before some other task, e.g. when performing cleanup when shutting down the event loop is not a good fit.

This can be achieved by calling the cancel() method on the periodic() task before closing the program.

For example:

...
# cancel the background task
task.cancel()
# wait for the task to cancel
try:
	await task
except asyncio.CancelledException:
	pass

This is called a cancel-and-wait pattern.

You can learn more about this pattern in the tutorial:

Asyncio Cancel Task and Wait

Now that we know how to run a periodic background task in asyncio, let's look at a worked example.

Example of Periodic Task

We can explore an example of running a periodic task in asyncio.

In this example, we will define a task that simply reports a message.

# general task
async def work():
    # report a message
    print('>Doing work:!')

We will then run this task every second using our periodic() coroutine developed above as a background task and allow it to run for a while.

# main coroutine
async def main():
    # report a message
    print('Main starting')
    # configure the periodic task
    task = asyncio.create_task(periodic(1.0, work))
    # do other things for a while
    await asyncio.sleep(5)
    # report a message
    print('Main done')

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of periodic task
import asyncio

# helper function for running a target periodically
async def periodic(interval_sec, coro_name, *args, **kwargs):
    # loop forever
    while True:
        # wait an interval
        await asyncio.sleep(interval_sec)
        # await the target
        await coro_name(*args, **kwargs)

# general task
async def work():
    # report a message
    print('>Doing work:!')

# main coroutine
async def main():
    # report a message
    print('Main starting')
    # configure the periodic task
    task = asyncio.create_task(periodic(1.0, work))
    # do other things for a while
    await asyncio.sleep(5)
    # report a message
    print('Main done')

# start the event loop
asyncio.run(main())

Running the example first starts the asyncio event loop and runs the main() coroutine.

The main() coroutine runs and reports a message.

It then creates and schedules the periodic() coroutine as a background task. The arguments to the periodic() coroutine are one second and the name of the "work" coroutine.

The main() coroutine then suspends and sleeps for 5 seconds.

The periodic() task runs and starts the while loop that runs forever.

In each iteration of the loop the periodic() task sleeps for one second, then creates and awaits the work() coroutine with no arguments.

The work() coroutine runs and reports a message, then terminates.

This cycle of sleeping and awaiting the work() coroutine is repeated a number of times.

Eventually, the main() coroutine resumes and reports a final message before terminating.

This closes the asyncio event loop and causes the periodic background task to terminate.

This highlights how we can execute a coroutine periodically in the background in our asyncio program.

Main starting
>Doing work:!
>Doing work:!
>Doing work:!
>Doing work:!
Main done

Takeaways

You now know how to develop a periodic task in asyncio programs.