Daemon Asyncio Task in Python

January 7, 2024 Python Asyncio

You can develop a daemon asyncio task that runs in the background by running a coroutine in the background.

Background asyncio tasks will be canceled automatically when the event loop is terminated, meaning that we don't need to explicitly terminate them at the end of our programs.

In this tutorial, you will discover how to develop a daemon asyncio task in Python.

Let's get started.

What is a Daemon Task?

A daemon task is a background task in a computer program.

In multitasking computer operating systems, a daemon is a computer program that runs as a background process, rather than being under the direct control of an interactive user.
-- Daemon (computing), Wikipedia.

In Python, we can explicitly start new threads that are daemon threads, as well as new daemon processes.

These threads and processes run in the background and do not prevent the main program from exiting. Instead, they are forcefully terminated when all of the non-daemon threads and processes have terminated.

You can learn more about how to develop daemon threads in the tutorial:

Daemon Threads in Python

You can learn more about how to develop daemon processes in the tutorial:

Daemon Process in Python

Background tasks can be varied in type and specific to your application.

Some properties of these tasks might include:

Sporadic: Tasks that run only when specific conditions arise (e.g. ad hoc logging).
Periodic: Tasks that run after a consistent interval (e.g. data save/load every minute).
Long-Running: Tasks that run for the duration of the program (e.g. monitoring a resource).

Now that we know what a daemon task is, let's consider their need in asyncio programs.

Need Daemon Tasks in Asyncio

We need daemon tasks when using asyncio.

A daemon task in asyncio serves various purposes, depending on the specific application and use case.

Below are five reasons why we might want to use a daemon task in asyncio:

Background Jobs: Daemon tasks are ideal for running background jobs or periodic tasks that don't need to be closely monitored or synchronized with the main application. This can include tasks like log cleanup, data synchronization, or periodic maintenance.
Resource Management: You can use daemon tasks to manage resources or connections in the background. For example, in a server application, a daemon task could periodically check and close idle database connections or release other resources that are no longer needed.
Heartbeats and Health Checks: Daemon tasks are useful for implementing health checks or heartbeats in networked applications. These tasks can run periodically to ensure that services are responsive and functioning correctly.
Caching and Preloading: In scenarios where you want to preload or refresh cache data at regular intervals, daemon tasks can handle these caching operations without affecting the responsiveness of the main application.
Monitoring and Metrics: Daemon tasks can be used to collect and send metrics or monitoring data to external systems. For instance, you can use daemon tasks to periodically sample performance metrics, gather usage statistics, and send this information to a monitoring service.

Daemon tasks in asyncio are typically lightweight and designed to run indefinitely in the background, helping to offload routine or non-critical tasks from the main application flow.

They contribute to better system resource utilization and maintainability, particularly in long-running asyncio applications.

Next, let's consider how we might develop background daemon tasks in asyncio.

How to Develop a Daemon Task

We can develop a daemon task in asyncio using a background task.

A coroutine can be defined that runs in a loop, potentially forever.

For example:

# daemon task
asyncio daemon_task():
	# loop forever
	while True:
		# do things

The coroutine can then be scheduled as a background asyncio task.

For example:

...
# schedule the task for execution
daemon = asyncio.create_task(daemon_task())

You can learn more about running asyncio tasks in the background in the tutorial:

How to Run Asyncio Coroutine in the Background

It is a good idea to keep a reference to the task, either in the main coroutine or as a global variable. This is to avoid the disappearing task bug.

You can learn more about the disappearing task bug in the tutorial:

Asyncio Disappearing Task Bug

We might then want to suspend the current task for a moment to allow the background task to start running.

For example:

...
# suspend a moment
await asyncio.sleep(0)

You can learn more about asyncio.sleep(0) in the tutorial:

What is asyncio.sleep(0)

The task will run in the background for as long as the event loop is running.

When the event loop is terminated, it will cancel all running tasks, including the daemon task.

Alternatively, we can cancel the task manually before exiting.

For example:

...
# cancel the daemon task
daemon.cancel()

You can learn more about when tasks are canceled automatically in the tutorial:

When Are Asyncio Tasks Canceled

And that is all there is to it.

Now that we know how to develop a daemon asyncio task, let's look at a worked example.

Example of Daemon Alarm Task

We can explore how to develop a daemon asyncio task.

In this case, the daemon task will be an alarm clock that will tick every second for 5 seconds, and then report an alarm message. This is a good placeholder for a general monitoring task or timed task.

Firstly, we can develop the daemon task itself.

The task will loop 5 times and each iteration report a message and sleep for one second. Once the loop exits, a final alarm message is reported.

The alarm() coroutine below implements this.

# alarm task
async def alarm():
    # main loop
    for i in range(5):
        # report a message
        print('>tick')
        # block a moment
        await asyncio.sleep(1)
    # alarm action
    print('ALARM!')

Next, we can start the alarm() coroutine as a background task and then proceed with the main task of the program.

In this case, the main program does nothing interesting, it just sleeps for the duration.

The main() coroutine below implements this.

# main coroutine
async def main():
    # schedule the alarm task in the background
    task = asyncio.create_task(alarm())
    # report a message
    print('Main doing other stuff...')
    # simulate continue on with other things
    await asyncio.sleep(6)

Finally, we can start the asyncio event loop.

...
# start the event loop
asyncio.run(main())

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of a daemon asyncio task that acts like an alarm clock
import asyncio

# alarm task
async def alarm():
    # main loop
    for i in range(5):
        # report a message
        print('>tick')
        # block a moment
        await asyncio.sleep(1)
    # alarm action
    print('ALARM!')

# main coroutine
async def main():
    # schedule the alarm task in the background
    task = asyncio.create_task(alarm())
    # report a message
    print('Main doing other stuff...')
    # simulate continue on with other things
    await asyncio.sleep(6)

# start the event loop
asyncio.run(main())

Running the example starts the asyncio event loop and runs the main() coroutine.

The main() coroutine runs and creates and schedules the alarm() coroutine as a task. It then reports a message and suspends with a sleep for 6 seconds.

The alarm() coroutine runs its loop, reporting a message and sleeping for one second each iteration. The loop ends and the alarm message is reported.

Finally, the main() coroutine resumes and terminates the event loop.

This highlights how we can run a background daemon task in an asyncio program.

Main doing other stuff...
>tick
>tick
>tick
>tick
>tick
ALARM!

Next, let's explore what happens if the event loop terminates while the daemon task is still running.

Example of Daemon Alarm Task With Cancellation

We can explore an example of the daemon task being canceled automatically when the asyncio event loop is terminated.

In this case, we can update the above example so that the main() coroutine terminates before the alarm daemon task is able to be completed, e.g. in less than 5 seconds.

# main coroutine
async def main():
    # schedule the alarm task in the background
    task = asyncio.create_task(alarm())
    # report a message
    print('Main doing other stuff...')
    # simulate continue on with other things
    await asyncio.sleep(3)

The expectation is that when the event loop is terminated it explicitly cancels all remaining running tasks.

This means that the alarm daemon task will be canceled.

We can confirm this by updating the alarm() daemon task to catch the CancelledError exception, report a message, and then re-raise it to allow the task to be terminated.

# alarm task
async def alarm():
    try:
        # main loop
        for i in range(5):
            # report a message
            print('>tick')
            # block a moment
            await asyncio.sleep(1)
        # alarm action
        print('ALARM!')
    except asyncio.CancelledError:
        # alert that the alarm was cancelled
        print('Alarm cancelled')
        # re-raise the exception
        raise

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of a daemon alarm task with cancellation
import asyncio

# alarm task
async def alarm():
    try:
        # main loop
        for i in range(5):
            # report a message
            print('>tick')
            # block a moment
            await asyncio.sleep(1)
        # alarm action
        print('ALARM!')
    except asyncio.CancelledError:
        # alert that the alarm was cancelled
        print('Alarm cancelled')
        # re-raise the exception
        raise

# main coroutine
async def main():
    # schedule the alarm task in the background
    task = asyncio.create_task(alarm())
    # report a message
    print('Main doing other stuff...')
    # simulate continue on with other things
    await asyncio.sleep(3)

# start the event loop
asyncio.run(main())

Running the example starts the asyncio event loop and runs the main() coroutine.

The main() coroutine runs and creates and schedules the alarm() coroutine as a task. It then reports a message and suspends with a sleep for 3 seconds.

The alarm() coroutine runs its loop, reporting a message and sleeping for one second each iteration.

The main() coroutine resumes after 3 seconds before the alarm daemon task has been completed. It terminates the event loop.

This causes the alarm() daemon task to be canceled. A CancelledError exception is raised in alarm(). It is handled and the cancellation message is reported. The CancelledError is then re-raised, allowing the task to be terminated.

This highlights how a background daemon task in an asyncio program can be canceled automatically when the event loop is terminated.

Main doing other stuff...
>tick
>tick
>tick
Alarm cancelled

Takeaways

You now know how to develop a daemon asyncio task in Python.