You can develop a daemon asyncio task that runs in the background by running a coroutine in the background.
Background asyncio tasks will be canceled automatically when the event loop is terminated, meaning that we don’t need to explicitly terminate them at the end of our programs.
In this tutorial, you will discover how to develop a daemon asyncio task in Python.
Let’s get started.
What is a Daemon Task?
A daemon task is a background task in a computer program.
In multitasking computer operating systems, a daemon is a computer program that runs as a background process, rather than being under the direct control of an interactive user.
— Daemon (computing), Wikipedia.
In Python, we can explicitly start new threads that are daemon threads, as well as new daemon processes.
These threads and processes run in the background and do not prevent the main program from exiting. Instead, they are forcefully terminated when all of the non-daemon threads and processes have terminated.
You can learn more about how to develop daemon threads in the tutorial:
You can learn more about how to develop daemon processes in the tutorial:
Background tasks can be varied in type and specific to your application.
Some properties of these tasks might include:
- Sporadic: Tasks that run only when specific conditions arise (e.g. ad hoc logging).
- Periodic: Tasks that run after a consistent interval (e.g. data save/load every minute).
- Long-Running: Tasks that run for the duration of the program (e.g. monitoring a resource).
Now that we know what a daemon task is, let’s consider their need in asyncio programs.
Run loops using all CPUs, download your FREE book to learn how.
Need Daemon Tasks in Asyncio
We need daemon tasks when using asyncio.
A daemon task in asyncio serves various purposes, depending on the specific application and use case.
Below are five reasons why we might want to use a daemon task in asyncio:
- Background Jobs: Daemon tasks are ideal for running background jobs or periodic tasks that don’t need to be closely monitored or synchronized with the main application. This can include tasks like log cleanup, data synchronization, or periodic maintenance.
- Resource Management: You can use daemon tasks to manage resources or connections in the background. For example, in a server application, a daemon task could periodically check and close idle database connections or release other resources that are no longer needed.
- Heartbeats and Health Checks: Daemon tasks are useful for implementing health checks or heartbeats in networked applications. These tasks can run periodically to ensure that services are responsive and functioning correctly.
- Caching and Preloading: In scenarios where you want to preload or refresh cache data at regular intervals, daemon tasks can handle these caching operations without affecting the responsiveness of the main application.
- Monitoring and Metrics: Daemon tasks can be used to collect and send metrics or monitoring data to external systems. For instance, you can use daemon tasks to periodically sample performance metrics, gather usage statistics, and send this information to a monitoring service.
Daemon tasks in asyncio are typically lightweight and designed to run indefinitely in the background, helping to offload routine or non-critical tasks from the main application flow.
They contribute to better system resource utilization and maintainability, particularly in long-running asyncio applications.
Next, let’s consider how we might develop background daemon tasks in asyncio.
How to Develop a Daemon Task
We can develop a daemon task in asyncio using a background task.
A coroutine can be defined that runs in a loop, potentially forever.
For example:
|
1 2 3 4 5 |
# daemon task asyncio daemon_task(): # loop forever while True: # do things |
The coroutine can then be scheduled as a background asyncio task.
For example:
|
1 2 3 |
... # schedule the task for execution daemon = asyncio.create_task(daemon_task()) |
You can learn more about running asyncio tasks in the background in the tutorial:
It is a good idea to keep a reference to the task, either in the main coroutine or as a global variable. This is to avoid the disappearing task bug.
You can learn more about the disappearing task bug in the tutorial:
We might then want to suspend the current task for a moment to allow the background task to start running.
For example:
|
1 2 3 |
... # suspend a moment await asyncio.sleep(0) |
You can learn more about asyncio.sleep(0) in the tutorial:
The task will run in the background for as long as the event loop is running.
When the event loop is terminated, it will cancel all running tasks, including the daemon task.
Alternatively, we can cancel the task manually before exiting.
For example:
|
1 2 3 |
... # cancel the daemon task daemon.cancel() |
You can learn more about when tasks are canceled automatically in the tutorial:
And that is all there is to it.
Now that we know how to develop a daemon asyncio task, let’s look at a worked example.
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Example of Daemon Alarm Task
We can explore how to develop a daemon asyncio task.
In this case, the daemon task will be an alarm clock that will tick every second for 5 seconds, and then report an alarm message. This is a good placeholder for a general monitoring task or timed task.
Firstly, we can develop the daemon task itself.
The task will loop 5 times and each iteration report a message and sleep for one second. Once the loop exits, a final alarm message is reported.
The alarm() coroutine below implements this.
|
1 2 3 4 5 6 7 8 9 10 |
# alarm task async def alarm(): # main loop for i in range(5): # report a message print('>tick') # block a moment await asyncio.sleep(1) # alarm action print('ALARM!') |
Next, we can start the alarm() coroutine as a background task and then proceed with the main task of the program.
In this case, the main program does nothing interesting, it just sleeps for the duration.
The main() coroutine below implements this.
|
1 2 3 4 5 6 7 8 |
# main coroutine async def main(): # schedule the alarm task in the background task = asyncio.create_task(alarm()) # report a message print('Main doing other stuff...') # simulate continue on with other things await asyncio.sleep(6) |
Finally, we can start the asyncio event loop.
|
1 2 3 |
... # start the event loop asyncio.run(main()) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
# SuperFastPython.com # example of a daemon asyncio task that acts like an alarm clock import asyncio # alarm task async def alarm(): # main loop for i in range(5): # report a message print('>tick') # block a moment await asyncio.sleep(1) # alarm action print('ALARM!') # main coroutine async def main(): # schedule the alarm task in the background task = asyncio.create_task(alarm()) # report a message print('Main doing other stuff...') # simulate continue on with other things await asyncio.sleep(6) # start the event loop asyncio.run(main()) |
Running the example starts the asyncio event loop and runs the main() coroutine.
The main() coroutine runs and creates and schedules the alarm() coroutine as a task. It then reports a message and suspends with a sleep for 6 seconds.
The alarm() coroutine runs its loop, reporting a message and sleeping for one second each iteration. The loop ends and the alarm message is reported.
Finally, the main() coroutine resumes and terminates the event loop.
This highlights how we can run a background daemon task in an asyncio program.
|
1 2 3 4 5 6 7 |
Main doing other stuff... >tick >tick >tick >tick >tick ALARM! |
Next, let’s explore what happens if the event loop terminates while the daemon task is still running.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Daemon Alarm Task With Cancellation
We can explore an example of the daemon task being canceled automatically when the asyncio event loop is terminated.
In this case, we can update the above example so that the main() coroutine terminates before the alarm daemon task is able to be completed, e.g. in less than 5 seconds.
|
1 2 3 4 5 6 7 8 |
# main coroutine async def main(): # schedule the alarm task in the background task = asyncio.create_task(alarm()) # report a message print('Main doing other stuff...') # simulate continue on with other things await asyncio.sleep(3) |
The expectation is that when the event loop is terminated it explicitly cancels all remaining running tasks.
This means that the alarm daemon task will be canceled.
We can confirm this by updating the alarm() daemon task to catch the CancelledError exception, report a message, and then re-raise it to allow the task to be terminated.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
# alarm task async def alarm(): try: # main loop for i in range(5): # report a message print('>tick') # block a moment await asyncio.sleep(1) # alarm action print('ALARM!') except asyncio.CancelledError: # alert that the alarm was cancelled print('Alarm cancelled') # re-raise the exception raise |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# SuperFastPython.com # example of a daemon alarm task with cancellation import asyncio # alarm task async def alarm(): try: # main loop for i in range(5): # report a message print('>tick') # block a moment await asyncio.sleep(1) # alarm action print('ALARM!') except asyncio.CancelledError: # alert that the alarm was cancelled print('Alarm cancelled') # re-raise the exception raise # main coroutine async def main(): # schedule the alarm task in the background task = asyncio.create_task(alarm()) # report a message print('Main doing other stuff...') # simulate continue on with other things await asyncio.sleep(3) # start the event loop asyncio.run(main()) |
Running the example starts the asyncio event loop and runs the main() coroutine.
The main() coroutine runs and creates and schedules the alarm() coroutine as a task. It then reports a message and suspends with a sleep for 3 seconds.
The alarm() coroutine runs its loop, reporting a message and sleeping for one second each iteration.
The main() coroutine resumes after 3 seconds before the alarm daemon task has been completed. It terminates the event loop.
This causes the alarm() daemon task to be canceled. A CancelledError exception is raised in alarm(). It is handled and the cancellation message is reported. The CancelledError is then re-raised, allowing the task to be terminated.
This highlights how a background daemon task in an asyncio program can be canceled automatically when the event loop is terminated.
|
1 2 3 4 5 |
Main doing other stuff... >tick >tick >tick Alarm cancelled |
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know how to develop a daemon asyncio task in Python.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any additional tips?
I’d love to hear about them!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Frankie Dixon on Unsplash
Do you have any questions?