Last Updated on September 12, 2022
You can run a periodic task in the background via a daemon thread.
In this tutorial you will discover how to run a periodic task in the background in Python.
Let’s get started.
Need for a Periodic Background Task
A thread is a thread of execution in a computer program.
Every Python program has at least one thread of execution called the main thread. Both processes and threads are created and managed by the underlying operating system.
Sometimes we may need to create additional threads in our program in order to execute code concurrently.
Python provides the ability to create and manage new threads via the threading module and the threading.Thread class.
You can learn more about Python threads in the guude:
In concurrent programming, we may need to run a task periodically in the background.
This may be for many reasons, such as:
- Reloading or refreshing data in the application.
- Storing or saving data from the application.
- Reporting or logging progress of the application.
Periodically means that the task is repeated after a consistent time interval, such as:
- Every few seconds.
- Every few minutes.
- Every few hours.
Background means that the task is not the main purpose of the application, but instead is a task that supports the main application.
How can we run a periodic task in the background in Python?
Run loops using all CPUs, download your FREE book to learn how.
How to Create a Periodic Background Task
You can run a periodic task in the background using a daemon thread with a while loop and a sleep.
A daemon thread is a background thread. Daemon is pronounced “dee-mon“, like the alternate spelling “demon“.
A thread may be configured to be a daemon or not, and most threads in concurrent programming, including the main thread, are non-daemon threads (not background threads) by default. The difference between daemon threads and non-daemon threads is that the process will exit if only daemon threads are running, whereas it cannot exit if at least one non-daemon thread is running.
As such, daemon threads are helpful for executing tasks in the background to support the non-daemon threads in an application.
If you are new to daemon threads, you can learn more about them here:
A Python threading.Thread instance can be configured to be a daemon thread.
We can configure a new thread to be a daemon thread by specifying the “daemon” argument to True in the constructor of the threading.Thread class.
For example:
|
1 2 3 |
... # create a new daemon thread thread = Thread(daemon=True, ...) |
We can configure a new daemon thread to execute a custom function that will execute a program specific task periodically.
For example we might define a new function named background_task() that takes an integer argument in seconds.
|
1 2 3 |
# task that runs at a fixed interval def background_task(interval_sec): # ... |
Then configure a new threading.Thread instance to execute this function via the “target” keyword and pass an argument to the function via the “args” argument. The thread can be made daemon via the “daemon” argument and the thread can be given a meaningful name via the “name” argument.
|
1 2 3 4 |
... # create and start the daemon thread print('Starting background task...') daemon = Thread(target=background_task, args=(3,), daemon=True, name='Background') |
Periodically means repeatedly after the same time interval for as long as the program is running.
This can be achieved with a while-loop that will execute our desired task each interval and sleep for a configured number of seconds.
For example:
|
1 2 3 4 5 6 7 |
... # run forever while True: # block for the interval sleep(interval_sec) # perform the task # ... |
Because the thread executing our task is a daemon thread, it will not prevent the program from exiting once the tasks of the main thread are finished.
Now that we know how to run a periodic task in the background, let’s look at a worked example.
Example of a Periodic Background Task
We can explore how to run a periodic background task in a daemon thread.
In this example, we will define a new function that will execute our task every few seconds. This new task will be started in the main thread and run in the background while the main thread proceeds with its own tasks.
Firstly, we can define a new function named background_task() that will execute our background task.
The function will take one argument which is the interval in seconds between each execution of the task.
|
1 2 3 |
# task that runs at a fixed interval def background_task(interval_sec): # ... |
We can then create a while-loop that will run for the duration of the program.
|
1 2 3 4 |
... # run forever while True: # ... |
Each iteration of the loop, the function will execute a custom background task and sleep for the configured interval. The order of the task execution and sleep does not matter and may be specific to your application, although a sleep first ensures an initial delay before the first execution of the task.
This may be desirable as it gives the main thread a chance to start running.
In this case, our periodic task will be to simply report a message, but it may be any application specific task you like.
|
1 2 3 4 5 |
... # block for the interval sleep(interval_sec) # perform the task print('Background task!') |
Tying this together, the complete background_task() function for executing our periodic background task is listed below.
|
1 2 3 4 5 6 7 8 |
# task that runs at a fixed interval def background_task(interval_sec): # run forever while True: # block for the interval sleep(interval_sec) # perform the task print('Background task!') |
Next, we can create a new threading.Thread instance and configure it to execute our background task function.
The name of the function can be specified via the “target” argument and the time interval can be passed via the “args” argument. We can also configure the thread to be a daemon background thread via the “daemon” argument and give the thread a meaningful name via the “name” argument.
We will configure the background task to run (about) every three seconds. We don’t have fine grained control over when the thread will run exactly, but it is close enough.
|
1 2 3 |
... print('Starting background task...') daemon = Thread(target=background_task, args=(3,), daemon=True, name='Background') |
Once defined, the new background thread can be started.
|
1 2 |
... daemon.start() |
Next, the main thread can carry on with the normal execution of the program.
We can simulate this with a sleep for ten seconds, allowing the background task to run a few times.
|
1 2 3 4 |
... # main thread is carrying on... print('Main thread is carrying on...') sleep(10) |
Finally, the main thread will finish its main task (of sleeping), and terminate. This will terminate the application as the background thread is a daemon thread and will be terminated automatically by the process.
|
1 2 |
... print('Main thread done.') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
# SuperFastPython.com # example of a periodic daemon thread from time import sleep from threading import Thread # task that runs at a fixed interval def background_task(interval_sec): # run forever while True: # block for the interval sleep(interval_sec) # perform the task print('Background task!') # create and start the daemon thread print('Starting background task...') daemon = Thread(target=background_task, args=(3,), daemon=True, name='Background') daemon.start() # main thread is carrying on... print('Main thread is carrying on...') sleep(10) print('Main thread done.') |
Running the example first creates and starts the daemon thread.
This runs our while-loop in our custom background task function. Each iteration of the loop, the thread will sleep for three seconds then report a message for the life-time of the program.
The main thread then carries on as per normal.
After ten seconds, the main thread wakes up, and terminates, terminating the entire program including the running daemon thread in the background.
Note, daemon threads are terminated abruptly and you may need to pay special attention to the handling of any open resources, like files and socket connections.
|
1 2 3 4 5 6 |
Starting background task... Main thread is carrying on... Background task! Background task! Background task! Main thread done. |
Free Python Threading Course
Download your FREE threading PDF cheat sheet and get BONUS access to my free 7-day crash course on the threading API.
Discover how to use the Python threading module including how to create and start new threads and how to use a mutex locks and semaphores
Further Reading
This section provides additional resources that you may find helpful.
Python Threading Books
- Python Threading Jump-Start, Jason Brownlee (my book!)
- Threading API Interview Questions
- Threading Module API Cheat Sheet
I also recommend specific chapters in the following books:
- Python Cookbook, David Beazley and Brian Jones, 2013.
- See: Chapter 12: Concurrency
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Threading: The Complete Guide
- Python ThreadPoolExecutor: The Complete Guide
- Python ThreadPool: The Complete Guide
APIs
References
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Takeaways
You now know how to run a periodic task in the background.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Christopher Burns on Unsplash
Thanks for all tutos.
Is it the same for long-running task if we setup while true in the main() before create daemon task?
Thanks
You’re welcome!
Yes, a long running daemon thread can use a while true loop and can be created and started from the main thread.
Here is an example:
How to Run a Long-Running Background Task in Python
https://superfastpython.com/thread-long-running-background-task/
This code is fine as long as the code to be executed is significantly faster than the period. The actual period of the task is interval_sec+code duration. In other word, this approach is ok as long no accuracy is required…
Great point!