Last Updated on September 12, 2022
You can cancel tasks in the ThreadPoolExecutor by calling the cancel() function on the Future object.
In this tutorial you will discover how to cancel tasks in a Python thread pool.
Let’s get started.
Unwanted Task Submitted to the ThreadPoolExecutor
The ThreadPoolExecutor in Python provides a pool of reusable threads for executing ad hoc tasks.
You can submit tasks to the thread pool by calling the submit() function and passing in the name of the function you wish to execute on another thread.
Calling the submit() function will return a Future object that allows you to check on the status of the task and get the result from the task once it completes.
You may submit tasks that you may later decide you no longer need to complete.
Letting the tasks run may use up unwanted resources.
Additionally, you may have to wait for all tasks in the thread pool to complete before you can shut it down.
The solution is to cancel unwanted tasks.
Run loops using all CPUs, download your FREE book to learn how.
Call cancel() on the Future to Cancel a Task
You can cancel tasks submitted to the ThreadPoolExecutor by calling the cancel() function on the Future object.
Recall that you will receive a Future object when you submit your task to the thread pool by calling the submit() function.
|
1 2 3 |
... # submit a task to the thread pool future = executor.submit(work) |
The Future object has a function called cancel() that will cancel the task if it has not yet started running and is not done.
|
1 2 3 |
... # cancel a task submitted to the thread pool was_canceled = future.cancel() |
The cancel() function returns True if the task was canceled, otherwise False if the task could not be canceled.
If you successfully cancel the task, it will not execute.
If you try to retrieve a result from a cancelled task by calling the result() function, a CancelledError will be raised. Similarly, if you try to get any exception raised while executing the task that was cancelled by calling the exception() function, a CancelledError will be raised.
For example:
|
1 2 3 4 5 6 |
... # get the result try: result = future.result() except CancelledError: print('No result, task was cancelled') |
You may fail to cancel a task submitted to the thread pool for one of three reasons:
- The task was already cancelled.
- The task is currently running.
- The task is done.
There is often some confusion with the difference between a task that has been submitted and a task that is running.
A ThreadPoolExecutor has a fixed number of threads, limiting the number of tasks that can be executed at once.
A task that is submitted to the ThreadPoolExecutor may not start running immediately if all threads in the pool are currently occupied. Once a thread in the pool becomes available, it will take a task from the internal queue of tasks and begin executing it, changing its status to running.
We can check if a task is running by calling the running() function on the Future object for the task.
Only tasks that are scheduled in the thread pool but have not yet started running can be cancelled.
Now that we know how to cancel tasks in a ThreadPoolExecutor, let’s look at a worked example.
How To Cancel a Task in the ThreadPoolExecutor
Let’s demonstrate how to cancel a task submitted to the ThreadPoolExecutor.
First, we can define a simple task that will sleep for a given number of seconds.
|
1 2 3 |
# mock task that will sleep for a moment def work(sleep_time): sleep(sleep_time) |
We can then create a thread pool with one thread and submit a long running task to occupy the single thread in the pool.
|
1 2 3 4 5 6 |
... # create a thread pool with ThreadPoolExecutor(1) as executor: # start a long running task future1 = executor.submit(work, 2) print(f'First task running={future1.running()}') |
Next, we can submit a second shorter task and confirm that it is not yet running.
|
1 2 3 4 |
... # start a second future2 = executor.submit(work, 0.1) print(f'Second task running={future2.running()}') |
We can then cancel the second task, and confirm that it indeed was canceled and will not run.
|
1 2 3 4 |
... # cancel the second task was_cancelled = future2.cancel() print(f'Second task was cancelled: {was_cancelled}') |
Tying this together, the complete example of cancelling a task in the ThreadPoolExecutor is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
# SuperFastPython.com # example of cancelling a task in a thread pool from time import sleep from concurrent.futures import ThreadPoolExecutor # mock task that will sleep for a moment def work(sleep_time): sleep(sleep_time) # create a thread pool with ThreadPoolExecutor(1) as executor: # start a long running task future1 = executor.submit(work, 2) print(f'First task running={future1.running()}') # start a second future2 = executor.submit(work, 0.1) print(f'Second task running={future2.running()}') # cancel the second task was_cancelled = future2.cancel() print(f'Second task was cancelled: {was_cancelled}') |
Running the example first starts the long running task and confirms that it is running.
The second short duration task is submitted and is not yet running.
We then cancel the second task and confirm that it will not run.
|
1 2 3 |
First task running=True Second task running=False Second task was cancelled: True |
Free Python ThreadPoolExecutor Course
Download your FREE ThreadPoolExecutor PDF cheat sheet and get BONUS access to my free 7-day crash course on the ThreadPoolExecutor API.
Discover how to use the ThreadPoolExecutor class including how to configure the number of workers and how to execute tasks asynchronously.
No Results or Exception from Cancelled Tasks
You cannot get results or exceptions from cancelled tasks.
If we try to get a result from a cancelled task by calling the result() function, a CancelledError would be raised. The same would be the case if we tried to get the exception by calling exception().
We can demonstrate this by updating the example from the previous section.
First, let’s update the target task function to return a value so we have a result to retrieve from the task. In this case, the argument that was passed in.
|
1 2 3 4 |
# mock task that will sleep for a moment def work(sleep_time): sleep(sleep_time) return sleep_time |
Next, we can attempt to get the result from the cancelled task (the short duration task) and handle the expected CancelledError that will be raised because the task was cancelled.
|
1 2 3 4 5 6 |
... # get the result try: result = future2.result() except CancelledError: print('No result, task was cancelled') |
Finally, we can try and get the exception from the cancelled task and handle the same expected CancelledError because the task was cancelled.
|
1 2 3 4 5 6 |
... # get the exception try: e = future2.exception() except CancelledError: print('No exception, task was cancelled') |
Tying this together, the complete example of attempting to get a result and an exception from a cancelled task is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# SuperFastPython.com # example of getting results and an exception from a cancelled task from time import sleep from concurrent.futures import ThreadPoolExecutor from concurrent.futures import CancelledError # mock task that will sleep for a moment def work(sleep_time): sleep(sleep_time) return sleep_time # create a thread pool with ThreadPoolExecutor(1) as executor: # start a long running task future1 = executor.submit(work, 2) print(f'First task running={future1.running()}') # start a second future2 = executor.submit(work, 0.1) print(f'Second task running={future2.running()}') # cancel the second task was_cancelled = future2.cancel() print(f'Second task was cancelled: {was_cancelled}') # get the result try: result = future2.result() except CancelledError: print('No result, task was cancelled') # get the exception try: e = future2.exception() except CancelledError: print('No exception, task was cancelled') |
Running the example first starts the long running task and confirms that it is running. The second short duration task is submitted, then successfully cancelled.
We then try to retrieve the result from the cancelled task and handle an expected CancelledError.
Finally, we try to retrieve any exception that occurred during the execution of the short duration task and a CancelledError is raised and handled, again because the task was cancelled.
|
1 2 3 4 5 |
First task running=True Second task running=False Second task was cancelled: True No result, task was cancelled No exception, task was cancelled |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Further Reading
This section provides additional resources that you may find helpful.
Books
- ThreadPoolExecutor Jump-Start, Jason Brownlee, (my book!)
- Concurrent Futures API Interview Questions
- ThreadPoolExecutor Class API Cheat Sheet
I also recommend specific chapters from the following books:
- Effective Python, Brett Slatkin, 2019.
- See Chapter 7: Concurrency and Parallelism
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python ThreadPoolExecutor: The Complete Guide
- Python ProcessPoolExecutor: The Complete Guide
- Python Threading: The Complete Guide
- Python ThreadPool: The Complete Guide
APIs
References
Takeaways
You now know how to cancel tasks that have been submitted to the Python ThreadPoolExecutor.
Do you have any questions about how to cancel tasks?
Ask your question in the comments below and I will do my best to answer.
Photo by Murillo de Paula on Unsplash
Hi Jason!
If ThreadPoolExecutor() is called with the parameter 1 your examples are showing the expected output; once you set it the parameter of max_workers to 2 or more, the output for the second task shows “False”.
I guess with 1 the second task is never starting at all …
Am I wrong?
ec2021
Yes, the example is contrived to demonstrate task cancellation.