You can mix concurrent.futures.Future and asyncio.Future objects in your Python program because they are not compatible.
This means that instances of the asyncio.Future class cannot be used in concurrent.futures module functions like concurrent.futures.wait() and concurrent.futures.as_completed() as it will cause an exception to be raised.
Similarly, instances of the concurrent.futures.Future class cannot be used in asyncio module functions like asyncio.wait() or asyncio.as_completed() as it can cause the program to hang or raise an exception.
In this tutorial, you will discover that concurrent.futures.Future and asyncio.Future objects are not compatible.
Let’s get started.
concurrent.futures.Future vs asyncio.Future
The Python standard library provides two Future classes.
The first is in the concurrent.futures module and the second is in the asyncio module.
The concurrent.futures.Future is a class that is part of the Executor framework for concurrency in Python.
It is used to represent a task executed asynchronously in the ThreadPoolExecutor and ProcessPoolExecutor classes.
The Future class encapsulates the asynchronous execution of a callable. Future instances are created by Executor.submit().
— concurrent.futures — Launching parallel tasks
An asyncio.Future class is a lower-level class in the asyncio module that represents a result that will eventually arrive.
Future objects are used to bridge low-level callback-based code with high-level async/await code.
— asyncio — Asynchronous I/O » Futures
The concurrent.futures.Future and asyncio.Future classes have many similarities.
This is not surprising as the asyncio.Future class was developed using the concurrent.futures.Future as inspiration.
The Future object was designed to mimic concurrent.futures.Future.
— asyncio — Asynchronous I/O » Futures
You can learn more about the concurrent.futures.Future versus the asyncio.Future class in the tutorial:
This raises an important question, are the concurrent.futures.Future and asyncio.Future classes compatible?
Run loops using all CPUs, download your FREE book to learn how.
concurrent.futures.Future and asyncio.Future Are Not Compatible
Both the concurrent.futures and asyncio modules provide utility module functions for working with collections of Future objects.
For example, the concurrent.futures module provides the following functions:
- concurrent.futures.wait() for waiting for a group of Future objects.
- concurrent.futures.as_completed() for processing Future objects in the order the tasks are completed.
The asyncio module provides the following functions:
- asyncio.gather() for running a collection of awaitable including Future objects.
- asyncio.wait() for waiting for a collection of awaitables including Future objects.
- asyncio.as_completed() for processing awaitable results, including Future objects, in the order they are completed.
Using a concurrent.futures.Future instances in the asyncio module functions will result in an exception.
Similarly, using a asyncio.Future instance in concurrent.futures module functions will result in an exception.
Additionally, type checking functions like asyncio.isfuture() will fail if passed an instance of the concurrent.futures.Future class.
The usage of the two classes within the Python standard library is not compatible.
Now that we know that the two Future objects are not compatible, let’s look at some worked examples.
Examples of asyncio.Future Not Compatible with concurrent.futures
In this section, we will explore examples that attempt to use instances of concurrent.futures.Future in asyncio module functions.
concurrent.futures.Future and asyncio.isfuture()
We can explore what happens if we try to use a concurrent.futures.Future in the asyncio.isfuture() function.
The asyncio.isfuture() function returns True if the provided object is an instance of an asyncio.Future, otherwise it returns False.
In this example, we will create a ThreadPoolExecutor and issue a task that reports a message, blocks a moment, and returns a value. The Future for the task is then checked via the asyncio.isfuture().
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
# SuperFastPython.com # example of concurrent.futures.Future in asyncio.isfuture() from time import sleep import concurrent.futures import asyncio # define a task function def task(value): # report a message print(f'Task {value} is running') # block for a moment sleep(1) # return a result return value # create a thread pool with concurrent.futures.ThreadPoolExecutor() as tpe: # issue a task future = tpe.submit(task, 0) # perform type check result = asyncio.isfuture(future) print(result) |
Running the example first creates a ThreadPoolExecutor with a default number of threads using the context manager interface.
Next, the task() function is issued as a task to the thread pool and a concurrent.futures.Future is returned.
The task runs, reports a message, and blocks a moment.
This Future is then checked via the asyncio.isfuture() function and the value is reported.
We can see that a result of False is reported showing that the concurrent.futures.Future is not recognized as an asyncio.Future.
|
1 2 |
Task 0 is running False |
concurrent.futures.Future and asyncio.wait()
We can explore what happens if we try to pass a concurrent.futures.Future object to the asyncio.wait() function.
In this example, we will issue multiple tasks to the ThreadPoolExecutor using submit(), which returns concurrent.futures.Future objects.
We will then pass a collection of the concurrent.futures.Future objects to the asyncio.wait() function.
The expectation is that this will fail with an exception as the asyncio.wait() is a coroutine and will not be awaited.
You can learn more about the asyncio.wait() function in the tutorial:
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
# SuperFastPython.com # example of concurrent.futures.Future in asyncio.wait() from time import sleep import concurrent.futures import asyncio # define a task function def task(value): # report a message print(f'Task {value} is running') # block for a moment sleep(1) # return a result return value # create a thread pool with concurrent.futures.ThreadPoolExecutor() as tpe: # issue tasks futures = [tpe.submit(task, i) for i in range(5)] # attempt to wait for the futures _ = asyncio.wait(futures) |
Running the example first creates the ThreadPoolExecutor using the context manager interface.
Next, five instances of the task() function are issued to the thread pool for execution using a list comprehension. This provides a list of concurrent.futures.Future objects.
This list of concurrent.futures.Future objects are then passed to the asyncio.wait() function that fails with a RuntimeWarning “coroutine ‘wait’ was never awaited“.
This is expected as the asyncio.wait() function is in fact a coroutine and was not awaited.
You can learn more about this warning in the tutorial:
|
1 2 3 4 5 6 7 8 |
Task 0 is running Task 1 is running Task 2 is running Task 3 is running Task 4 is running ...: RuntimeWarning: coroutine 'wait' was never awaited asyncio.wait(futures) RuntimeWarning: Enable tracemalloc to get the object allocation traceback |
We can update the example to correctly await the call to asyncio.wait().
This can be achieved by converting the program to an asyncio program.
A main() coroutine can be created in which the ThreadPoolExecutor is run and then the list of concurrent.futures.Future objects are passed to asyncio.wait() which is correctly awaited.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
# SuperFastPython.com # example of concurrent.futures.Future in asyncio.wait() from time import sleep import concurrent.futures import asyncio # define a task function def task(value): # report a message print(f'Task {value} is running') # block for a moment sleep(1) # return a result return value # main coroutine async def main(): # create a thread pool with concurrent.futures.ThreadPoolExecutor() as tpe: # issue tasks futures = [tpe.submit(task, i) for i in range(5)] # attempt to wait for the futures _ = await asyncio.wait(futures) # run the main coroutine asyncio.run(main()) |
Running the example first starts the asyncio runtime using the main() coroutine.
Next, the ThreadPoolExecutor is created using the context manager interface.
The five tasks are then issued to the thread pool using a list comprehension, providing a collection of concurrent.futures.Future objects.
The list of Future objects is then passed to the asyncio.wait() coroutine and awaited.
The tasks run and complete, reporting their messages.
The asyncio program hangs and the program must be terminated.
I suspect that the asyncio.wait() coroutine fails in some way, but the exception is consumed or masked internally.
This highlights that concurrent.futures.Future class is not compatible with the asyncio.wait() function.
|
1 2 3 4 5 6 7 |
Task 0 is running Task 1 is running Task 2 is running Task 3 is running Task 4 is running (has to be killed) |
concurrent.futures.Future and asyncio.as_completed()
We can explore what happens if we try to pass a concurrent.futures.Future object to the asyncio.as_completed() function.
In this example, we will issue multiple tasks to the ThreadPoolExecutor using submit(), which returns concurrent.futures.Future objects.
We will then pass a collection of the concurrent.futures.Future objects to the asyncio.as_completed() function and iterate over the return values.
The expectation is that this will fail with an exception as the asyncio.as_completed() function expects awaitable objects.
You can learn more about the asyncio.as_completed() function in the tutorial:
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
# SuperFastPython.com # example of concurrent.futures.Future in asyncio.as_completed() from time import sleep import concurrent.futures import asyncio # define a task function def task(value): # report a message print(f'Task {value} is running') # block for a moment sleep(1) # return a result return value # create a thread pool with concurrent.futures.ThreadPoolExecutor() as tpe: # issue tasks futures = [tpe.submit(task, i) for i in range(5)] # attempt to process futures in order of completion for future in asyncio.as_completed(futures): print(future.result()) |
Running the example first creates the ThreadPoolExecutor using the context manager interface.
Next, five instances of the task() function are issued to the thread pool for execution using a list comprehension. This provides a list of concurrent.futures.Future objects.
This list of concurrent.futures.Future objects are then passed to the asyncio.as_completed() function and the return values are iterated. This fails with a TypeError exception.
The asyncio.as_completed() function expects awaitables and the concurrent.futures.Future objects are not awaitable.
This highlights that concurrent.futures.Future objects are not compatible with the asyncio.as_completed() function.
|
1 2 3 4 5 6 7 8 |
Task 0 is running Task 1 is running Task 2 is running Task 3 is running Task 4 is running Traceback (most recent call last): ... TypeError: An asyncio.Future, a coroutine or an awaitable is required |
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Examples of concurrent.futures.Future Not Compatible with Asyncio
In this section, we will explore examples that attempt to use instances of asyncio.Future in concurrent.futures module functions.
Specifically, we will use instances of the asyncio.Task classes that are returned by asyncio. The asyncio.Task class extends the asyncio.Future class.
You can learn more about the asyncio.Task class in the tutorial:
asyncio.Future and concurrent.futures.wait()
We can explore what happens if we try to use asyncio.Future objects in the concurrent.futures.wait() function.
In this example, we will develop an asyncio program. A main() coroutine will run the program, issuing 5 task() coroutines via the asyncio.create_task() function. Each task reports a message, blocks a moment and returns a value. The asyncio.Task objects (that extend asyncio.Future) for each task() coroutine will be collected and passed to the concurrent.futures.wait() function.
The expectation is that an exception will be raised as the asyncio.Task objects will not have the expected methods and attributes of a concurrent.futures.Future object.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
# SuperFastPython.com # example of asyncio.Future in concurrent.futures.wait() import concurrent.futures import asyncio # define a coroutine task async def task(value): # report a message print(f'Task {value} is running') # block for a moment await asyncio.sleep(1) # return a result return value # main coroutine async def main(): # create coroutines and issue as tasks futures = [asyncio.create_task(task(i)) for i in range(5)] # wait for all tasks to complete done, rest = concurrent.futures.wait(futures) # run the asyncio event loop asyncio.run(main()) |
Running the example first runs the main() coroutine.
This then issues five task() coroutines for execution using a list comprehension and gathers the asyncio.Task objects (that extends the asyncio.Future object).
The tasks run as soon as they are able and report a message.
The concurrent.futures.wait() function is called with the list of Future objects. This fails with an AttributeError, as the concurrent.futures.wait() function expects the objects to have a _condition attribute.
This highlights that we cannot use asyncio.Task or asyncio.Future objects with the concurrent.futures.wait() function.
|
1 2 3 4 5 6 7 8 |
Task 0 is running Task 1 is running Task 2 is running Task 3 is running Task 4 is running Traceback (most recent call last): ... AttributeError: '_asyncio.Task' object has no attribute '_condition' |
We can update the example so that it works as expected.
This can be achieved by changing the call to concurrent.futures.wait() to a call to asyncio.wait() that is awaited.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
# SuperFastPython.com # example of asyncio.Future using asyncio.wait() import concurrent.futures import asyncio # define a coroutine task async def task(value): # report a message print(f'Task {value} is running') # block for a moment await asyncio.sleep(1) # return a result return value # main coroutine async def main(): # create coroutines and issue as tasks futures = [asyncio.create_task(task(i)) for i in range(5)] # wait for all tasks to complete done, rest = await asyncio.wait(futures) # run the asyncio event loop asyncio.run(main()) |
Running the example first runs the main() coroutine.
This then issues five task() coroutines for execution using a list comprehension and gathers the asyncio.Task objects (that extends the asyncio.Future object).
The main() coroutine blocks, waiting for all futures to finish.
The tasks run as soon as they are able and report a message, block, then return a value.
|
1 2 3 4 5 |
Task 0 is running Task 1 is running Task 2 is running Task 3 is running Task 4 is running |
asyncio.Future and concurrent.futures.as_completed()
We can explore what happens if we try to use asyncio.Future objects in the concurrent.futures.as_completed() function.
In this example, we will develop an asyncio program. A main() coroutine will run the program, issuing 5 task() coroutines via the asyncio.create_task() function. Each task reports a message, blocks a moment, and returns a value. The asyncio.Task objects (that extend asyncio.Future) for each task() coroutine will be collected and passed to the concurrent.futures.as_completed() and the results of each task are iterated.
The expectation is that an exception will be raised as the asyncio.Task objects will not have the expected methods and attributes of a concurrent.futures.Future object.
You can learn more about the concurrent.futures.as_completed() function in the tutorials below:
- How to Use as_completed() with the ThreadPoolExecutor in Python
- Get Results As Tasks Are Completed With ProcessPoolExecutor in Python
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
# SuperFastPython.com # example of asyncio.Future in concurrent.futures.as_completed() import concurrent.futures import asyncio # define a coroutine task async def task(value): # report a message print(f'Task {value} is running') # block for a moment await asyncio.sleep(1) # return a result return value # main coroutine async def main(): # create coroutines and issue as tasks futures = [asyncio.create_task(task(i)) for i in range(5)] # process task results as they complete for future in concurrent.futures.as_completed(futures): print(future.result()) # run the asyncio event loop asyncio.run(main()) |
Running the example first runs the main() coroutine.
This then issues five task() coroutines for execution using a list comprehension and gathers the asyncio.Task objects (that extends the asyncio.Future object).
The tasks run, reporting a message.
The list of Future objects is then passed to a call to the concurrent.futures.as_completed() function and the iterable of return values is iterated.
This fails with AttributeError as the concurrent.futures.as_completed() function expects concurrent.futures.Future objects that have a _condition attribute.
This highlights that we cannot pass asyncio.Future objects to the concurrent.futures.as_completed() function.
|
1 2 3 4 5 6 7 8 |
Task 0 is running Task 1 is running Task 2 is running Task 3 is running Task 4 is running Traceback (most recent call last): ... AttributeError: '_asyncio.Task' object has no attribute '_condition' |
We can update the example so that it works as expected.
This can be achieved by changing the call to the concurrent.futures.as_completed() function to the asyncio.as_completed() function and awaiting each coroutine that is returned to get the return value.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
# SuperFastPython.com # example of asyncio.Future in asyncio.as_completed() import concurrent.futures import asyncio # define a coroutine task async def task(value): # report a message print(f'Task {value} is running') # block for a moment await asyncio.sleep(1) # return a result return value # main coroutine async def main(): # create coroutines and issue as tasks futures = [asyncio.create_task(task(i)) for i in range(5)] # process task results as they complete for future in asyncio.as_completed(futures): print(await future) # run the asyncio event loop asyncio.run(main()) |
Running the example first runs the main() coroutine.
This then issues five task() coroutines for execution using a list comprehension and gathers the asyncio.Task objects (that extends the asyncio.Future object).
The list of Future objects are then passed to a call to the asyncio.as_completed() function and the iterable of return values is iterated.
The tasks run, reporting their message and returning a value. The asyncio.as_completed() iterates the tasks in the order they are completed, reporting the return values from each.
|
1 2 3 4 5 6 7 8 9 10 |
Task 0 is running Task 1 is running Task 2 is running Task 3 is running Task 4 is running 0 1 2 3 4 |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know that concurrent.futures.Future and asyncio.Future objects are not compatible.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Olav Tvedt on Unsplash
Do you have any questions?