Last Updated on November 14, 2023
It is a good practice that any waiting performed in an asyncio program be limited to a timeout.
Asyncio provides a way to wait on another task with a timeout via the asyncio.wait_for() function. If the timeout elapses before the task completes, the task is canceled.
In this tutorial, you will discover how to wait for an asyncio task with a timeout.
After completing this tutorial, you will know:
- How to wait on another task with a timeout.
- How to handle a timeout while waiting on another task.
- How to handle the failure or cancellation of task that is being waited on with a timeout.
Let’s get started.
What is Asyncio wait_for()
The asyncio.wait_for() function allows the caller to wait for an asyncio task or coroutine to complete with a timeout.
If no timeout is specified, the wait_for() function will wait until the task is completed.
If a timeout is specified and elapses before the task is complete, then the task is canceled.
Wait for the aw awaitable to complete with a timeout.
— Coroutines and Tasks
This allows the caller to both sets an expectation about how long they are willing to wait for a task to complete, and to enforce the timeout by canceling the task if the timeout elapses.
Now that we know what the asyncio.wait_for() function is, let’s look at how to use it.
Run loops using all CPUs, download your FREE book to learn how.
How to Use Asyncio wait_for()
The asyncio.wait_for() function takes an awaitable and a timeout.
The awaitable may be a coroutine or a task.
A timeout must be specified and may be None for no timeout, an integer or floating point number of seconds.
The wait_for() function returns a coroutine that is not executed until it is explicitly awaited or scheduled as a task.
For example:
|
1 2 3 |
... # wait for a task to complete await asyncio.wait_for(coro, timeout=10) |
If a coroutine is provided, it will be converted to the task when the wait_for() coroutine is executed.
If the timeout elapses before the task is completed, the task is canceled, and an asyncio.TimeoutError is raised, which may need to be handled.
For example:
|
1 2 3 4 5 6 7 |
... # execute a task with a timeout try: # wait for a task to complete await asyncio.wait_for(coro, timeout=1) except asyncio.TimeoutError: # ... |
If the waited-for task fails with an unhandled exception, the exception will be propagated back to the caller that is awaiting on the wait_for() coroutine, in which case it may need to be handled.
For example
|
1 2 3 4 5 6 7 8 9 |
... # execute a task that may fail try: # wait for a task to complete await asyncio.wait_for(coro, timeout=1) except asyncio.TimeoutError: # ... except Exception: # ... |
Next, let’s consider some details on how the wait_for() function works.
How Does Asyncio wait_for() Work
The asyncio.wait_for() function does not block.
Instead, it returns a coroutine.
For example:
|
1 2 3 |
... # wait for a task with a timeout wait_coro = asyncio.wait_for(task) |
If the provided task that is waited for is a coroutine, then the coroutine is not executed until the wait_for() coroutine is executed either directly with an await expression or by scheduling it as a task.
For example:
|
1 2 3 |
... # execute the wait_for await wait_coro |
The wait_for() coroutine can be wrapped in an asyncio.Task and scheduled independently.
This will provide a Task object that can be queried and canceled. Canceling the task that wraps the wait_for() coroutine will also cancel the task or coroutine that the wait_for() coroutine is waiting on.
For example:
|
1 2 3 4 5 6 |
... # wrap the wait_for in a task and schedule for execution wait_task = asyncio.create_task(wait_coro) ... # cancel the wait for the task and the waited for coroutine or task wait_task.cancel() |
Now that we know how the wait_for() function works, let’s look at some worked examples.
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Example of Asyncio wait_for() With No Timeout
We can explore how to wait for a coroutine without a timeout.
In this example we will define a coroutine that reports a message, blocks for a fraction of a second, then reports a message before it is done.
The main coroutine creates the task coroutine and then waits for it via the wait_for() function without a timeout.
This is equivalent to waiting for the coroutine directly.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
# SuperFastPython.com # example of waiting for a coroutine without a timeout from random import random import asyncio # coroutine to execute in a new task async def task_coro(arg): # generate a random value between 0 and 1 value = random() # report message print(f'>task got {value}') # block for a moment await asyncio.sleep(value) # report all done print('>task done') # main coroutine async def main(): # create a task task = task_coro(1) # execute and wait for the task without a timeout await asyncio.wait_for(task, timeout=None) # start the asyncio program asyncio.run(main()) |
Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.
The main() coroutine creates the task coroutine. It then calls wait_for() and passes the task coroutine and sets the timeout to None.
The main() coroutine is suspended and the task_coro() is executed. It reports a message, sleeps a moment, then reports its final message before terminating.
The main() coroutine resumes and terminates the program.
This highlights how we can call the wait_for() function without a timeout.
The output from the program will differ each time it is run given the use of random numbers.
|
1 2 |
>task got 0.8259873938851356 >task done |
Next, let’s look at how we can call wait_for() with a timeout.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Asyncio wait_for() With a Timeout
We can explore how to wait for a coroutine with a timeout that elapses before the task is completed.
In this example, we execute a coroutine as above, except the caller waits a fixed timeout of 0.2 seconds or 200 milliseconds.
Recall that one second is equal to 1,000 milliseconds.
The task coroutine is modified so that it sleeps for more than one second, ensuring that the timeout always expires before the task is complete.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
# SuperFastPython.com # example of waiting for a coroutine with a timeout from random import random import asyncio # coroutine to execute in a new task async def task_coro(arg): # generate a random value between 0 and 1 value = 1 + random() # report message print(f'>task got {value}') # block for a moment await asyncio.sleep(value) # report all done print('>task done') # main coroutine async def main(): # create a task task = task_coro(1) # execute and wait for the task without a timeout try: await asyncio.wait_for(task, timeout=0.2) except asyncio.TimeoutError: print('Gave up waiting, task canceled') # start the asyncio program asyncio.run(main()) |
Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.
The main() coroutine creates the task coroutine. It then calls wait_for() and passes the task coroutine and sets the timeout to 0.2 seconds.
The main() coroutine is suspended and the task_coro() is executed. It reports a message and sleeps for a moment.
The main() coroutine resumes after the timeout has elapsed. The wait_for() coroutine cancels the task_coro() coroutine and the main() coroutine is suspended.
The task_coro() runs again and responds to the request to be terminated. It raises a TimeoutError exception and terminates.
The main() coroutine resumes and handles the TimeoutError raised by the task_coro().
This highlights how we can call the wait_for() function with a timeout and to cancel a task if it does not completed within a timeout.
The output from the program will differ each time it is run given the use of random numbers.
|
1 2 |
>task got 0.685375224799321 Gave up waiting, task canceled |
Next, let’s look at what happens if we wait on a task that fails with an unhandled exception.
Example of Asyncio wait_for() and the Task Fails with an Exception
We can explore how to wait for a coroutine with a timeout and a task that fails with an exception.
If a waited-for task fails with an unhandled exception, the exception is propagated back to the caller.
In this example, we update the task coroutine so that it fails with an unhandled exception after sleeping. The caller waits for the task to complete and the exception is propagated back to the caller and handled.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# SuperFastPython.com # example of waiting for a coroutine that fails with exception from random import random import asyncio # coroutine to execute in a new task async def task_coro(arg): # generate a random value between 0 and 1 value = 1 * random() # report message print(f'>task got {value}') # block for a moment await asyncio.sleep(value) # fail with an exception raise Exception('Something bad happened') # report all done (never reached) print('>task done') # main coroutine async def main(): # create a task task = task_coro(1) # execute and wait for the task without a timeout try: await asyncio.wait_for(task, timeout=2.0) except asyncio.TimeoutError: print('Gave up waiting, task canceled') except Exception as e: print(f'Task failed with: {e}') # start the asyncio program asyncio.run(main()) |
Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.
The main() coroutine creates the task coroutine. It then calls wait_for() and passes the task coroutine and sets a timeout.
The main() coroutine is suspended and the task_coro() is executed. It reports a message, sleeps a moment, then raises an unhandled exception.
The main() coroutine resumes catches the unhandled exception and reports a message
This highlights how unhandled exceptions raised in the waited-for task or coroutine are propagated to the caller and may need to be handled.
The output from the program will differ each time it is run given the use of random numbers.
|
1 2 |
>task got 0.5142627201191862 Task failed with: Something bad happened |
Next, let’s look at what happens if we wait on a task that is canceled.
Example of Asyncio wait_for() and the Task is Canceled
We can explore how to wait for a task with a timeout and the waited-for task is canceled by another task.
If the waited-for task is canceled, the wait_for() coroutine stops waiting, and the asyncio.CancelledError exception is propagated to the caller.
In the example below we wrap and schedule our task coroutine in an asyncio.Task object so that it can be canceled.
We then pass the task to the wait_for() function and retrieve the coroutine, although do not await it yet.
We then create a second task for a coroutine that takes a task object, waits a moment, and cancels it. This task is scheduled.
Finally, the wait_for coroutine is awaited with a timeout.
The waited-for task is canceled and the asyncio.CancelledError exception is propagated back to the caller and handled.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
# SuperFastPython.com # example of waiting for a task that is canceled from random import random import asyncio # coroutine to execute in a new task async def task_coro(arg): # generate a random value between 0 and 1 value = 1 + random() # report message print(f'>task got {value}') # block for a moment await asyncio.sleep(value) # report all done print('>task done') # another coroutine that cancels a task async def task_cancel(other_task): # wait a moment await asyncio.sleep(0.3) # cancel the other task other_task.cancel() # main coroutine async def main(): # create a task task = asyncio.create_task(task_coro(1)) # create the wait for coroutine wait_coro = asyncio.wait_for(task, timeout=1) # create and run the cancel task asyncio.create_task(task_cancel(task)) # await the wait-for coroutine try: await wait_coro except asyncio.TimeoutError: print('Gave up waiting, task canceled') except asyncio.CancelledError: print('Task was canceled externally') print(task) # start the asyncio program asyncio.run(main()) |
Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.
The main() coroutine creates and schedules the task_coro() as a task.
The wait_for() is then called to wait on the task with a timeout of one second and the returned coroutine is assigned.
Next, the task_cancel() coroutine is wrapped in a task and scheduled for execution, and passed the task object for the task_coro() coroutine.
The main() coroutine is suspended and the task_coro() is executed.
The task_coro() task reports a message and then sleeps. The task_cancel() sleeps, then resumes and cancels the task_coro() task.
The task_coro() task resumes and raises an asyncio.CancelledError exception.
The main() coroutine resumes and the asyncio.CancelledError exception is re-raised and handled. The details of the waited-for task are reported and it shows that it is done and was canceled.
This highlights how a waited-for task can be canceled externally and that this may need to be handled.
The output from the program will differ each time it is run given the use of random numbers.
|
1 2 3 |
>task got 1.5848230431004935 Task was canceled externally <Task cancelled name='Task-2' coro=<task_coro() done, defined at ...>> |
Next, let’s look at what happens if the wait_for() task itself is canceled.
Example of Asyncio wait_for() that is Canceled
We can explore how to wait for a task with a timeout and the wait_for() task itself is canceled.
The wait_for() coroutine can be wrapped in an asyncio.Task object and scheduled for execution. This object can be canceled which will cancel the wait_for() and in turn cancel the task or coroutine that is being waited for.
In the example below we update the example above. The wait_for() coroutine is wrapped in a task and passed to the canceled task that cancels it after a moment.
The wait_for() is executed and is canceled, which in turn cancels the task being waited for.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
# SuperFastPython.com # example of waiting for a task and the waiting task is canceled from random import random import asyncio # coroutine to execute in a new task async def task_coro(arg): # generate a random value between 0 and 1 value = 1 + random() # report message print(f'>task got {value}') # block for a moment await asyncio.sleep(value) # report all done print('>task done') # another coroutine that cancels a task async def task_cancel(other_task): # wait a moment await asyncio.sleep(0.3) # cancel the other task other_task.cancel() # main coroutine async def main(): # create a task task = asyncio.create_task(task_coro(1)) # create the wait for coroutine wait_coro = asyncio.wait_for(task, timeout=1) # wrap the wait coroutine in a task and execute wait_task = asyncio.create_task(wait_coro) # create and run the cancel task asyncio.create_task(task_cancel(wait_task)) # await the wait-for task try: await wait_task except asyncio.TimeoutError: print('Gave up waiting, task canceled') except asyncio.CancelledError: print('Wait for task was canceled externally') print(task) print(wait_task) # start the asyncio program asyncio.run(main()) |
Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.
The main() coroutine creates and schedules the task_coro() as a task.
The wait_for() is then called to wait on the task with a timeout of one second. The wait_for() coroutine is wrapped in a task and scheduled for execution
Next, the task_cancel() coroutine is wrapped in a task and scheduled for execution, and passed the task object for the wait_for() coroutine.
The main() coroutine is suspended and the task_coro() is executed.
The task_coro() task reports a message and then sleeps. The task_cancel() sleeps, then resumes and cancels the wait_for() task. The exception is propagated down to the task_coro() task.
The task_coro() task resumes and raises an asyncio.CancelledError exception.
The main() coroutine resumes and the asyncio.CancelledError exception is re-raised and handled. The details of the task_coro() task and the wait_for() task are reported. We can see that both were canceled and are marked as done.
This highlights how the wait_for() task itself can be canceled which will propagate the cancellation down to the task that is being waited for.
The output from the program will differ each time it is run given the use of random numbers.
|
1 2 3 4 |
>task got 1.022061277758374 Wait for task was canceled externally <Task cancelled name='Task-2' coro=<task_coro() done, defined at ...>> <Task cancelled name='Task-3' coro=<wait_for() done, defined at ...>> |
Next, let’s look at a common error of forgetting to await the wait_for() coroutine.
Example of Asyncio wait_for() that is never Awaited
We can explore how to wait for a coroutine that is never awaited.
In the example below we fail to await the coroutine returned from wait_for() and instead wait around for the waited-for task to complete.
It never does because the wait_for() coroutine is not executed with an await expression.
This is a common error when using the wait_for() function.
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
# SuperFastPython.com # example of never awaiting the wait_for coroutine from random import random import asyncio # coroutine to execute in a new task async def task_coro(arg): # generate a random value between 0 and 1 value = random() # report message print(f'>task got {value}') # block for a moment await asyncio.sleep(value) # report all done print('>task done') # main coroutine async def main(): # create a task task = task_coro(1) # get the coroutine coro = asyncio.wait_for(task, timeout=None) # report the details print(coro) # wait a while await asyncio.sleep(2) # start the asyncio program asyncio.run(main()) |
Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.
The main() coroutine creates the task coroutine. It then calls wait_for() and passes the task coroutine and sets the timeout to None. A coroutine is returned and assigned and the details of the coroutine are reported.
The main() coroutine is suspended with a call to wait. It resumes and the program is terminated,
The wait_for() coroutine is never executed, e.g. never awaited or scheduled as a task. As such, two RuntimeWarnings are reported.
This highlights that the coroutine returned from wait_for() must be executed, such as with an await expression or being scheduled as a task.
The output from the program will differ each time it is run given the use of random numbers.
|
1 2 3 4 5 6 7 |
<coroutine object wait_for at 0x107d059a0> ...: RuntimeWarning: coroutine 'wait_for' was never awaited self._context.run(self._callback, *self._args) RuntimeWarning: Enable tracemalloc to get the object allocation traceback ...: RuntimeWarning: coroutine 'task_coro' was never awaited self._context.run(self._callback, *self._args) RuntimeWarning: Enable tracemalloc to get the object allocation traceback |
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know how to wait for an asyncio task with a timeout in Python.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Meik Schneider on Unsplash
Do you have any questions?