Last Updated on December 11, 2023
You can have running background tasks in asyncio suddenly disappear. This is a known bug and can be avoided by ensuring that you keep a strong reference to all tasks that run in the background.
In this tutorial, you will discover the asyncio disappearing task bug and how to avoid it.
Let’s get started.
Asyncio Tasks Can Disappear
Tasks in asyncio can disappear.
Specifically, we can create a task to run in the background via asyncio.create_task(), it can start running, then at some point, it can be terminated by the Python garbage collector.
For example:
|
1 2 3 4 |
... # schedule a task asyncio.create_task(coro()) ... |
The reason is that the asyncio event loop only maintains a weak reference to scheduled tasks, e.g., you can see them in asyncio.all_tasks().
Important Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done.
— Coroutines and Tasks
Although a rare event, it is very real and does affect modern versions of Python e.g. 3.11 and beyond.
Now that we know about the asyncio disappearing task bug, let’s look at how we might avoid it.
Run loops using all CPUs, download your FREE book to learn how.
How to Avoid Disappearing Tasks in Asyncio
The asyncio API documentation recommends keeping a reference for all scheduled tasks.
Specifically, for background tasks that are created and ignored, so-called “fire and forget” tasks.
For reliable “fire-and-forget” background tasks, gather them in a collection …
— Coroutines and Tasks
This can be achieved by assigning the result of asyncio.create_task() to a variable within the calling task, even if you do nothing with it.
For example:
|
1 2 3 4 |
... # schedule a task task = asyncio.create_task(coro()) ... |
The preference in the asyncio API documentation is to create a global variable in your program that maintains a collection of all tasks.
A done callback is then added to each task to automatically remove each task from the collection of tasks once it is finished.
The snippet from the API documentation is as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
background_tasks = set() for i in range(10): task = asyncio.create_task(some_coro(param=i)) # Add task to the set. This creates a strong reference. background_tasks.add(task) # To prevent keeping references to finished tasks forever, # make each task remove its own reference from the set after # completion: task.add_done_callback(background_tasks.discard) |
Many developers turn this into a helper function that can be called to create tasks in the program and automatically perform these operations.
For example, a helper function that does this looks as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
... # keep track of all tasks (avoid garbage collection) ALL_TASKS = set() # helper function for creating tasks that are not gc'ed def create_task_helper(coroutine): # wrap and schedule the coroutine task = asyncio.create_task(coroutine) # store in collection of all tasks global ALL_TASKS ALL_TASKS.add(task) # add a callback to remove from the collection task.add_done_callback(ALL_TASKS.discard) # return the task that was created return task |
This helper function could then be used to create background fire-and-forget tasks that cannot disappear as follows:
|
1 2 3 |
... # create a task create_task_helper(work()) |
Now that we know how to avoid the disappearing task bug, let’s look at a modern solution.
Modern Way to Avoid Disappearing Tasks (v3.11)
The modern approach to avoiding disappearing tasks is to use an asyncio.TaskGroup.
This is a context manager that can be used to create one or more tasks, keep track of them, and exit only once all the tasks are done.
Background tasks can be issued safely, and will only be awaited for when exiting the context manager.
For example:
|
1 2 3 4 5 6 |
... # create a taskgroup async with asyncio.TaskGroup() as group: # create a background task group.create_task(coro()) ... |
This feature was added to Python in version 3.11.
Added the TaskGroup class, an asynchronous context manager holding a group of tasks that will wait for all of them upon exit. For new code this is recommended over using create_task() and gather() directly.
— WHAT’S NEW IN PYTHON 3.11
You can learn more about how to use the asyncio.TaskGroup in the tutorial:
Now that we know how to avoid the asyncio disappearing task bug, let’s take a look at how we might trigger it.
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Example That Does NOT Trigger Disappearing Tasks
The asyncio disappearing task bug is relatively hard to trigger.
For example, we might naively assume that if we issue one of the background asyncio tasks and force the garbage collector to run a lot we will trigger the bug.
This is what I assumed at first.
Sadly, this is not the case.
We can demonstrate this with a worked example.
Firstly, we can define a task that will run in the background. It needs to let the user know it has started, that it is unique, and when it is done.
The background() coroutine below implements this, taking a unique id and reporting when it is started and when it is done.
|
1 2 3 4 5 6 |
# background task async def background(i): print(f'Task {i} starting') # wait around a while await asyncio.sleep(5) print(f'Task {i} done') |
We then need to start many of these tasks in the background and not keep a reference.
We can then loop many times and force full garbage collection each iteration. We will ensure that this loop runs longer than the background tasks so that we can see if they are done or not.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
# entry point async def main(): # kick of a ton of tasks, with no local reference for i in range(20): asyncio.create_task(background(i)) # allow tasks to start await asyncio.sleep(0) # loop and force lots of garbage collection for i in range(12): # run garbage collector gc.collect() # sleep a moment await asyncio.sleep(0.5) # report a message print(f'Main at iteration {i}') # report a final message print('Main done.') |
And that’s it.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# SuperFastPython.com # example of attempting to trigger the disappearing task bug (fails) import asyncio import gc # background task async def background(i): print(f'Task {i} starting') # wait around a while await asyncio.sleep(5) print(f'Task {i} done') # entry point async def main(): # kick of a ton of tasks, with no local reference for i in range(20): asyncio.create_task(background(i)) # allow tasks to start await asyncio.sleep(0) # loop and force lots of garbage collection for i in range(12): # run garbage collector gc.collect() # sleep a moment await asyncio.sleep(0.5) # report a message print(f'Main at iteration {i}') # report a final message print('Main done.') # start event loop asyncio.run(main()) |
Running the example does not trigger the disappearing task bug, as I might expect.
Instead, the tasks are started and completed normally.
Hammering the garbage collector 12 times while the background tasks are running did not force them to be terminated.
There’s more to it.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
Task 0 starting Task 1 starting Task 2 starting Task 3 starting Task 4 starting Task 5 starting Task 6 starting Task 7 starting Task 8 starting Task 9 starting Task 10 starting Task 11 starting Task 12 starting Task 13 starting Task 14 starting Task 15 starting Task 16 starting Task 17 starting Task 18 starting Task 19 starting Main at iteration 0 Main at iteration 1 Main at iteration 2 Main at iteration 3 Main at iteration 4 Main at iteration 5 Main at iteration 6 Main at iteration 7 Main at iteration 8 Task 0 done Task 1 done Task 2 done Task 3 done Task 4 done Task 5 done Task 6 done Task 7 done Task 8 done Task 9 done Task 10 done Task 11 done Task 12 done Task 13 done Task 14 done Task 15 done Task 16 done Task 17 done Task 18 done Task 19 done Main at iteration 9 Main at iteration 10 Main at iteration 11 Main done. |
Next, let’s look at how we might really trigger the disappearing task bug.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
How to Trigger Disappearing Tasks?
Forcing the disappearing task bug is more challenging.
People have tried, and some have succeeded, hence why it is mentioned prominently in the asyncio API documentation.
In fact, there is a closed GitHub issue on the need to mention the bug in the documentation titled “asyncio.create_task() documentation should mention user needs to keep reference to the task“.
The describes the bug and a symptom, which is the asyncio event loop reporting the message:
- Task was destroyed but it is pending!
For example:
asyncio will only keep weak references to alive tasks (in _all_tasks). If a user does not keep a reference to a task and the task is not currently executing or sleeping, the user may get “Task was destroyed but it is pending!”.
— asyncio.create_task() documentation should mention user needs to keep reference to the task
As such, if this warning is emitted by our event loop, perhaps on exit, then we might suspect that we have suffered this bug.
In fact, we can dig further, and find the original bug report titled “asyncio doesn’t warn if a task is destroyed during its execution” circa 2014 in which the warning message was added to the code base.
The issue links to a StackOverflow answer to the question “Python asyncio.create_task() – really need to keep a reference?“.
The answer is written by User “S.B” aka “Amir Soroush” on August 02, 2023, and is extensive. Hats off to you sir!
Critically, it provides an example to reproduce the fault and an explanation of why the fault is caused.
As I realized more and more people(including me) are struggling to understand “why” they need to keep references to the tasks as their code already works just fine, I intended to explain what’s going on behind the scene and give more information about the references in different steps and show when their code works, when it doesn’t.
— S.B’s Answer: Python asyncio.create_task() – really need to keep a reference?
The crux appears to be to have a task that is suspended but does not schedule another task on the event loop, at least not yet.
One approach to force this case is to request that the event loop create a new asyncio.Future object, as though we plan to use it, but then await it.
The task will then be suspended, it will be awaiting an asyncio.Future that has no reference outside of the task, and the garbage collector may decide to reclaim it, forcing it to terminate.
For example:
|
1 2 3 4 5 6 7 |
... # get the event loop loop = asyncio.get_running_loop() # create an empty future future = loop.create_future() # await the empty future (do nothing) await future |
The disappearing task cannot just await a call to asyncio.sleep(), as this will create a strong reference to the task.
Yielding from asyncio.sleep() prevents the consumers from being collected: it creates a strong ref to the future in the loop. I suspect also all network-related asyncio coroutines behave this way.
— asyncio doesn’t warn if a task is destroyed during its execution
Additionally, the bug cannot be forced by having the task do a blocking operation, as this will prevent the event loop from progressing, e.g. the task won’t be suspended.
The author then goes on to provide a real-world example with an echo server and client, where the server executes a similar pattern of code internally and triggers the fault.
Recall, a major symptom of the fault is the warning message: “Task was destroyed but it is pending!“
Now that we know how to trigger the disappearing task bug on demand, let’s look at a worked example.
Example of Forcing Disappearing Tasks in Asyncio
We can explore an example of forcing the disappearing task bug.
All credit to “S.B” aka Amir Soroush. This is a rewritten version of his answer posed on StackOverflow.
In this example we will have main() coroutine that will create a background task that sleeps, runs the garbage collector, and prints a message. A reference to this task is kept. The main() coroutine then creates another background task that creates a future and awaits it. A reference to this task is not kept. The main() coroutine then waits a moment and terminates.
This will trigger the disappearing task bug, specifically for the background task for which no reference is kept that awaits a future to which there are no references outside of the task.
Firstly, we can define our normal worker task.
|
1 2 3 4 5 6 7 8 |
# a task that just sleeps and prints async def work(): # block await asyncio.sleep(1) # run the garbage collector gc.collect() # report a final message print(f'Task is done') |
Note the call to the garbage collector. This will cause the disappearing task defined below to be garbage collected.
Next, we can define our disappearing task.
|
1 2 3 4 5 6 7 8 9 10 |
# a task that will disappear async def disappear(): # get the event loop loop = asyncio.get_running_loop() # create an empty future future = loop.create_future() # await the empty future (do nothing) await future # report a final message print(f'Disappear is done') |
Note, the final print message is never reached, regardless of the bug. The awaiting of the future never returns.
Finally, we can define the main() coroutine that sets things up.
|
1 2 3 4 5 6 7 8 9 10 |
# entry point async def main(): # create and schedule work task task = asyncio.create_task(work()) # create and schedule disappear task asyncio.create_task(disappear()) # all the tasks to run await asyncio.sleep(3) # report a final message print('Main is done') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
# SuperFastPython.com # example of trigger a disappearing task import asyncio import gc # a task that just sleeps and prints async def work(): # block await asyncio.sleep(1) # run the garbage collector gc.collect() # report a final message print(f'Task is done') # a task that will disappear async def disappear(): # get the event loop loop = asyncio.get_running_loop() # create an empty future future = loop.create_future() # await the empty future (do nothing) await future # report a final message print(f'Disappear is done') # entry point async def main(): # create and schedule work task task = asyncio.create_task(work()) # create and schedule disappear task asyncio.create_task(disappear()) # all the tasks to run await asyncio.sleep(3) # report a final message print('Main is done') # start event loop asyncio.run(main()) |
Running the example first starts the event loop and runs the main() coroutine.
The main() coroutine runs and creates the work() and keeps a reference, then creates the disappear() and does not keep a reference. It then suspends for 3 seconds.
The work() task runs and sleeps for one second.
The disappear() task runs and creates a future and awaits it. This task is now suspended.
Two preconditions of the bug are now met:
- There are no strong references to disappear()
- disappear() is suspended
The work() task runs and executes the garbage collector.
This terminates the disappear() and reports the warning message “Task was destroyed but it is pending!” and the details of the asyncio.Task object.
The work() task reports its message and terminates normally.
The main() task resumes, reports its final message, and terminates normally.
This highlights how we can force the disappearing task bug in asyncio.
|
1 2 3 4 |
Task was destroyed but it is pending! task: <Task pending name='Task-3' coro=<disappear() done, defined at ...> wait_for=<Future pending cb=[Task.task_wakeup()]>> Task is done Main is done |
Next, let’s look at how this bug can be avoided with one change, keeping a strong reference to the background task.
Example of Avoiding Disappearing Tasks By Keeping a Reference
The disappearing task bug forced in the previous section can be avoided by keeping a strong reference to the asyncio.Task object created for the disappear() task.
For example:
|
1 2 3 |
... # create and schedule disappear task task2 = asyncio.create_task(disappear()) |
And that’s it.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
# SuperFastPython.com # example of fixed disappearing task by keeping reference import asyncio import gc # a task that just sleeps and prints async def work(): # block await asyncio.sleep(1) # run the garbage collector gc.collect() # report a final message print(f'Task is done') # a task that will disappear async def disappear(): # get the event loop loop = asyncio.get_running_loop() # create an empty future future = loop.create_future() # await the empty future (do nothing) await future # report a final message print(f'Disappear is done') # entry point async def main(): # create and schedule work task task = asyncio.create_task(work()) # create and schedule disappear task task2 = asyncio.create_task(disappear()) # all the tasks to run await asyncio.sleep(3) # report a final message print('Main is done') # start event loop asyncio.run(main()) |
Running the example first starts the event loop and runs the main() coroutine.
The main() coroutine runs and creates the work() and keeps a reference, then creates the disappear(), and this time it keeps a reference to it. It then suspends for 3 seconds.
The work() task runs and sleeps for one second.
The disappear() task runs and creates a future and awaits it. This task is now suspended, and will never resume.
The work() task runs and executes the garbage collector.
This has no effect on disappear(), as the main() task has a strong reference to it.
The work() task reports its message and terminates normally.
The main() task resumes, reports its final message and terminates normally.
The disappear() task does not resume, it awaits forever because the asyncio.Future does nothing and never returns.
This highlights how we can avoid the asyncio disappearing task bug by explicitly keeping a strong reference to background tasks.
|
1 2 |
Task is done Main is done |
Frequently Ask Questions
This section lists some common questions about the disappearing task bug in asyncio.
Is the Disappearing Task Bug Still a Thing?
Yes.
As of Python 3.11 in 2023. Probably in Python 3.12 and probably for the foreseeable future.
How Do I Know I Suffer From The Disappearing Task Bug?
You will see a warning message with the text:
- Task was destroyed but it is pending!
The warning message may be reported when the disappeared task is deleted, or it may be when the asyncio event loop is terminated.
How Do I Avoid The Disappearing Task Bug?
Keep a strong reference to background tasks.
For example:
- Assign the task to a variable and keep the variable around, or:
- Add background tasks to a set and remove them from the set once they are done, or:
- Create tasks using asyncio.TaskGroup.
See the examples above.
What Are The Preconditions For The Disappearing Task Bug?
- You schedule a background task via asyncio.create_task() without keeping a reference.
- The background task is suspended at some point.
- The thing the background task awaits is not known outside the task.
- Garbage collection runs while the background task is suspended.
At least, that’s my understanding of the bug, please correct me if you believe otherwise.
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know about the asyncio disappearing task bug and how to avoid it.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any additional tips?
I’d love to hear about them!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Varla Scooter on Unsplash
Do you have any questions?