SuperFastPython

When Are Asyncio Tasks Canceled

December 17, 2023 Python Asyncio

Asyncio tasks can be canceled at any time.

Asyncio tasks can be canceled manually while they are scheduled or running.

Additionally, tasks can be automatically canceled by the asyncio infrastructure, such as after a timeout, after a deadline, when one task in the group fails, and, by the event loop when shutting down.

In this tutorial, you will discover when asyncio tasks can be canceled in Python.

Let's get started.

What is Asyncio Task Cancellation?

Asyncio tasks can be canceled.

This can be achieved by calling the cancel() method on the asyncio.Task. This will request that the task be canceled as soon as possible and return True if the request was successful or False if it was not, e.g. the task is already done.

Request the Task to be cancelled. This arranges for a CancelledError exception to be thrown into the wrapped coroutine on the next cycle of the event loop.

-- Coroutines and Tasks

For example:

...
# cancel a task
if task.cancel():
	# request successful

The next time that the target task that was canceled resumes, a CancelledError exception will be raised.

This exception will bubble up to the top level of the task and cause it to stop running.

You can learn more about the mechanics of cancellation a task in the tutorial:

Next, let's consider when a task can be canceled.

When Are Tasks Canceled?

A task can be canceled only if it is pending (scheduled) or running.

A done task cannot be canceled.

There are two main ways a task may be canceled:

  1. Manually
  2. Automatically
    1. Timeouts
    2. Failure in Group
    3. Event Loop Shutdown

Let's consider each of these cases.

Manually Canceling a Task

A task can be canceled manually by another task calling the cancel() method.

For example:

...
# cancel a task
_ = task.cancel()

This is reasonably straightforward.

Automatic task cancellation is more interesting.

Automatically Canceling a Task

A task may be canceled automatically by infrastructure in the asyncio module.

This includes:

  1. After a timeout.
  2. When another task in the group fails.
  3. When the event loop closes.

Let's take a closer look at each of these in turn.

Tasks Canceled After Timeout

A task may be canceled automatically by a timeout.

This can be achieved via the asyncio.wait_for() call which will cancel the target task after a fixed number of seconds by raising a CancelledError in the task and transforming it into a TimeoutError.

If a timeout occurs, it cancels the task and raises TimeoutError.

-- Coroutines and Tasks

For example:

...
# handle timeout
try:
	# cancel task after a timeout
	asyncio.wait_for(task, timeout=5)
except asyncio.TimeoutError:
	# handle timeout

You can learn more about the asyncio.wait_for() call in the tutorial:

A task may also be canceled by a timeout when using the asyncio.timeout() or asyncio.timeout_at() context managers.

Like asyncio.wait_for(), the timeout context managers will cancel the target task and raise an asyncio.TimeoutError if the timeout or deadline elapses. The difference is the context manager allows multiple tasks to be created and canceled.

the context manager will cancel the current task and handle the resulting asyncio.CancelledError internally, transforming it into a TimeoutError which can be caught and handled.

-- Coroutines and Tasks

For example:

...
# handle timeout
try:
	# run tasks with timeout
	asyncio with asyncio.timeout(5):
		await task1
		await task2
		await task3
except asyncio.TimeoutError:
	# handle timeout

You can learn more about the asyncio.timeout() context manager in the tutorial:

You can learn more about the asyncio.timeout_at() context manager in the tutorial:

Tasks Canceled By asyncio.TaskGroup

Tasks may also be canceled automatically if they are part of a group and one task in the group is canceled.

This can be achieved by creating tasks as part of an asyncio.TaskGroup. If one task in the group fails with an unhandled exception, all remaining tasks in the group will be canceled.

The first time any of the tasks belonging to the group fails with an exception other than asyncio.CancelledError, the remaining tasks in the group are cancelled.

-- Coroutines and Tasks

For example:

...
# handle exceptions
try:
    # create task group
    async with asyncio.TaskGroup() as group:
    	await group.create_task(task1())
    	await group.create_task(task2())
    	await group.create_task(task3())
except asyncio.CancelledError:
	# handle timeout

You can learn more about how to use asyncio.TaskGroup in the tutorial:

Tasks Canceled When Event Loop Terminates

Tasks will also be canceled when the asyncio event loop is shut down.

This may be when the main coroutine terminates normally, or it may be when the event loop is terminated via an unexpected exception.

Now that we know about task cancellation and when tasks are canceled, let's consider some best practices.

Example of Manually Canceling a Pending Task

We can explore an example of manually canceling a pending (scheduled, but not running) task.

In this example, we will define a task that runs for a long time. It does not catch the CancelledError but does have a try-except block so that we can report when the task is exiting. The program will schedule this task and immediately cancel it, then wait for it to be canceled.

The expectation is that the task will be canceled before it has had an opportunity to execute the body of the coroutine.

Firstly, we can define the work() coroutine task. It sleeps for 10 seconds and then reports a message that it completed its work. The finally block reports a separate message so that we can see the difference between a normal completion and an early completion.

# task that take a long time
async def work():
    try:
        # sleep a long time
        await asyncio.sleep(10)
        print('Task sleep completed normally')
    finally:
        print('Task is done')

Next, the main() coroutine creates and schedules the task, cancels it, and then awaits the task in order to see the expected CancelledError that will be raised by the task.

# main coroutine
async def main():
    # create and schedule the task
    task = asyncio.create_task(work())
    # cancel the task
    task.cancel()
    # await the task
    try:
        await task
    except asyncio.CancelledError:
        print('Main saw the task cancel')
    print('Main is done')

We can then start the event loop.

...
# start the event loop
asyncio.run(main())

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of canceling a pending task
import asyncio

# task that take a long time
async def work():
    try:
        # sleep a long time
        await asyncio.sleep(10)
        print('Task sleep completed normally')
    finally:
        print('Task is done')

# main coroutine
async def main():
    # create and schedule the task
    task = asyncio.create_task(work())
    # cancel the task
    task.cancel()
    # await the task
    try:
        await task
    except asyncio.CancelledError:
        print('Main saw the task cancel')
    print('Main is done')

# start the event loop
asyncio.run(main())

Running the example first starts the event loop.

The main() coroutine runs and creates and schedules the work() task.

Before the work() task has an opportunity to run, it is canceled.

The main() coroutine then awaits the task.

The CancelledError is handled and a message is reported indicating that the task was canceled.

Finally, the main coroutine is done and reports its final message.

No message was seen from the work() coroutine.

This highlights that a task can be manually canceled when it is pending (scheduled, not yet running) and that it will not enter the body of the coroutine.

Main saw the task cancel
Main is done

Next, let's look at an example of canceling a running task.

Example of Manually Canceling a Running Task

We can explore an example of manually canceling a running task.

In this case, we can update the above example to allow the new work() task to run a moment before it is canceled.

...
# create and schedule the task
task = asyncio.create_task(work())
# allow the task to start running
await asyncio.sleep(1)
# cancel the task
task.cancel()

This will allow the body of the work() task to begin execution. Once canceled, we should see the finally block of the task executed before it is done.

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of canceling a running task
import asyncio

# task that take a long time
async def work():
    try:
        # sleep a long time
        await asyncio.sleep(10)
        print('Task sleep completed normally')
    finally:
        print('Task is done')

# main coroutine
async def main():
    # create and schedule the task
    task = asyncio.create_task(work())
    # allow the task to start running
    await asyncio.sleep(1)
    # cancel the task
    task.cancel()
    # await the task
    try:
        await task
    except asyncio.CancelledError:
        print('Main saw the task cancel')
    print('Main is done')

# start the event loop
asyncio.run(main())

Running the example first starts the event loop.

The main() coroutine runs and creates and schedules the work() task.

The main() coroutine suspends for a moment, allowing the work() coroutine to begin running.

The main() coroutine resumes and cancels the work() task, then awaits the task.

The work() task is canceled. A CancelledError exception is raised and the finally block is executed, reporting a message from the task before it is terminated.

The main() coroutine handles the CancelledError exception and a message is reported indicating that the task was canceled.

Finally, the main coroutine is done and reports its final message.

This highlights that a task can be manually canceled while it is running and that we can use a try-finally block to perform cleanup within the task without handling the CancelledError exception.

Task is done
Main saw the task cancel
Main is done

Next, let's look at an example of canceling a done task.

Example of Manually Canceling a Done Task

We can explore an example of manually canceling a done task.

In this case, we can update the above example to cancel the work() task after it is done.

This can be achieved by first awaiting the work() task until it is done.

We can then request a cancellation of the task via the cancel() method on the task and check the return value to see if the request was successful.

...
# create and schedule the task
task = asyncio.create_task(work())
# wait for the task to complete
await task
# cancel the task
if task.cancel():
    print('Request to cancel was successful')
else:
    print('Request to cancel was not successful')

Because the task is done, we do not expect the request for cancellation to be successful.

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of canceling a done task
import asyncio

# task that take a long time
async def work():
    try:
        # sleep a long time
        await asyncio.sleep(10)
        print('Task sleep completed normally')
    finally:
        print('Task is done')

# main coroutine
async def main():
    # create and schedule the task
    task = asyncio.create_task(work())
    # wait for the task to complete
    await task
    # cancel the task
    if task.cancel():
        print('Request to cancel was successful')
    else:
        print('Request to cancel was not successful')
    print('Main is done')

# start the event loop
asyncio.run(main())

Running the example first starts the event loop.

The main() coroutine runs and creates and schedules the work() task.

The main() coroutine suspends and allows the work() task to be completed.

The work() task runs and completes normally, reporting its normal completion message, then executes the finally block reporting the task done message.

The main() coroutine resumes and attempts to cancel the work() task.

This request fails, as expected because the task is already done.

Finally, the main coroutine is done and reports its final message.

This highlights that a task that is done cannot be canceled.

Task sleep completed normally
Task is done
Request to cancel was not successful
Main is done

Next, let's look at an example of automatically canceling a task via a timeout with asyncio.wait_for().

Example of Automatically Canceling a Task with asyncio.wait_for()

We can explore an example of automatically canceling a running task using a timeout with asyncio.wait_for().

In this case, we can update the above example so that the work() task is created and is allowed to start running for a moment.

...
# create and schedule the task
task = asyncio.create_task(work())
# allow the task to start running
await asyncio.sleep(1)

The main() coroutine can then await the task with a timeout of 3 seconds using the asyncio.wait_for() call.

This timeout of 3 seconds will not be long enough for the work() task to complete, which will trigger a timeout.

The work() task will be canceled and the asyncio.wait_for() will transform the CancelledError exception into a TimeoutError exception.

The main() coroutine will handle the TimeoutError exception and report a message.

...
# handle timeout
try:
    # wait for the task with a timeout
    await asyncio.wait_for(task, timeout=3)
except asyncio.TimeoutError:
    print('Main saw the task timeout')

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of canceling a task via asyncio.wait_for()
import asyncio

# task that take a long time
async def work():
    try:
        # sleep a long time
        await asyncio.sleep(10)
        print('Task sleep completed normally')
    finally:
        print('Task is done')

# main coroutine
async def main():
    # create and schedule the task
    task = asyncio.create_task(work())
    # allow the task to start running
    await asyncio.sleep(1)
    # handle timeout
    try:
        # wait for the task with a timeout
        await asyncio.wait_for(task, timeout=3)
    except asyncio.TimeoutError:
        print('Main saw the task timeout')
    print('Main is done')

# start the event loop
asyncio.run(main())

Running the example first starts the event loop.

The main() coroutine runs and creates and schedules the work() task.

The main() coroutine suspends for a moment, allowing the work() coroutine to begin running.

The work() runs and suspends at the call to sleep.

The main() coroutine resumes and awaits the task with a timeout of 3 seconds.

The timeout expires and the asyncio.wait_for() cancels the work() task.

The work() task is canceled. A CancelledError exception is raised and the finally block within the task is executed, reporting a message from the task before it is terminated.

The CancelledError exception is transformed into a TimeoutError by asyncio.wait_for() and raised in the main() coroutine.

The main() coroutine handles the TimeoutError exception and a message is reported indicating that the task was timed out.

Finally, the main coroutine is done and reports its final message.

This highlights that a task can be automatically canceled after a timeout via asyncio.wait_for().

Task is done
Main saw the task timeout
Main is done

Next, let's look at an example of automatically canceling a task via a timeout with asyncio.timeout().

Example of Automatically Canceling a Task with asyncio.timeout()

We can explore an example of automatically canceling a running task using a timeout with asyncio.timeout().

In this case, we can update the above example to use the asyncio.timeout() context manager with a timeout of 3 seconds.

This will not be long enough for the work() task to complete.

Within the context manager, we will then await the work() task.

As with asyncio.wait_for(), a TimeoutError will be raised by the asyncio.timeout() context manager if the task is not completed within the timeout. This can be handled by the main() coroutine.

...
# handle timeout
try:
    # wait for the task with a timeout
    async with asyncio.timeout(3):
        await task
except asyncio.TimeoutError:
    print('Main saw the task timeout')

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of canceling a task via asyncio.timeout()
import asyncio

# task that take a long time
async def work():
    try:
        # sleep a long time
        await asyncio.sleep(10)
        print('Task sleep completed normally')
    finally:
        print('Task is done')

# main coroutine
async def main():
    # create and schedule the task
    task = asyncio.create_task(work())
    # allow the task to start running
    await asyncio.sleep(1)
    # handle timeout
    try:
        # wait for the task with a timeout
        async with asyncio.timeout(3):
            await task
    except asyncio.TimeoutError:
        print('Main saw the task timeout')
    print('Main is done')

# start the event loop
asyncio.run(main())

Running the example first starts the event loop.

The main() coroutine runs and creates and schedules the work() task.

The main() coroutine suspends for a moment, allowing the work() coroutine to begin running.

The work() runs and suspends at the call to sleep.

The main() coroutine resumes and opens the asyncio.timeout() context manager with a timeout of 3 seconds, then awaits the work() task.

The timeout expires and the asyncio.timeout() cancels the work() task.

The work() task is canceled. A CancelledError exception is raised and the finally block within the task is executed, reporting a message from the task before it is terminated.

The CancelledError exception is transformed into a TimeoutError by asyncio.timeout() and raised in the main() coroutine.

The main() coroutine handles the TimeoutError exception and a message is reported indicating that the task was timed out.

Finally, the main coroutine is done and reports its final message.

This highlights that a task can be automatically canceled after a timeout via asyncio.timeout() context manager.

Task is done
Main saw the task timeout
Main is done

Next, let's look at an example of automatically canceling a task via a deadline with asyncio.timeout_at().

Example of Automatically Canceling a Task with asyncio.timeout_at()

We can explore an example of automatically canceling a running task using a deadline with asyncio.timeout_at().

In this case, we can update the above example to use the asyncio.timeout_at() context manager with a deadline of 3 seconds in the future.

This will not be long enough for the work() task to complete.

Within the context manager, we will then await the work() task.

As with asyncio.timeout() context manager above, a TimeoutError will be raised by the asyncio.timeout_at() context manager if the task is not completed before the deadline. This can be handled by the main() coroutine.

...
# handle deadline
try:
    # calculate a deadline
    deadline = asyncio.get_running_loop().time() + 3
    # wait for the task with a deadline
    async with asyncio.timeout_at(deadline):
        await task
except asyncio.TimeoutError:
    print('Main saw the task timeout')

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of canceling a task via asyncio.timeout_at()
import asyncio

# task that take a long time
async def work():
    try:
        # sleep a long time
        await asyncio.sleep(10)
        print('Task sleep completed normally')
    finally:
        print('Task is done')

# main coroutine
async def main():
    # create and schedule the task
    task = asyncio.create_task(work())
    # allow the task to start running
    await asyncio.sleep(1)
    # handle deadline
    try:
        # calculate a deadline
        deadline = asyncio.get_running_loop().time() + 3
        # wait for the task with a deadline
        async with asyncio.timeout_at(deadline):
            await task
    except asyncio.TimeoutError:
        print('Main saw the task timeout')
    print('Main is done')

# start the event loop
asyncio.run(main())

Running the example first starts the event loop.

The main() coroutine runs and creates and schedules the work() task.

The main() coroutine suspends for a moment, allowing the work() coroutine to begin running.

The work() runs and suspends at the call to sleep.

The main() coroutine resumes and opens the asyncio.timeout_at() context deadline of 3 seconds in the future, then awaits the work() task.

The deadline passes and the asyncio.timeout_at() context manager cancels the work() task.

The work() task is canceled. A CancelledError exception is raised and the finally block within the task is executed, reporting a message from the task before it is terminated.

The CancelledError exception is transformed into a TimeoutError by asyncio.timeout_at() and raised in the main() coroutine.

The main() coroutine handles the TimeoutError exception and a message is reported indicating that the task was timed out.

Finally, the main coroutine is done and reports its final message.

This highlights that a task can be automatically canceled after a deadline via asyncio.timeout_at() context manager.

Task is done
Main saw the task timeout
Main is done

Next, let's look at an example of automatically canceling a task when one task fails via asyncio.TaskGroup.

Example of Automatically Canceling a Task with asyncio.TaskGroup

We can explore an example of automatically canceling a running task when one task fails via asyncio.TaskGroup.

In this case, we can define an additional new task that runs for a moment and then fails with an unhandled exception. The main() coroutine will create an asyncio.TaskGroup and then create and schedule the long-running task and the new task that will fail. When the new task fails, it will cause the long-running task to be automatically canceled.

Firstly, we can define a new task that runs for a moment and then fails with an unhandled exception.

The bad_work() coroutine below implements this.

# task that will fail
async def bad_work():
    # sleep a moment
    await asyncio.sleep(2)
    # fail for some reason
    raise Exception('Something Bad Happened!')

Next, in the main() coroutine, we can create an asyncio.TaskGroup using the context manager interface, then create and schedule the work() and bad_work() coroutines, then wait for them to complete.

This context manager must be wrapped in a try-except for the expected exception raised by the bad_work() coroutine.

...
# handle task failure
try:
    # create the task group
    async with asyncio.TaskGroup() as group:
        # create long running task
        task1 = group.create_task(work())
        # create task that fails
        task2 = group.create_task(bad_work())
except Exception as e:
    print(f'Main saw error: {e}')

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of canceling a task via asyncio.TaskGroup
import asyncio

# task that take a long time
async def work():
    try:
        # sleep a long time
        await asyncio.sleep(10)
        print('Task sleep completed normally')
    finally:
        print('Task is done')

# task that will fail
async def bad_work():
    # sleep a moment
    await asyncio.sleep(2)
    # fail for some reason
    raise Exception('Something Bad Happened!')

# main coroutine
async def main():
    # handle task failure
    try:
        # create the task group
        async with asyncio.TaskGroup() as group:
            # create long running task
            task1 = group.create_task(work())
            # create task that fails
            task2 = group.create_task(bad_work())
    except Exception as e:
        print(f'Main saw error: {e}')
    print('Main is done')

# start the event loop
asyncio.run(main())

Running the example first starts the event loop.

The main() coroutine runs and creates the asyncio.TaskGroup.

The long-running and failure coroutines are created and scheduled using the asyncio.TaskGroup, then main() coroutine suspends and awaits the tasks via the context manager's exit call.

The work() runs and suspends on the call to sleep.

The bad_work() runs, suspends on sleep, then resumes and fails with an unhandled exception.

The exception bubbles up to the TaskGroup. This causes the remaining work() task to be canceled.

The work() task is canceled. A CancelledError exception is raised and the finally block within the task is executed, reporting a message from the task before it is terminated.

The main() task resumes and handles the unhandled exception from bad_work() and reports a message.

Finally, the main coroutine is done and reports its final message.

This highlights that a task can be automatically canceled if it is part of a TaskGroup and another task in the group fails with an unhandled exception.

Task is done
Main saw error: unhandled errors in a TaskGroup (1 sub-exception)
Main is done

Next, let's look at an example of automatically canceling a task when the event loop is exited.

Example of Automatically Canceling a Task When The Event Loop Terminates

We can explore an example of automatically canceling a running task when the event loop terminates.

In this case, we can allow the work() coroutine to run for a moment before allowing the main() coroutine to resume and then terminate the event loop.

This will cancel all remaining tasks running in the event loop, such as the work() task.

...
# create and schedule the task
task = asyncio.create_task(work())
# allow the task to start running
await asyncio.sleep(1)
# report a message
print('Main is done')
# terminate the event loop...

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of canceling a running task when terminating the event loop
import asyncio

# task that take a long time
async def work():
    try:
        # sleep a long time
        await asyncio.sleep(10)
        print('Task sleep completed normally')
    finally:
        print('Task is done')

# main coroutine
async def main():
    # create and schedule the task
    task = asyncio.create_task(work())
    # allow the task to start running
    await asyncio.sleep(1)
    # report a message
    print('Main is done')
    # terminate the event loop...

# start the event loop
asyncio.run(main())

Running the example first starts the event loop.

The main() coroutine runs and creates and schedules the work() task.

The main() coroutine suspends for a moment, allowing the work() coroutine to begin running.

The work() runs and suspends at the call to sleep.

The main coroutine is done and reports its final message.

The work() task is canceled by the event loop. A CancelledError exception is raised and the finally block within the task is executed, reporting a message from the task before it is terminated.

This highlights that a task can be automatically canceled by the event loop when shutting down.

Main is done
Task is done

Takeaways

You now know when asyncio tasks can be canceled in Python.



If you enjoyed this tutorial, you will love my book: Python Asyncio Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.