How to Run Coroutine From Thread
You can execute a coroutine from another thread via the run_coroutine_threadsafe() function.
In this tutorial, you will discover how to execute coroutines from another thread in Python.
Let's get started.
Need to Run a Coroutine From Thread
We may need to execute a coroutine from outside of a coroutine program.
Recall that asyncio provides coroutine-based concurrency.
We may be executing tasks in other threads, separate from a coroutine event loop.
Tasks executed in other threads may need to perform a non-blocking IO or to interact with an asyncio program via a coroutine.
How can a thread execute a coroutine?
Run Coroutine From Another Thread
The run_coroutine_threadsafe() function allows a coroutine to be run in an asyncio program from another thread.
Submit a coroutine to the given event loop. Thread-safe.
-- Coroutines and Tasks
Here, "another thread", refers to a thread that is not currently executing the asyncio event loop.
This function is meant to be called from a different OS thread than the one where the event loop is running.
-- Coroutines and Tasks
This function takes a coroutine object and an event loop and returns a concurrent.futures.Future which is a handle on the task.
For example:
...
# execute a coroutine from another thread
future = asyncio.run_coroutine_threadsafe(coro, loop)Note, that the return value is not an asyncio.Future object, it is an instance of the Future object used in the executor framework, such as with the ThreadPoolExecutor and ProcessPoolExecutor classes.
The returned futures can be used to access the return value from the task, such as via the result() method. This method will block until the result is available.
For example:
...
# get the return value from the coroutine
value = future.result()You can learn more about Future objects in the tutorial:
The thread that is executing the asyncio program can get a reference to the current event loop via the asyncio.get_running_loop() or asyncio.get_event_loop() functions.
For example:
...
# get the event loop
loop = asyncio.get_event_loop()This loop object can then be shared with another thread that needs to execute coroutines.
Because the run_coroutine_threadsafe() requires a loop, it means that a coroutine or task first retrieved the loop and made it available to the new thread. This suggests that new threads that need to execute coroutines are likely created and managed by the asyncio program itself, e.g. to execute a background task that blocks.
Now that we know how to use the run_coroutine_threadsafe() function, let's look at some worked examples.
Example of Running a Coroutine From Another Thread
We can explore how to execute a coroutine from another thread.
In this example, we will run an asyncio program that reports some messages and runs a coroutine in the background. While the coroutine is running a new thread will run and will execute another coroutine in the asyncio program and wait for it to finish.
There are a few steps to this example, let's work through them.
Firstly, we can define the coroutine that will be executed by the new thread.
This coroutine will report a message, sleep for two seconds, then report a final message.
The task_coro2() coroutine function below defines this.
# another task coroutine
async def task_coro2():
# report a message
print(f'>>task2 running')
# block for a moment
await asyncio.sleep(2)
# report a message
print(f'>>task2 done')
Next, we can define a coroutine that is executed by the asyncio program, normally.
This coroutine is a long-running task. It loops five times, reports a message, and sleeps every iteration.
The task_coro() coroutine function below defines this.
# task coroutine
async def task_coro():
# loop a few times
for i in range(5):
# report a message
print(f'>task at {i}')
# block a moment
await asyncio.sleep(1)
Next, we can define a function that will execute in a new thread.
The function will take the asyncio event loop object as an argument. This is needed so that the task in the new thread can execute a coroutine in the asyncio program.
The thread function will report a message, sleep a moment, then execute the task_coro2() coroutine in the asyncio program.
This returns a concurrent.futures.Future object that the thread then blocks on via the result() method, within for the coroutine to complete.
The task_thread() function below implements this.
# function executed in another thread
def task_thread(loop):
# report a message
print('thread running')
# wait a moment
time.sleep(1)
# create a coroutine
coro = task_coro2()
# execute a coroutine
future = asyncio.run_coroutine_threadsafe(coro, loop)
# wait for the task to finish
future.result()
# report a message
print('thread done')
We can then define the main coroutine that is the entry point to the asyncio program.
This coroutine reports a message, then gets the event loop object and creates a new thread to execute our task_thread() thread function, passing the event loop object as an argument.
It then executes and awaits the task_coro() coroutine.
The main() coroutine function below implements this.
# entry point to the asyncio program
async def main():
# report a message
print('asyncio running')
# get the event loop
loop = asyncio.get_running_loop()
# start a new thread
threading.Thread(target=task_thread, args=(loop,)).start()
# execute a task
await task_coro()
# report a message
print('asyncio done')
Finally, the entry point of the program runs the main() coroutine.
...
# start the asyncio program
asyncio.run(main())The complete example is listed below.
# SuperFastPython.com
# example of executing a coroutine from another thread
import threading
import time
import asyncio
# another task coroutine
async def task_coro2():
# report a message
print(f'>>task2 running')
# block for a moment
await asyncio.sleep(2)
# report a message
print(f'>>task2 done')
# task coroutine
async def task_coro():
# loop a few times
for i in range(5):
# report a message
print(f'>task at {i}')
# block a moment
await asyncio.sleep(1)
# function executed in another thread
def task_thread(loop):
# report a message
print('thread running')
# wait a moment
time.sleep(1)
# create a coroutine
coro = task_coro2()
# execute a coroutine
future = asyncio.run_coroutine_threadsafe(coro, loop)
# wait for the task to finish
future.result()
# report a message
print('thread done')
# entry point to the asyncio program
async def main():
# report a message
print('asyncio running')
# get the event loop
loop = asyncio.get_running_loop()
# start a new thread
threading.Thread(target=task_thread, args=(loop,)).start()
# execute a task
await task_coro()
# report a message
print('asyncio done')
# start the asyncio program
asyncio.run(main())
Running the example first creates the main() coroutine and uses it to start the asyncio program.
The main() coroutine then reports a message. It then gets a reference to the current asyncio event loop.
A new thread is created and started, configured to execute the task_thread() function, and passed the event loop object as an argument.
This begins executing the task_thread() function in a new thread, reporting a message and blocking.
The main() coroutine then creates the task_coro() coroutine and awaits it. This suspends the main() coroutine and executes the task_coro() coroutine.
The task_coro() loops, reporting a message and sleeping each iteration.
The new thread resumes from its sleep and creates a task_coro2() coroutine and executes it in the asyncio event loop. It then awaits it to execute.
The second task coroutine is scheduled in the event loop and executes as soon as it is able, e.g. when the first task coroutine is suspended with a sleep. The second task reports a message, sleeps, and reports a second message.
The second task coroutine completes and the new thread resumes and terminates.
The first task coroutine eventually completes.
Finally, the main() coroutine resumes and terminates the program.
This highlights how an asyncio program can create and run a new separate thread and how a separate thread from the asyncio program can execute a coroutine and wait for it to finish.
asyncio running
thread running
>task at 0
>>task2 running
>task at 1
>task at 2
>>task2 done
>task at 3
thread done
>task at 4
asyncio done
Takeaways
You now know how to execute coroutines from another thread with the run_coroutine_threadsafe() function.