4 ThreadPoolExecutor Common Errors in Python

You may encounter one among a number of common errors when using the ThreadPoolExecutor in Python.

These errors are typically easy to identify and often involve a quick fix.

In this tutorial, you will discover the common errors when using thread pools in Python and how to fix each in turn.

Let's get started.

Common Errors When Using ThreadPoolExecutor

The ThreadPoolExecutor is a flexible and powerful thread pool for executing ad hoc tasks in an asynchronous manner.

When you are getting started with the ThreadPoolExecutor in Python, you may encounter one of many common errors.

These errors are typically made because of bugs introduced by copy-and-pasting code, or from a slight misunderstanding in how the ThreadPoolExecutor works.

We will take a closer look at some of the more common errors made when using the ThreadPoolExecutor; they are:

• Error 1: Using a Function Call in submit()
• Error 2: Using a Function Call in map()
• Error 3: Incorrect Function Signature for map()
• Error 4: Incorrect Function Signature for Future Callbacks

Do you have an error using the ThreadPoolExecutor?
Let me know in the comments so I can recommend a fix and add the case to this tutorial.

Error 1: Using a Function Call in submit()

A common error is to call your function when using the submit() function.

For example:

...
# submit the task
future = executor.submit(task())

A complete example with this error is listed below.

# SuperFastPython.com
# example of calling submit with a function call
from time import sleep
from random import random
from concurrent.futures import ThreadPoolExecutor

# custom task that will sleep for a variable amount of time
def task():
    # sleep for less than a second
    sleep(random())
    return 'all done'

# start the thread pool
with ThreadPoolExecutor() as executor:
    # submit the task
    future = executor.submit(task())
    # get the result
    result = future.result()
    print(result)

Running this example will fail with an error.

Traceback (most recent call last):
...
    result = self.fn(*self.args, **self.kwargs)
TypeError: 'str' object is not callable

You can fix the error by updating the call to submit() to take the name of your function and any arguments, instead of calling the function in the call to submit.

For example:

...
# submit the task
future = executor.submit(task)

Error 2: Using a Function Call in map()

A common error is to call your function when using the map() function.

For example:

...
# submit all tasks
for result in executor.map(task(), range(5)):
    print(result)

A complete example with this error is listed below.

# SuperFastPython.com
# example of calling map with a function call
from time import sleep
from random import random
from concurrent.futures import ThreadPoolExecutor

# custom task that will sleep for a variable amount of time
def task(value):
    # sleep for less than a second
    sleep(random())
    return value

# start the thread pool
with ThreadPoolExecutor() as executor:
    # submit all tasks
    for result in executor.map(task(), range(5)):
        print(result)

Running the example results in a TypeError:

Traceback (most recent call last):
...
TypeError: task() missing 1 required positional argument: 'value'

This error can be fixed by changing the call to map() to pass the name of the target task function instead of a call to the function.

...
# submit all tasks
for result in executor.map(task, range(5)):
    print(result)

Error 3: Incorrect Function Signature for map()

Another common error when using map() is to provide no second argument to function, e.g. the iterable.

For example:

...
# submit all tasks
for result in executor.map(task):
    print(result)

A complete example with this error is listed below.

# SuperFastPython.com
# example of calling map without an iterable
from time import sleep
from random import random
from concurrent.futures import ThreadPoolExecutor

# custom task that will sleep for a variable amount of time
def task(value):
    # sleep for less than a second
    sleep(random())
    return value

# start the thread pool
with ThreadPoolExecutor() as executor:
    # submit all tasks
    for result in executor.map(task):
        print(result)

Running the example does not issue any tasks to the thread pool as there was no iterable for the map() function to iterate over.

In this case, no output is displayed.

The fix involves providing an iterable in the call to map() along with your function name.

...
# submit all tasks
for result in executor.map(task, range(5)):
    print(result)

Error 4: Incorrect Function Signature for Future Callbacks

Another common error is to forget to include the Future in the signature for the callback function registered with a Future object.

For example:

...
# callback function to call when a task is completed
def custom_callback():
    print('Custom callback was called')

A complete example with this error is listed below.

# SuperFastPython.com
# example of the wrong signature on the callback function
from time import sleep
from concurrent.futures import ThreadPoolExecutor

# callback function to call when a task is completed
def custom_callback():
    print('Custom callback was called')

# mock task that will sleep for a moment
def work():
    sleep(1)
    return 'Task is done'

# create a thread pool
with ThreadPoolExecutor() as executor:
    # execute the task
    future = executor.submit(work)
    # add the custom callback
    future.add_done_callback(custom_callback)
    # get the result
    result = future.result()
    print(result)

Running this example will result in an error when the callback is called by the thread pool.

Task is done
exception calling callback for <Future at 0x10a05f190 state=finished returned str>
Traceback (most recent call last):
...
TypeError: custom_callback() takes 0 positional arguments but 1 was given

Fixing this error involves updating the signature of your callback function to include the future object.

...
# callback function to call when a task is completed
def custom_callback(future):
    print('Custom callback was called')

Takeaways

You now know about the common errors when using the ThreadPoolExecutor in Python.

If you enjoyed this tutorial, you will love my book: Python ThreadPoolExecutor Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.