ThreadPoolExecutor vs. the Global Interpreter Lock (GIL)

Last Updated on September 12, 2022

Python uses a Global Interpreter Lock, or GIL, which makes the interpreter thread-safe at the cost of allowing only one thread to execute at a time, in most circumstances.

In this tutorial, you will discover how the Global Interpreter Lock impacts the ThreadPoolExecutor.

Let’s get started.

Table of Contents

What Is ThreadPoolExecutor?

The ThreadPoolExecutor class provides a thread pool in Python.

A thread is a thread of execution.

Each thread belongs to a process and can share memory (state and data) with other threads in the same process. In Python, like many modern programming languages, threads are created and managed by the underlying operating system, so-called system-threads or native threads.

You can create a thread pool by instantiating the class and specifying the number of threads via the max_workers argument; for example:

...

# create a thread pool

executor = ThreadPoolExecutor(max_workers=10)

You can then submit tasks to be executed by the thread pool using the map() and the submit() functions.

The map() function matches the built-in map() function and takes a function name and an iterable of items. The target function will then be called for each item in the iterable as a separate task in the process pool. An iterable of results will be returned if the target function returns a value.

The call to map() does not block, but each result yielded in the returned iterator will block until the associated task is completed.

For example:

...

# call a function on each item in a list and process results

for result in executor.map(task, items):

# process result...

You can also issue tasks to the pool via the submit() function that takes the target function name and any arguments and returns a Future object.

The Future object can be used to query the status of the task (e.g. done(), running(), or cancelled()), and can be used to get the result or exception raised by the task once completed. The calls to result() and exception() will block until the task associated with the Future is done.

For example:

...

# submit a task to the pool and get a future immediately

future = executor.submit(task, item)

# get the result once the task is done

result = future.result()

Once you are finished with the thread pool, it can be shut down by calling the shutdown() function in order to release all of the worker threads and their resources.

For example:

...

# shutdown the thread pool

executor.shutdown()

The process of creating and shutting down the thread pool can be simplified by using the context manager that will automatically call the shutdown() function.

For example:

...

# create a thread pool

with ThreadPoolExecutor(max_workers=10) as executor:

# call a function on each item in a list and process results

for result in executor.map(task, items):

# process result...

# ...

# shutdown is called automatically

You can learn more about the ThreadPoolExecutor class here:

concurrent.futures — Launching parallel tasks

Now that we are familiar with the ThreadPoolExecutor, let’s take a look at the Global Interpreter Lock.

Run loops using all CPUs, download your FREE book to learn how.

What Is the Global Interpreter Lock?

The internals of the Python interpreter are not thread safe.

This means that there can be race conditions between multiple threads within a single Python process, potentially resulting in unexpected behavior and corrupt data.

As such, the Python interpreter makes use of a Global Interpreter Lock, or GIL for short, to make instructions executed by the Python interpreter (called Python bytecodes) thread-safe.

The GIL is a programming pattern in the reference Python interpreter called CPython, although similar locks exist in other interpreted languages, such as Ruby. It is a lock in the sense that it uses a synchronization primitive called a mutual exclusion or mutex lock to ensure that only one thread of execution can execute instructions at a time within a Python process.

The effect of the GIL is that whenever a thread within a Python program wants to run, it must acquire the lock before executing. This is not a problem for most Python programs that have a single thread of execution, called the main thread.

It can become a problem in multi-threaded Python programs, such as programs that make use of the threading.Thread class or the concurrent.futures.ThreadPoolExecutor class.

The lock is explicitly released and re-acquired periodically by each Python thread, specifically after approximately every 100 bytecode instructions executed within the interpreter. This allows other threads within the Python process to run, if present.

The lock is also released in some circumstances, allowing other threads to run.

An important example is when a thread performs an I/O operation, such as reading or writing from an external resource like a file, socket, or device.

The lock is also explicitly released by some third-party Python libraries when performing computationally expensive operations in C-code, such as many array operations in NumPy.

The GIL is a simple and effective solution to thread safety in the Python interpreter, but it has the major downside that full multithreading is not supported by Python.

An alternative solution might be to explicitly make the interpreter thread-safe by protecting each critical section. This has been tried a number of times and typically results in worse performance of single-threaded Python programs by up to 30%.

Unfortunately, both experiments exhibited a sharp drop in single-thread performance (at least 30% slower), due to the amount of fine-grained locking necessary to compensate for the removal of the GIL.
— Python Global Interpreter Lock, Python Wiki.

Now that we are familiar with the GIL, let’s look at how the ThreadPoolExecutor is impacted.

Download Now: Free ThreadPoolExecutor PDF Cheat Sheet

ThreadPoolExecutor vs. the Global Interpreter Lock

The presence of the GIL in Python impacts the ThreadPoolExecutor.

The ThreadPoolExecutor maintains a fixed-sized pool of worker threads that supports concurrent tasks, but the presence of the GIL means that most tasks will not run in parallel.

You may recall that concurrency is a general term that suggests an order independence between tasks, e.g. they can be completed at any time or at the same time. Parallel might be considered a subset of concurrency and explicitly suggests that tasks are executed simultaneously.

The GIL means that worker threads cannot run in parallel, in most cases.

Specifically, in cases where the target task functions are CPU-bound tasks. These are tasks that are limited by the speed of the CPU in the system, such as working no data in memory or calculating something.

Nevertheless, worker threads can run in parallel in some special circumstances, one of which is when an IO task is being performed.

These are tasks that involve reading or writing from an external resource.

Examples include:

Reading or writing a file from the hard drive.
Reading or writing to standard output, input, or error (stdin, stdout, stderr).
Printing a document.
Downloading or uploading a file.
Querying a server.
Querying a database.
Taking a photo or recording a video.
And so much more.

When a Python thread executes a blocking IO task, it will release the GIL and allow another Python thread to execute.

This still means that only one Python thread can execute Python bytecodes at any one time. But it also means that we will achieve seemingly parallel execution of tasks if tasks perform blocking IO operations.

Luckily, many potentially blocking or long-running operations, such as I/O, image processing, and NumPy number crunching, happen outside the GIL. Therefore it is only in multithreaded programs that spend a lot of time inside the GIL, interpreting CPython bytecode, that the GIL becomes a bottleneck.
— Python Global Interpreter Lock, Python Wiki.

This suggests that the ThreadPoolExecutor should be limited to those tasks that release the GIL.

It also suggests that if tasks that do not release the GIL are executed by worker threads, such as CPU-bound tasks, we may expect worse performance because of the locking of the GIL required by each thread before executing and switching between tasks every 100 instructions.

Free Python ThreadPoolExecutor Course

Download your FREE ThreadPoolExecutor PDF cheat sheet and get BONUS access to my free 7-day crash course on the ThreadPoolExecutor API.

Discover how to use the ThreadPoolExecutor class including how to configure the number of workers and how to execute tasks asynchronously.

Learn more