Why Do Python Developers Hate The GIL?

Python does not support parallelism via threads because of the Global Interpreter Lock or GIL.

As such, many Python developers HATE the GIL. This refrain is then repeated and repeated by newer developers, leaving behind the nuance of the limitations of the GIL and leading to statements like “Python does not support concurrency“, which is dead wrong.

In this tutorial, you will discover why many Python developers hate the Global Interpreter Lock.

Let’s get started.

Table of Contents

Python Developers Hate the GIL

When it comes to Python concurrency, the number one complaint by Python developers is about the “Global Interpreter Lock” or the GIL.

I want to share some examples I found (without trying too hard) around the web.

The GIL sucks ass. The lack of good concurrency sucks ass
— Richard Kenneth Eng, Quora.

The Global Interpreter Lock (GIL) hampers programmers from writing multi-threaded code. To some extent it can be done using coroutines, asynchronous patterns, or emulations of threading based on separate processes, but these workarounds are all less efficient and more complex than simply using threads.
— Michael Gogins, Quora.

Python threading sucks
— Andrew Gaffney, Blog.

Python sucks so hard!! Slow Interpreter, Broken Multithreading. It really sucks. Overrated language.
JaaavaScript, devRant.

Parallelism is very bad on CPython and PyPy; threads are subject to the Global Interpreter Lock (GIL), and one must use multiprocessing (which comes with its own bag of oddities) to get real parallelism.
— darkf, blog [internet archive]

Almost all modern languages support true multi-threading, but python cannot
— yywww, blog comment.

You forgot the biggest elephant in the room: parallel execution. From problems with their language: “threads” (which dont’ execute in parallel), to the problems with the workarounds, Dask, and multiprocessing and pickle, it’s practically impossible to do parallel execution in python.
— oafwiejfaoiwj, blog comment.

Also multithreading still second-class citizen. Oh well.
— Thomas, blog comment.

[…] I’d like to be able to parallelize the different sections, but multiprocessing is a mess (only pickable data can be used, no shared state), threading doesn’t give me any benefit due to the GIL, and PyPy hasn’t worked the last three times I tried. Also, improvements can stack. If I get threading working, I can still move more things to the C extension to improve performance, or migrate finally to PyPy (if they manage to remove the GIL).
— gjulianm, hacker news comment.

I’ve been bitten by the GIL multiple times since switching to a company writing mostly python. Not that I write that much multithreaded code, but the GIL influences everything so lots of patterns in other languages no longer work.
— matsemann, hacker news comment.

So what is the GIL exactly?

Run loops using all CPUs, download your FREE book to learn how.

What is the Global Interpreter Lock (GIL)

The internals of the Python interpreter are not thread-safe.

This means that there can be race conditions between multiple threads within a single Python process, potentially resulting in unexpected behavior and corrupt data.

As such, the Python interpreter makes use of a Global Interpreter Lock, or GIL for short, to make instructions executed by the Python interpreter (called Python bytecodes) thread-safe.

The GIL is a programming pattern in the reference Python interpreter called CPython, although similar locks exist in other interpreted languages, such as Ruby. It is a lock in the sense that it uses a synchronization primitive called a mutual exclusion or mutex lock to ensure that only one thread of execution can execute instructions at a time within a Python process.

In CPython, the global interpreter lock, or GIL, is a mutex that protects access to Python objects, preventing multiple threads from executing Python bytecodes at once. The GIL prevents race conditions and ensures thread safety.
— Global Interpreter Lock, Python Wiki.

The effect of the GIL is that whenever a thread within a Python program wants to run, it must acquire the lock before executing. This is not a problem for most Python programs that have a single thread of execution, called the main thread.

It can become a problem in multi-threaded Python programs, such as programs that make use of the threading.Thread class, the multiprocessing.pool.ThreadPool class, or the concurrent.futures.ThreadPoolExecutor class.

The lock is explicitly released and re-acquired periodically by each Python thread, specifically after approximately every 100 bytecode instructions executed within the interpreter. This allows other threads within the Python process to run, if present.

The lock is also released in some circumstances, allowing other threads to run.

An important example is when a thread performs an I/O operation, such as reading or writing from an external resource like a file, socket, or device.

Luckily, many potentially blocking or long-running operations, such as I/O, image processing, and NumPy number crunching, happen outside the GIL. Therefore it is only in multithreaded programs that spend a lot of time inside the GIL, interpreting CPython bytecode, that the GIL becomes a bottleneck.
— Global Interpreter Lock, Python Wiki.

The lock is also explicitly released by some third-party Python libraries when performing computationally expensive operations in C-code, such as many array operations in NumPy.

In CPython, due to the Global Interpreter Lock, only one thread can execute Python code at once (even though certain performance-oriented libraries might overcome this limitation).
— threading — Thread-based parallelism

The GIL is a simple and effective solution to thread safety in the Python interpreter, but it has the major downside that full multithreading is not supported by Python.

An alternative solution might be to explicitly make the interpreter thread-safe by protecting each critical section. This has been tried a number of times and typically results in worse performance of single-threaded Python programs by up to 30%.

Unfortunately, both experiments exhibited a sharp drop in single-thread performance (at least 30% slower), due to the amount of fine-grained locking necessary to compensate for the removal of the GIL.
— Python Global Interpreter Lock, Python Wiki.

Now that we know what the Python GIL is, let’s take a closer look at why Python developers hate it.

Download Now: Free Threading PDF Cheat Sheet

Why Developers Hate the GIL

Many Python developers hate the GIL, or at least, complain a lot about it.

I’ll use “hate” here, but it may be more of a “frustration” or “general dislike“.

We know that the GIL ensures thread safety of the Python interpreter, specifically the CPython reference interpreter shared for free on Python.org.

The crux of the issue is that the capability of threads in Python is crippled or limited.

Threads are objects created and managed by the underlying operating system. They can be used in other programming languages, like C++, C#, and Java without any imposed limitations, but not so in Python.

Specifically, in other programming languages without a GIL, threads may be used within a process in order to achieve full parallelism. That is, for CPU-bound or IO-bound tasks.

This is not so in Python, with the caveats mentioned above (e.g. some C-based extensions will release the lock).

The hate comes from knowing a capability is possible but unavailable in the language.

The hate comes from being forced to use a “workaround” to achieve the desired effect of parallel execution.

There is nuance related to the GIL, as we have seen, but many newer developers may just repeat the “GIL hate” sentiment without digging deeper, leading to refrains like:

Python threads are broken.
Python threads are buggy.
Python does not support concurrency

A quick google search finds people making these exact comments, for example:

Tricky bits? Thanks to GIL, it’s just broken and doesn’t work.
— tasubotadas, hacker news comment.

And still worse:

If you need concurrency, use a different language.

At this point, I think we are off the rails.

If you really care about speed and would like to parallelize CPU-bound code, you’ll probably do better using another language like C/C++ (or Java, Go, D etc.) for this particular task
— sleepysysadmin, hacker news comment.

Well many developers end up using different languages for this reason.
— jillesvangurp, hacker news comment.

… If you have somehow arranged to bang your head on that one specific thing that Python doesn’t do, and you cannot figure out how to not bang your head on the GIL, then for goodness’ sake use Go or Erlang or something that DOES do the one thing you can’t live without.
— carapace, hacker news comment.

Switching programming language mid-project is probably not an option for any serious project.

Starting with a requirement for full parallelism via threads may allow an alternate language to be chosen at the beginning project. However, that’s an awfully specific requirement.

I caution newer Python developers to not throw the Python baby out with the bathwater.

Python threads are limited because of the GIL, but we work with the tool not against it. All languages have limitations and make trade-offs.

Other extremely popular languages have made the same trade-off, like Ruby.

True parallel is available in Python, directly via multiprocessing, and less directly via c-extensions and alternate interpreters.

Free Python Threading Course

Download your FREE threading PDF cheat sheet and get BONUS access to my free 7-day crash course on the threading API.

Discover how to use the Python threading module including how to create and start new threads and how to use a mutex locks and semaphores

Learn more