Poetry and Python Versions

Keywords

Python, Python 3.13, GIL, free threading, Poetry, Benchmark

After a long pause of more than 4 years, I have finally decided to come back to my personal blog and start writing again.

For this first post in a long while, I wanted to do something simple: benchmark the performance of Poetry the python package manager when using different Python versions. The idea came to me after I saw that Python 3.13 was released with experimental free threading support where the global interpreter lock (GIL) is disabled.

As a daily user of Poetry either for work or for personal projects, I was curious to see whether its performance changes significantly based on the Python version used. To answer this question, I decided to take inspiration from the python-package-manager-shootout repository, which benchmarks different Python package managers using a fixed Python version.

Unfortunately, as of writing this blog post Poetry can’t installed be directly from PyPI using Pyhton 3.13 with the GIL disabled due to an issue with some of its dependencies, namely msgpack and cryptography. once that’s resolved, I will try to update this post or write another one.

The remainder of this post is structured as follows:

• we will briefly see how to install the new Python version with free threading enabled,
• then, we will describe the different operations as well as the setup that will used in the benchmarks
• finally, we will visually analyze the result of the benchmarks.

Disclaimer This experiment has several limitations that should be considered when interpreting the results. Firstly, we’re only testing with a limited set of Python versions (namely 3.11, 3.12, and 3.13). Additionaly, we’re using a fixed set of packages (Sentry’s requirements.txt file) to test Poetry’s performance. Both of these choices may not capture the full range of use cases and scenarios that Poetry may encounter in real-world use cases.

Installation

We will now see how to install Python 3.13 version with and without free threading using pyenv, a popular tool for managing multiple Python versions.

Let’s first start by listing all available python versions for installation:

pyenv install -l

The result should look something like:

Available versions:
  2.1.3
  2.2.3
  2.3.7
  ...
  ...
  3.13.0
  3.13.0t
  ...
  ...

If you don’t see python 3.13 listed, it probably means that you need to update pyenv to the latest version using:

pyenv update

To install Python 3.13 with free threading disabled (i.e. GIL enabled), you would use:

pyenv install 3.13.0

To install Python 3.13 with free threading enabled (e.g. GIL disabled), you would use:

pyenv install 3.13.0t

To test whether this worked, you would use:

python -VV

The output should contain “experimental free-threading build” for the latter and not for the former.

For more detailed instructions and explanations, please refer to this blog post from realpython.com

Methodology

To benchmark Poetry’s performance, I used the latest version of Poetry (version 1.8.4 as of writing this blog post) with a few different Python versions: 3.11, 3.12 and 3.13

All the files related to this blog post can be found in this repository. It is heavily inspired by the python-package-manager-shootout repository.

Similarly to that repository, we use a list of packages from a fixed version of Sentry’s requirements.txt file which was chosen arbitrarily as a non-trivial real-world example.

Unlike in python-package-manager-shootout, we use a newer version of Sentry’s requirements to avoid any issues during package installation with newer Python versions. Additionally, we use hyperfine, a command-line benchmarking tool, to handle the execution and timing of each operation.

I benchmarked the following operations:

• import: For this operation, we call poetry add --lock --no-cache to import all of the requirements in Sentry’s requirements.txt file into a newly initialized pyproject.toml file (We use Poetry’s --lock flag to prevent it from installing packages at this point).
• lock: For this operation, we call poetry lock both with and without the --no-cache. We delete the poetry.lock before every call to make sure the lock creation starts from scratch each time.
• install: For this operation, we call poetry install both with and without the --no-cache. We delete the created virtual environment before every call to make sure the installation starts from scratch each time.
• update: For this operation, we call poetry update both with and without the --no-cache. We delete the poetry.lock and restore it before every call to make sure the update starts from the same point each time.
• add package: For this operation, we call poetry add numpy --no-cache. We remove the numpy package after every call to make sure adding the package starts from the same point each time.

The --no-cache flag is used to tell poetry to ignore its own cache stored, by default on Linux, under ~/.cache/pypoetry/cache.

Results

I ran the benchmarks in CI to have a more or less consistent environment. The workflow runs the operations, collects and combines the results and then creates plots based on them.

The results and plots can be found as artifacts of the repository’s benchmark CI workflow. The plots used in this post come from this specific workflow, from the benchmark-plots artifact.

In Figure 1, we can see that poetry with python 3.13 performs the best on average for the import operation.

Figure 1: Benchmark result of poetry import operation

In Figure 2, we can see that poetry with python 3.13 performs the best on average for the lock operation both with and without caching.

Figure 2: Benchmark result of poetry lock operation

In Figure 3, we can see that poetry with python 3.11 performs the best on average for the install operation both with and without caching.

Figure 3: Benchmark result of poetry install operation

In Figure 4, we can see that poetry with python 3.13 performs the best on average for the update operation both with and without caching.

Figure 4: Benchmark result of poetry update operation

In Figure 5, we can see that poetry with python 3.11 and 3.13 perform similarily on average for the add package operation.

Figure 5: Benchmark result of poetry add package operation

As we can see from the results, python 3.13 without free threading does improve poetry’s performance on average in 3 out of the 5 operations (import, lock, update) both with and without caching. In the other 2 operations, the performance difference is minimal.

Conclusion

In conclusion, our little experiment has shown that Poetry with Python 3.13 performs better than Poetry with Python 3.11 and 3.12 on average in most cases. However the performance is not that significant, and it remains to be seen whether the experimental free threading feature would make a significant difference.

For now, I will keep on eye on Python 3.13 with free threading’s support for Poetry, and update my findings as more information becomes available. This experiment is just one of many possible tests of Poetry’s performance with different Python versions, and the results should be taken as a rough indication only as they may not necessarily reflect real-world use cases.