A few notes on Rye

Rye is Armin Ronacher's new experimental Python packaging tool. I decided to take it for a test-run.

Installing Rye

Rye is built in Rust, and currently needs to be installed using Cargo. I had Cargo from previous dalliances with Rust, so I could install Rye using:

cargo install --git https://github.com/mitsuhiko/rye rye

This resulted in a 6.4MB binary file in:


I added the following line to my ~/.zshrc file to make it available on my path:

export PATH=$PATH:$HOME/.cargo/bin

Installing Python with Rye

The Rye docs suggested running this:

rye pin cpython@3.11

As far as I can tell all this did was write 3.11.1 to a ~/.python-version file. I'm not sure if there are any tools other than Rye which pay attention to this file. (UPDATE: pyenv uses this file too)

Actually fetching versions of Python can be done using rye toolchain fetch VERSION, for example:

% rye toolchain fetch 3.9
Downloading cpython@3.9.16
success: Downloaded cpython@3.9.16

This placed a whole bunch of files in ~/.rye/py/cpython@3.9.16/ - find . | wc -l reported 4,085 in that directory.

The one that matters most is:


Which gives me a 3.9.16 Python interpreter.

This is the feature of Rye I'm most excited about: the Python bundles it installs come from Gregory Szorc's indygreg/python-build-standalone project.

This should mean that they completely ignore the many other weird ways that Python can end up installed on a system. Admittedly, this is a new weird way to install Python - but at least it shouldn't clash with anything else.

Normally though you wouldn't use rye toolchain fetch at all. The Rye suggested workflow is to run rye init in a new directory to create a pyproject.toml file:

% cd /tmp
/tmp % mkdir my-project   
/tmp % cd my-project 
my-project % rye init
success: Initialized project in /private/tmp/my-project/.
my-project % ls
README.md	pyproject.toml
my-project % cat pyproject.toml 
name = "my-project"
version = "0.1.0"
description = "Add a short description here"
authors = [
    { name = "Simon Willison", email = "...@gmail.com" }
dependencies = []
readme = "README.md"
requires-python = ">= 3.8"
license = { text = "MIT" }

requires = ["hatchling"]
build-backend = "hatchling.build"

managed = true
my-project % cat README.md 
# my-project

Describe your project here.

* License: MIT

To add dependencies to that project, use rye add:

rye add httpx

This added the following line to pyproject.toml:

dependencies = ["httpx~=0.24.0"]

Then to actually install the dependencies, run rye sync:

my-project % rye sync
Initializing new virtualenv in /private/tmp/my-project/.venv
Python version: cpython@3.11.1
Generating production lockfile: /private/tmp/my-project/requirements.lock
Generating dev lockfile: /private/tmp/my-project/requirements-dev.lock
Installing dependencies
Successfully built my-project
Installing collected packages: sniffio, idna, h11, certifi, anyio, httpcore, httpx, my-project
Successfully installed anyio-3.6.2 certifi-2022.12.7 h11-0.14.0 httpcore-0.17.0 httpx-0.24.0 idna-3.4 my-project-0.1.0 sniffio-1.3.0

This adds a requirements.lock file that looks like this:

# generated by rye
# use `rye lock` or `rye sync` to update this lockfile
-e file:.

And a requirements-dev.lock file with identical contents. Presumably this starts to differ as you install dev requirements (another Rye feature).

Your folder will now have a .venv hidden folder in it. Inside that is a Python virtual environment containing your installed dependencies and a file called rye-venv.json which just contains:

  "python": "cpython@3.11.1"

Installing global applications

The rye add X command adds a dependency to your current virtual environment / Rye project.

rye install does something completely different: it installs new global packages, in a similar fashion to pipx in that they get their own isolated environments so their dependencies don't clash with other installed applications.

% rye install cowsay
Collecting cowsay
  Downloading cowsay-5.0.tar.gz (25 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: cowsay
  Building wheel for cowsay (pyproject.toml) ... done
  Created wheel for cowsay: filename=cowsay-5.0-py2.py3-none-any.whl size=25707 sha256=de872e9ef328d25cd9ac58df693c7bfd913033f696282c60562854d0db38737e
  Stored in directory: /Users/simon/Library/Caches/pip/wheels/d4/2d/c7/c018bd8e6f825d6b47ae38f28baabd4588b3857e0e7dbc8cd3
Successfully built cowsay
Installing collected packages: cowsay
Successfully installed cowsay-5.0
installed script cowsay

You can now run cowsay with ~/.rye/shims/cowsay:

% ~/.rye/shims/cowsay hello
| hello |
        (__)\       )\/\
            ||----w |
            ||     ||

You can add ~/.rye/shims to your $PATH to make these commands available everywhere.

Failing and then succeeding to install Datasette

I tried to install Datasette using rye install datasette and got this error:

% ~/.rye/shims/datasette 
Traceback (most recent call last):
  File "/Users/simon/.rye/shims/datasette", line 5, in <module>
    from datasette.cli import cli
  File "/Users/simon/.rye/tools/datasette/lib/python3.11/site-packages/datasette/cli.py", line 17, in <module>
    from .app import (
  File "/Users/simon/.rye/tools/datasette/lib/python3.11/site-packages/datasette/app.py", line 14, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

Rye has strong opinions, including omitting pip and setuptools entirely from the environments that it creates.

[ UPDATE: I released Datasette 0.64.3 with a fix for this and now it installs correctly under Rye ]

It turns out Datasette includes code that imports pkg_resources, assuming that setuptools will be present because it's usually there as a Python environment default!

I added setuptools to Datasette's setup.py dependencies in an attempt to fix that, in issue #2065.

When I'm using pip I often install development copies of my projects by feeding them the URL to a .zip generated by GitHub, for example:

pip install https://github.com/simonw/datasette/archive/main.zip

I tried that with rye install and got an error:

 % rye install https://github.com/simonw/datasette/archive/main.zip
Error: Expected one of `@`, `(`, `<`, `=`, `>`, `~`, `!`, `;`, found `:`

Evidently Rye doesn't (yet?) support installing from URLs.

UPDATE: Armin showed me a way to do this:

rye install 'datasette @ https://github.com/simonw/datasette/archive/main.zip'

This worked - running ~/.rye/shims/datasette --version confirmed the installation.

Instead, I downloaded the main.zip file and ran Rye install against that directly:

% rye install main.zip 
Processing ./main.zip
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
Successfully built datasette

That seemed to work. Oddly it didn't add datasette to the .rye/shims directory - but I did find a working installation here instead:



Created 2023-04-26T21:50:27-07:00, updated 2023-04-27T08:19:21-07:00 · History · Edit