If you ever run into instructions that tell you to do this: …
I finally figured out a process that works for me for hacking on Python CLI utilities using uv to manage my development environment, thanks to a little bit of help from Charlie Marsh. …
Inspired by py-free-threading.github.io I decided to try out a beta of Python 3.13 with the new free-threaded mode enabled, which removes the GIL. …
I found myself wanting to compare the version numbers 0.63.1, 1.0 and the 1.0a13 in Python code, in order to mark a pytest test as skipped if the installed version of Datasette was pre-1.0. …
This issue by Parand Darugar pointed out that Datasette doesn't currently run on Linux systems with FIPS enabled, due to the way it uses MD5 hashes. …
While parsing a CSV file using csv.DictReader today I noticed the following surprising result: …
For simonw/til issue #82 I needed to embed some binary content directly in a Python script. …
I wanted a mechanism to check if a given Python object was "callable" - could be called like a function - or "async callable" - could be called using await obj(). …
Datasette currently has a few API internals that return sqlite3.Row objects. I was thinking about how this might work in the future - if Datasette ever expands beyond SQLite (plugin-provided backends for PostgreSQL and DuckDB for example) I'd want a way to return data from other stores using objects that behave like sqlite3.Row but are not exactly that class. …
tree-sitter is a "parser generator tool and an incremental parsing library". It has a very good reputation these days. …
I had a bug that was only showing up in CI against Python 3.8. …
I've been using setuptools and setup.py for my Python packages for a long time: I like that it works without me having to think about installing and learning any additional tools such as Flit or pip-tools or Poetry or Hatch. …
Seth Michael Larson pointed out that the Python gzip module can be used as a CLI tool like this: …
I finally got around to fully learning TOML. Some notes, including how to read and write it from Python. …
pip uses a cache to avoid downloading packages again: …
Rye is Armin Ronacher's new experimental Python packaging tool. I decided to take it for a test-run. …
I've long wanted to run some kind of large language model on my own computer. Now that I have a M2 MacBook Pro I'm even more keen to find interesting ways to keep all of those CPU cores busy. …
I ran into this error while trying to run pip install lxml on an M2 Mac, inside a virtual environment I had intitially created using pipenv shell: …
I wanted to upgrade my git-history installation to a new alpha version. …
Today Carlton told me about the interact command in the Python debugger. …
I puzzled over this one for quite a while this morning. I had this test that was failing with Windows on Python 3.11: …
I've been using Locust recently to run some load tests - most significantly these tests against SQLite running with Django and this test exercising Datasette and Gunicorn. …
I was getting this intermittent error running my Datasette test suite: …
Towards Inserting One Billion Rows in SQLite Under A Minute includes this snippet: …
For sqlite-utils issue 464 I implemented a fix to a tiny bug in a dependency in my own fork on GitHub. …
TIL the Python standard library struct module defaults to interpreting binary strings using the endianness of your machine. …
I tried pip-tools for the first time today to pin the requirements for the natbat/pillarpointstewards Django app. …
TLDR: Use shutil.copyfileobj(fsrc, fdst) …
I needed to generate a grid calendar that looks like this (design by Natalie Downe): …
I wanted to produce the following output from a command-line tool: …
David Beazley on Twitter: …
Carelessly including the output of json.dumps() in an HTML page can lead to an XSS hole, thanks to the following: …
I tried to install my datasette-bplist plugin today in a fresh Python 3.10 virtual environment on macOS and got this error: …
David Beazley on Twitter said: …
I needed to tell both flake8 and mypy to ignore the same line of code. …
My csvs-to-sqlite README includes a section that shows the output of the csvs-to-sqlite --help command (relevant issue). …
For complicated reasons I found myself wanting to write Python code to resolve a graph of dependencies and produce a plan for efficiently executing them, in parallel where possible. …
Pyodide provides "Python with the scientific stack, compiled to WebAssembly" - it's an incredible project which lets you run a full working Jupyter notebook, complete with complex packages such as numpy and pandas, entirely in your browser without any server-side Python component running at all. …
Inspired by this tweet by Mike Driscoll I decided to try using Fabric to run commands over SSH from a Python script, using a public key for authentication. …
For sqlite-utils issue #309 I had an error that looked like this: …
Today I discovered codespell via this Rich commit. codespell is a really simple spell checker that can be run locally or incorporated into a CI flow. …
Today I learned how to use the Python trace module to output every single executed line of Python code in a program - useful for figuring out exactly when a crash or infinite loop happens. …
When building the --sniff option for sqlite-utils insert (which attempts to detect the correct CSV delimiter and quote character by looking at the first 2048 bytes of a CSV file) I had the need to peek ahead in an incoming stream of data. …
Users were reporting the following error using sqlite-utils to import some CSV files: …
PyInstaller can take a Python script and bundle it up as a standalone executable for macOS, Linux and apparently Windows too (I've not tried it on Windows yet). …
asyncio in Python is a form of co-operative multitasking, where everything runs in a single thread but asynchronous tasks can yield to other tasks to allow them to execute. …
I had a list of Python dictionaries I wanted to output as YAML, but I wanted to control the style of the output. …
I figured out how to run a subprocess with a time limit for datasette-ripgrep, using the asyncio.create_subprocess_exec() method. The pattern looks like this: …
sqlite-utils provides a decorator for registering custom Python functions that looks like this: …
I wanted to add a --encoding option to sqlite-utils insert which could be used to change the file encoding used to read the incoming CSV or TSV file - see sqlite-utils #182. …
I hit a bug today where I had defined a Click option called open but in doing so I replaced the Python bulit-in open() function: …
This tip is for when you are working on a Python command-line application that runs using that program's name, as opposed to typing python my_script.py. I usually need this when I'm working on applications built using Click, e.g. projects I start using my click-app cookiecutter template. …
datasette-leaflet-geojson outputs GeoJSON geometries in HTML pages in a way that can be picked up by JavaScript and used to plot a Leaflet map. …
I needed this for the datasette install and datasette uninstall commands, see issue #925. …
I was researching password hashing for datasette-auth-passwords. I wanted very secure defaults that would work using the Python standard library without any extra dependencies. …
For https://github.com/simonw/datasette/issues/581 I want to be able to inspect a Python function to determine which named parameters it accepts and send only those arguments. …
First, checkout the cpython repo: …
For photos-to-sqlite I needed to install osxphotos as a dependency, but only if the platform is macOS - it's not available for Linux. …
If you installed datasette using pipx install datasette you can install additional plugins with pipx inject like so: …
I was trying to figure out the shape of the JSON object from https://github.com/simonw/coronavirus-data-gov-archive/blob/master/data_latest.json?raw=true - which is 3.2MB and heavily nested, so it's difficult to get a good feel for the shape. …
Once you have installed the "command line tools" for Catalina using the following terminal command: …
I wanted to convert a datetime object (from GitPython) to UTC without adding the pytz dependency. …