Simon Willison: TIL

Things I've learned, collected in simonw/til. You may also enjoy my blog.

Atom feed

Browse by topic: amplitude 1 · asgi 1 · auth0 2 · aws 6 · azure 1 · bash 8 · caddy 1 · cloudflare 1 · cloudrun 8 · cookiecutter 2 · datasette 8 · deno 1 · digitalocean 1 · django 15 · docker 7 · electron 6 · firefox 1 · fly 6 · gis 2 · git 2 · github 10 · github-actions 21 · google-sheets 1 · googlecloud 4 · gpt3 1 · graphql 2 · heroku 3 · homebrew 5 · ics 1 · imagemagick 2 · javascript 14 · jinja 2 · jq 6 · jupyter 1 · kubernetes 2 · linux 4 · macos 12 · markdown 3 · mediawiki 1 · networking 1 · nginx 1 · node 1 · npm 3 · observable-plot 1 · pixelmator 1 · pluggy 1 · postgresql 6 · presenting 1 · purpleair 1 · pypi 1 · pytest 16 · python 38 · quarto 1 · readthedocs 5 · reddit 1 · selenium 2 · service-workers 1 · shot-scraper 1 · spatialite 3 · sphinx 3 · sql 3 · sqlite 28 · svg 1 · tailscale 1 · tesseract 1 · twitter 2 · typescript 1 · vega 1 · vim 1 · vscode 2 · web-components 1 · webassembly 1 · webauthn 1 · wikipedia 1 · zeit-now 2 · zsh 2

Recent TILs

quarto Trying out Quarto on macOS - 2022-08-18

I decided to try out Quarto, the new notebook/markdown/publishing system built on Pandoc. …

sqlite Sort by number of JSON intersections - 2022-08-17

This post on Reddit asked how to run a query that takes a list of items (in this case ingredients) as the input and returns all rows with at least one of those items in a JSON list, ordered by the most matches. …

googlecloud Analyzing Google Cloud spend with Datasette - 2022-08-16

Google Cloud provide extremely finely grained billing, but you need to access it through BigQuery which I find quite inconvenient. …

python Defining setup.py dependencies using a URL - 2022-08-13

For sqlite-utils issue 464 I implemented a fix to a tiny bug in a dependency in my own fork on GitHub. …

javascript Creating a tiled zoomable image with OpenSeadragon and vips - 2022-08-12

The San Francisco Microscopical Society has some extremely high resolution scanned images - one of them is a 1.67GB PNG file with a 25,088 × 17,283 pixel resolution. …

gpt3 Using GPT-3 to figure out jq recipes - 2022-08-10

I like jq, but I always have to think pretty hard about how to construct the right syntax for it. …

pytest Mocking a Textract LimitExceededException with boto - 2022-08-07

For s3-ocr issue #21 I needed to write a test that simulates what happens when Amazon Textract returns a "LimitExceededException". When using boto this error presents itself as an exception: …

sqlite Trying out SQLite extensions on macOS - 2022-08-03

Alex Garcia has been building some really cool new custom extensions for SQLite, working in C and Go. So far he's released two: …

aws Using boto3 from the command line - 2022-08-02

I found a useful pattern today for automating more complex AWS processes as pastable command line snippets, using Boto3. …

sqlite Related content with SQLite FTS and a Datasette template function - 2022-07-31

Today I added "related TILs" to this TIL website - so each TIL now shows five related TILs at the bottom of the page. …

docker Emulating a big-endian s390x with QEMU - 2022-07-29

I got a bug report concerning my sqlite-fts4 project running on PPC64 and s390x architectures. …

github Migrating a GitHub wiki from one repository to another - 2022-07-28

I figured out how to migrate a GitHub wiki (public or private) from one repository to another while preserving all history. …

python struct endianness in Python - 2022-07-28

TIL the Python standard library struct module defaults to interpreting binary strings using the endianness of your machine. …

docker Testing things in Fedora using Docker - 2022-07-27

I got a report of a bug with my s3-ocr tool running on Fedora. …

fly Deploying a redbean app to Fly - 2022-07-24

redbean is a fascinating project - it provides a web server in a self-contained executable which you can add assets (or dynamic Lua code) to just by zipping them into the same binary package. …

pytest Using pytest and Playwright to test a JavaScript web application - 2022-07-24

I decided to add automated tests to my Datasette Lite project. Datasette Lite bundles my Datasette Python web application as a client-side application running inside WebAssembly using Pyodide. …

python Freezing requirements with pip-tools - 2022-07-14

I tried pip-tools for the first time today to pin the requirements for the natbat/pillarpointstewards Django app. …

electron Testing Electron apps with Playwright and GitHub Actions - 2022-07-13

Yesterday I figured out (issue 133) how to use Playwright to run tests against my Electron app, and then execute those tests in CI using GitHub Actions, for my datasett-app repo for my Datasette Desktop macOS application. …

npm Upgrading packages with npm - 2022-07-13

There's a new version of Vite out (3.0) and I wanted to upgrade my datasette-table package to use it. …

github-actions Conditionally running a second job in a GitHub Actions workflow - 2022-07-11

My simonwillisonblog-backup workflow periodically creates a JSON backup of my blog's PostgreSQL database, using db-to-sqlite and sqlite-diffable. It then commits any changes back to the repo using this pattern: …

github Reporting bugs in GitHub to GitHub - 2022-07-07

I found out today (via this post) about a dedicated interface for reporting bugs in GitHub to GitHub: …

zsh Passing command arguments using heredoc syntax - 2022-07-07

This trick works for both Bash and zsh. …

aws Running OCR against a PDF file with AWS Textract - 2022-06-28

Textract is the AWS OCR API. It's very good - I've fed it hand-written notes from the 1890s and it read them better than I could. …

bash Ignoring errors in a section of a Bash script - 2022-06-27

For simonw/museums#32 I wanted to have certain lines in my Bash script ignore any errors: lines that used sqlite-utils to add columns and configure FTS, but that might fail with an error if the column already existed or FTS had already been configured. …

sqlite One-liner for running queries against CSV files with SQLite - 2022-06-20

I figured out how to run a SQL query directly against a CSV file using the sqlite3 command-line utility: …

datasette Registering new Datasette plugin hooks by defining them in other plugins - 2022-06-17

I'm experimenting with a Datasette plugin that itself adds new plugin hooks which other plugins can then interact with. …

linux Using iconv to convert the text encoding of a file - 2022-06-14

In sqlite-utils issue 439 I was testing against a CSV file that used UTF16 little endian encoding, also known as utf-16-le. …

jinja Formatting thousands in Jinja - 2022-06-08

Here's how to format a number in Jinja with commas for thousands, without needing any custom filters or template functions: …

networking Making HTTP calls using IPv6 - 2022-06-08

Tiny TIL today: I learned how to make an HTTP call to an IPv6 address. The trick is to enclose the address in the URL in square braces: …

django Using just with Django - 2022-06-06

Jeff Triplett convinced me to take a look at just as a command automation tool - sort of an alternative to Make, except with a focus on commands rather than managing build dependencies. …

Browse all 313 TILs