TIL search: Snippet
...That's what this bit does:
```sql
substr(
snippet,
0,
min(
case
when instr(snippet, '&') > 0 then instr(snippet, '&')
else 128
end,
case
when instr(snippet, ']') > 0 then instr(snippet, ']')
else 128
end,
case
when instr(snippet, ' ') > 0 then instr(snippet, ' ')
else 128
end,
case
when instr(snippet, '.') > 0 then instr(snippet, '.')
else 128
end
)
) as refcode
```
I'm trying...
I grabbed [a quote](https://simonwillison.net/2023/Dec/1/jeremy-howard/) from a transcript of a snippet of a YouTube video today for my blog.
I use the [MacWhisper](https://goodsnooze.gumroad.com/l/macwhisper) macOS desktop app to run Whisper. It's a very pleasant GUI wrapper around the Whisper transcription model.
Usually I pull a full YouTube...
I ran a Google search this morning for `s3-credentials simon willison` and when I clicked on the top result it jumped me to a highlighted snippet of text on the page, despite that page not having relevant anchor links.
...I ended up using a pattern inspired by [this Django Snippet](https://djangosnippets.org/snippets/10767/), but with an added touch for more efficient exports. In order to avoid using up too much memory for the export, I use keyset pagination to fetch 500 rows at a time.
The `keyset_pagination_iterator()` helper function accepts any queryset, orders it by...
I found a useful pattern today for automating more complex AWS processes as pastable command line snippets, using [Boto3](https://aws.amazon.com/sdk-for-python/).
The trick is to take advantage of the fact that `python3 -c '...'` lets you pass in a multi-line Python string which will be executed directly.
I used that to create a new IAM...
...sql: |-
select
path as key,
title,
body as search_1,
1 as is_public
from
tutorials
display_sql: |-
select
highlight(
body, :q
) as snippet
from
tutorials
where
tutorials.path = :key
display: |-
<h3>Tutorial: <a href="{{ key }}">{{ title }}</a></h3>
<p>{{ display.snippet|safe }}</p>
```
See [Building a search engine for datasette.io](https://simonwillison.net/2020/Dec/19/dogsheep-beta...
Facebook's [musicgen](https://huggingface.co/facebook/musicgen-small) is a model that generates snippets of audio from a text description - it's effectively a Stable Diffusion for music.
It turns out it's pretty easy to run it using Python, thanks to the Hugging Face [transformers](https://pypi.org/project/transformers/) library.
Here's the code that worked for...
...Here's the full HTML and JavaScript snippet I used:
```html
<div>
<textarea class="copyable">{{ copyable }}</textarea>
<p class="raw-link"><a href="{{ raw_link }}">Raw data</a></p>
</div>
<script>
var ta = document.querySelector("textarea.copyable");
var p = document.querySelector("p.raw-link");
var button = document.createElement("button");
button.className = "copyable-copy-button";
button.innerHTML = "Copy to clipboard";
button...
[Towards Inserting One Billion Rows in SQLite Under A Minute](https://avi.im/blag/2021/fast-sqlite-inserts/) includes this snippet:
> All I had to do was run my existing code, without any change, using PyPy. It worked and the speed bump was phenomenal. The batched version took only 2.5 minutes to insert 100M rows. I got close to...
...It gains access to three new actions:
* `wikipedia: <search term>` - search Wikipedia and returns the snippet of the first result
* `simon_blog_search: <search term>` - search my blog and return the truncated text of the first result
* `calculate: <expression>` - evaluate an expression using Python's `eval()` function (so dangerous! This should absolutely use something like [a WebAssembly sandbox](https://til...
...I ended up [adding this snippet](https://github.com/simonw/datasette/commit/ffca55dfd7cc9b53522c2e5a2fa1ff67c9beadf2) of jQuery JavaScript to my custom ` docs/_templates/layout.html` template:
```html+jinja
{% block footer %}
{{ super() }}
<script>
jQuery(function ($) {
// Show banner linking to /stable/ if this is a /latest/ page
if (!/\/latest\//.test(location.pathname)) {
return;
}
var stableUrl = location.pathname.replace("/latest/", "/stable/");
// Check it's not...
...like', 'max', 'max', 'min', 'min', 'lead', 'lead', 'lead', 'log10', 'lower', 'ltrim', 'ltrim', 'first_value', 'pi', 'length', 'likely', 'json_set', 'escape_fts', 'prepare_connection_args', 'convert_units', 'sleep', 'rtreedepth', 'match', 'snippet', 'fts5_source_id', 'offsets', 'matchinfo', 'matchinfo', 'optimize', 'rtreecheck', 'rtreenode', 'highlight', 'bm25', 'fts3_tokenizer', 'fts3_tokenizer', 'fts5'`
## Comparing via copy-and-paste
To see the functions that were registered for https...
...mapping an incoming HTTP request to some code based on both the URL path and the HTTP verb
>
> Don't show installation instructions, just get straight to the code snippets
(I added that second line after it first started spitting out installation instructions for each framework, when I just wanted to see some illustrative code examples.)
I got really great...
...0.25516,
"formattedSearchTime": "0.26",
"totalResults": "65200",
"formattedTotalResults": "65,200"
},
"items": [
{
"kind": "customsearch#result",
"title": "hhs",
"htmlTitle": "hhs",
"link": "https://hhscovid.publicaccountability.org/hhs",
"displayLink": "hhscovid.publicaccountability.org",
"snippet": "Powered by Datasette · Queries took 5.536ms · Data source: U.S. Department of Health & Human Services · Home · Name Search · Dataset Search · Browse Datasets.",
"htmlSnippet": "<b>Powered by Datasette</b> · Queries took...
...feature["geometry"]}
' --pk tzid
```
This Bash one-liner loads the entire `combined.json` file (140MB of it) into memory and then runs a snippet of Python to loop through all of those features and yield `{"tzid": "...", "geometry": {...}}` dictionaries for each timezone.
`sqlite-utils` then creates a table with this schema and inserts those records as JSON strings:
```sql
CREATE TABLE...
...a forum, news aggregator, or a blog commenting system.
>
> Here's an explanation of key components related to read ribbons and their functionality as inferred from the provided SQL snippets:
>
> 1. **Purpose:** Read ribbons seem to serve the purpose of tracking whether users have read or followed specific stories or posts. This feature is likely used to manage and display...
...The code itself is a tiny snippet of TypeScript:
```typescript
/**
* Yassify a string of text by appending emoji
*
* @param str The string of text to yassify.
* @returns a string of text with emoji appended
*/
export function yassify(str: string): string {
return `${str} 💅✨👑`;
}
```
It's published to JSR [here](https://jsr.io/@kwhinnery/yassify). That page includes `npm` instructions for using...
...true
}));
});
```
This snippet loops through every `<textarea>` element on the page and dispatches an artificial `input` event on it - which triggers our `onInput=` attribute and updates the `data-replicated-value` attribute, which then updates the size of the accompanying pseudo-element.
This also means that if you load the page with a value already in the `<textarea>` (on an edit...
...testserver/blog/{entry.created.year}/{entry.slug}/"
)
assert (
xml_entry.find(
"{http://www.w3.org/2005/Atom}author/{http://www.w3.org/2005/Atom}name"
).text
== "author"
)
```
## The finished blog
Check it out at https://www.datasette.cloud/blog/
Consider the code snippets in this TIL licensed under [Apache License, Version 2.0](https://opensource.org/license/apache-2-0/).
...Broken up here into copy-pastable snippets:
```
.mode qbox
```
This turned on a new-to-me SQLite output mode which is very pleasant.
```
-- create tables as normal
create table foo (a primary key, b);
create table baz (a primary key, b, c, d);
```
```
-- update those tables to be crrs / crdts
select crsql_as_crr('foo');
select crsql_as_crr('baz...