Using cog to update --help in a Markdown README file

My csvs-to-sqlite README includes a section that shows the output of the csvs-to-sqlite --help command (relevant issue).

I had been manually copying this in, but I decided to try using cog to automate the process.

Here's what I came up with:

<!-- [[[cog
import cog
from csvs_to_sqlite import cli
from click.testing import CliRunner
runner = CliRunner()
result = runner.invoke(cli.cli, ["--help"])
help = result.output.replace("Usage: cli", "Usage: csvs-to-sqlite")
cog.out(
    "```\n{}\n```".format(help)
)
]]] -->
```
Usage: csvs-to-sqlite [OPTIONS] PATHS... DBNAME
...
```
<!-- [[[end]]] -->

Then to update the README file, run this:

cog -r README.md

The -r option causes it to modify that file in place.

Cog works by scanning for a [[[cog ... ]]] section, executing the code there, capturing the cog.out() output and using that to replace everything from the end of the code block up to the line containing the [[[end]]] marker.

It's designed to interact well with comments - in this case HTML comments - such that the cog generation code can be hidden.

Testing with cog --check

A version of Cog released after I first wrote this TIL added a new --check option, so you can run a test in CI to check if the file needs to be updated using:

cog --check README.md

Writing a test (before cog --check)

Any time I generate content like this in a repo I like to include a test that will fail if I forget to update the content.

cog clearly isn't designed to be used as an independent library, but I came up with the following pattern pytest test which works well, in my tests/test_csvs_to_sqlite.py module:

from cogapp import Cog
import sys
from io import StringIO
import pathlib


def test_if_cog_needs_to_be_run():
    _stdout = sys.stdout
    sys.stdout = StringIO()
    readme = pathlib.Path(__file__).parent.parent / "README.md"
    result = Cog().main(["cog", str(readme)])
    output = sys.stdout.getvalue()
    sys.stdout = _stdout
    assert (
        output == readme.read_text()
    ), "Run 'cog -r README.md' to update help in README"

The key line here is this one:

result = Cog().main(["cog", str(readme)])

In cog's implementation, that code is called like this:

Cog().main(sys.argv)

Here I'm faking the command-line arguments to pass in just the path to my README.md file.

Cog then writes the generated output to stdout - which I capture with that sys.stdout trick.

Finally, I compare the generated output to the current file content and fail the test with a reminder to run cog -r if they do not match.

Cog for reStructuredText

Here's an example of cog in a .rst file:

.. [[[cog
    import tabulate
    cog.out("\n" + "\n".join('- ``{}``'.format(t) for t in tabulate.tabulate_formats) + "\n\n")
.. ]]]

- ``fancy_grid``
- ``fancy_outline``
- ``github``

.. [[[end]]]

The trailing and leading newlines are important to avoid a warning about "Explicit markup ends without a blank line; unexpected unindent".

CLI reference pages

I added these to both Datasette and sqlite-utils: full pages that list the help for every command provided by those tools.

Datasette CLI reference - underlying source code
sqlite-utils CLI reference - underlying source code

Created 2021-11-18T08:27:19-08:00, updated 2022-02-12T08:07:17-08:00 · History · Edit

Using cog to update --help in a Markdown README file

Testing with cog --check

Writing a test (before cog --check)

Cog for reStructuredText

CLI reference pages

Related