I built a new API reference page today for
sqlite-utils, using the Sphinx autodoc extension to extract docstrings from the code and use them to build a full class reference.
I've avoided this kind of documentation in the past because I think narrative prose is a much better way of providing documentation - but
sqlite-utils already has detailed narrative prose, so I felt that adding reference documentation powered by docstrings could enhance that project - while also providing better inline document for tools such as Visual Studio Code and Jupyter.
Getting autodoc working in my local documentation environment took a few steps.
First, I added
"sphinx.ext.autodoc" to the
extensions =  list in my
Then I created a new page in
docs/reference.rst and added the following:
=============== API Reference =============== .. _reference_db_database: sqlite_utils.db.Database ======================== .. autoclass:: sqlite_utils.db.Database :members: :undoc-members:
This was enough to start generating documentation for the
:members: means "show documentation for everything in the class that has a docstring".
:undoc-members: means "also include class members that don't have a docstring yet".
I added my own rST headings and reference links - this ensured that the classes would automatically show up in the table of contents for my documentation.
I use sphinx-autobuild to automatically rebuild and reload the browser while I'm editing documentation.
This has a known limitation that it won't pick up on edits made to the
*.py files that should cause a reload.
I found a workaround: add
-a to disable incremental builds (rebuilding everything) and
--watch ../sqlite_utils to trigger a new build when any file in that folder was updated.
I added this to the Makefile is the
livehtml: sphinx-autobuild -a -b html "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(0) --watch ../sqlite_utils
make livehtml started a web server for the documentation that would refresh the browser any time I edited a docstring.
I ended up using the following:
.. autoclass:: sqlite_utils.db.Database :members: :undoc-members: :show-inheritance: :special-members: __getitem__ :exclude-members: use_counts_table, execute_returning_dicts, resolve_foreign_keys
:show-inheritance: adds a note about the inheritance hierarchy for the class.
__special__ methods like
__getitem__ are not included.
:special-members: fixes this for specified methods.
:exclude-members; accepts a comma-separated list of methods that shouldn't be shown in the documentation. I used this to remove some internal methods.
I could have renamed these to
_use_counts_table but that would be a breaking API change, even though these methods are deliberately undocumented and shouldn't be used by anyone.
The class methods default to being shown in alphabetical order - adding
autodoc_member_order = "bysource" to
docs/conf.py changed that to source order instead.
Adding reStructuredText to the docstrings works for formatting the generated API documentation. You can also use
:ref:reference-links. Here's an example of a docstring with more complex formatting:
def rows_where( self, where: str = None, where_args: Optional[Union[Iterable, dict]] = None, order_by: str = None, select: str = "*", limit: int = None, offset: int = None, ) -> Generator[dict, None, None]: """ Iterate over every row in this table or view that matches the specified where clause. Returns each row as a dictionary. See :ref:`python_api_rows` for more details. :param where: SQL where fragment to use, for example ``id > ?`` :param where_args: Parameters to use with that fragment - an iterable for ``id > ?`` parameters, or a dictionary for ``id > :id`` :param order_by: Column or fragment of SQL to order by :param select: Comma-separated list of columns to select - defaults to ``*`` :param limit: Integer number of rows to limit to :param offset: Integer for SQL offset """ if not self.exists(): # ...
If you add
autodoc_typehints = "description" to the
conf.py configuration the
:param name: Description lines will be shown in a list of parameters that also includes the type hints:
I learned about this in sqlite-utils/issues/413 while trying to improve the display of complex type annotations.
Read The Docs has the option to deploy branches - you can set that up in the "Versions" tab. I was working in an
autodoc branch - I set that branch to be "Active" and "Hidden", so it would be built by Read The Docs but not linked to from anywhere.
My first attempt failed to render the generated documentation, because Read The Docs did not know where to import my code from.
I solved that by adding a
.readthedocs.yaml configuration file to my repository containing the following:
version: 2 sphinx: configuration: docs/conf.py python: version: "3.8" install: - method: pip path: . extra_requirements: - docs
The key thing here is that
install block, which tells Read The Docs to run the equivalent of this before building the documentation:
pip install -e '.[docs]'
For my repository that installs the
docs optional dependencies from the
setup.py, but also installs the
sqlite_utils Python package itself. That way Sphinx can find and import
sqlite_utils.db.Database in order to generate the documentation.
The new page of documentation is now live at en/latest/reference.html. The pull request in which I figured this all out is sqlite-utils/pull/312.
Created 2021-08-10T16:31:15-07:00, updated 2022-03-11T10:06:59-08:00 · History · Edit