Snapshot testing with Syrupy

I'm a big fan of snapshot testing - writing tests where you compare the output of some function to a previously saved version, and can re-generate that version from scratch any time something changes.

I usually do this by hand - I run pytest -x --pdb to stop at the first failing test and drop into a debugger, then copy out the representation of the generated value and copy it into the test. I wrote about how I use this pattern a few years ago in How to cheat at unit tests with pytest and Black.

Today I learned how to do the same thing with the Syrupy plugin for pytest. I think I'll be using this for many of my future projects.

Some initial tests

I created a tests/test_stuff.py file with the following contents:

def test_one(snapshot):
    assert "hello" == snapshot


def test_two(snapshot):
    assert snapshot == {"foo": [1, 2, 3], "bar": {"baz": "qux"}}

Then I installed both pytest and syrupy:

pip install pytest syrupy

Now in my parent folder I can run this:

pytest

And the tests fail:

tests/test_stuff.py FF                                                              [100%]

======================================== FAILURES =========================================
________________________________________ test_one _________________________________________

snapshot = SnapshotAssertion(name='snapshot', num_executions=1)

    def test_one(snapshot):
>       assert "hello" == snapshot
E       AssertionError: assert [+ received] == [- snapshot]
E         Snapshot 'test_one' does not exist!
E         + 'hello'

tests/test_stuff.py:2: AssertionError
________________________________________ test_two _________________________________________

snapshot = SnapshotAssertion(name='snapshot', num_executions=1)

    def test_two(snapshot):
>       assert snapshot == {"foo": [1, 2, 3], "bar": {"baz": "qux"}}
E       AssertionError: assert [- snapshot] == [+ received]
E         Snapshot 'test_two' does not exist!
E         + dict({
E         +   'bar': 
E         
E         ...Full output truncated (9 lines hidden), use '-vv' to show

tests/test_stuff.py:5: AssertionError
--------------------------------- snapshot report summary ---------------------------------
2 snapshots failed.
================================= short test summary info =================================
FAILED tests/test_stuff.py::test_one - AssertionError: assert [+ received] == [- snapshot]
FAILED tests/test_stuff.py::test_two - AssertionError: assert [- snapshot] == [+ received]
==================================== 2 failed in 0.05s ====================================

The snapshots don't exist yet. But I can create them automatically by running this:

pytest --snapshot-update

Which outputs passing tests along with:

--------------------------------- snapshot report summary ---------------------------------
2 snapshots generated.
==================================== 2 passed in 0.01s ====================================

And sure enough, there's now a new folder called tests/__snapshots__ with a file called test_stuff.ambr which contains this:

# serializer version: 1
# name: test_one
  'hello'
# ---
# name: test_two
  dict({
    'bar': dict({
      'baz': 'qux',
    }),
    'foo': list([
      1,
      2,
      3,
    ]),
  })
# ---

Running pytest again passes, because the snapshots exist and continue to match the test output.

The serialized snapshot format is designed to be checked into Git. It's pleasantly readable - I can review that and see what it's testing, and I could even update it by hand - though I'll much more likely use the --snapshot-update flag and then eyeball the differences.

Adding a dataclass

My snapshots so far are pretty simple - a string and a nested dictionary. I decided to add a dataclass to my code and see what that looks like:

import dataclasses


@dataclasses.dataclass
class Foo:
    bar: int
    baz: str


def test_one(snapshot):
    assert "hello" == snapshot


def test_two(snapshot):
    assert snapshot == {"foo": [1, 2, 3], "bar": {"baz": "qux"}}


def test_three(snapshot):
    assert Foo(1, "hello") == snapshot

Running pytest again failed. pytest --snapshot-update passed and updated my snapshot file, adding this to it:

# name: test_three
  Foo(bar=1, baz='hello')

OK, neat - it looks like it's using the Dataclass's __repr__ method to serialize the object.

I tried it with a custom non-dataclass object... and it worked too!

class WeirdClass:
    def __init__(self, foo, bar):
        self.foo = foo
        self.bar = bar

def test_four(snapshot):
    assert WeirdClass(1, 2) == snapshot

Serialized to:

# name: test_four
  WeirdClass(
    bar=2,
    foo=1,
  )

I wasn't expecting this to work. The Syrupy documentation says:

The default serializer supports all python built-in types and provides a sensible default for custom objects.

It looks like there are a bunch of more advanced ways to customize objects to make them work well with Syrupy, but I haven't dived into those yet.

First impressions are that this looks like exactly the snapshot tool I've been waiting for.

Related

• cookiecutter Testing cookiecutter templates with pytest - 2021-01-27
• pytest Using pytest and Playwright to test a JavaScript web application - 2022-07-24
• pytest Mocking subprocess with pytest-subprocess - 2023-03-08
• pytest pytest coverage with context - 2022-03-04
• pytest Using VCR and pytest with pytest-recording - 2021-11-02
• pytest Quick and dirty mock testing with mock_calls - 2021-11-02
• pytest Writing pytest tests against tools written with argparse - 2022-01-08
• datasette Using pytest-httpx to run intercepted requests through an in-memory Datasette instance - 2023-07-24
• python The pdb interact command - 2022-10-31
• pytest Registering temporary pluggy plugins inside tests - 2020-07-21

Created 2023-09-26T16:39:12-07:00 · Edit