Snapshot testing with Syrupy

I'm a big fan of snapshot testing - writing tests where you compare the output of some function to a previously saved version, and can re-generate that version from scratch any time something changes.

I usually do this by hand - I run pytest -x --pdb to stop at the first failing test and drop into a debugger, then copy out the representation of the generated value and copy it into the test. I wrote about how I use this pattern a few years ago in How to cheat at unit tests with pytest and Black.

Today I learned how to do the same thing with the Syrupy plugin for pytest. I think I'll be using this for many of my future projects.

Some initial tests

I created a tests/ file with the following contents:

def test_one(snapshot):
    assert "hello" == snapshot

def test_two(snapshot):
    assert snapshot == {"foo": [1, 2, 3], "bar": {"baz": "qux"}}

Then I installed both pytest and syrupy:

pip install pytest syrupy

Now in my parent folder I can run this:


And the tests fail:

tests/ FF                                                              [100%]

======================================== FAILURES =========================================
________________________________________ test_one _________________________________________

snapshot = SnapshotAssertion(name='snapshot', num_executions=1)

    def test_one(snapshot):
>       assert "hello" == snapshot
E       AssertionError: assert [+ received] == [- snapshot]
E         Snapshot 'test_one' does not exist!
E         + 'hello'

tests/ AssertionError
________________________________________ test_two _________________________________________

snapshot = SnapshotAssertion(name='snapshot', num_executions=1)

    def test_two(snapshot):
>       assert snapshot == {"foo": [1, 2, 3], "bar": {"baz": "qux"}}
E       AssertionError: assert [- snapshot] == [+ received]
E         Snapshot 'test_two' does not exist!
E         + dict({
E         +   'bar': 
E         ...Full output truncated (9 lines hidden), use '-vv' to show

tests/ AssertionError
--------------------------------- snapshot report summary ---------------------------------
2 snapshots failed.
================================= short test summary info =================================
FAILED tests/ - AssertionError: assert [+ received] == [- snapshot]
FAILED tests/ - AssertionError: assert [- snapshot] == [+ received]
==================================== 2 failed in 0.05s ====================================

The snapshots don't exist yet. But I can create them automatically by running this:

pytest --snapshot-update

Which outputs passing tests along with:

--------------------------------- snapshot report summary ---------------------------------
2 snapshots generated.
==================================== 2 passed in 0.01s ====================================

And sure enough, there's now a new folder called tests/__snapshots__ with a file called test_stuff.ambr which contains this:

# serializer version: 1
# name: test_one
# ---
# name: test_two
    'bar': dict({
      'baz': 'qux',
    'foo': list([
# ---

Running pytest again passes, because the snapshots exist and continue to match the test output.

The serialized snapshot format is designed to be checked into Git. It's pleasantly readable - I can review that and see what it's testing, and I could even update it by hand - though I'll much more likely use the --snapshot-update flag and then eyeball the differences.

Adding a dataclass

My snapshots so far are pretty simple - a string and a nested dictionary. I decided to add a dataclass to my code and see what that looks like:

import dataclasses

class Foo:
    bar: int
    baz: str

def test_one(snapshot):
    assert "hello" == snapshot

def test_two(snapshot):
    assert snapshot == {"foo": [1, 2, 3], "bar": {"baz": "qux"}}

def test_three(snapshot):
    assert Foo(1, "hello") == snapshot

Running pytest again failed. pytest --snapshot-update passed and updated my snapshot file, adding this to it:

# name: test_three
  Foo(bar=1, baz='hello')

OK, neat - it looks like it's using the Dataclass's __repr__ method to serialize the object.

I tried it with a custom non-dataclass object... and it worked too!

class WeirdClass:
    def __init__(self, foo, bar): = foo = bar

def test_four(snapshot):
    assert WeirdClass(1, 2) == snapshot

Serialized to:

# name: test_four

I wasn't expecting this to work. The Syrupy documentation says:

The default serializer supports all python built-in types and provides a sensible default for custom objects.

It looks like there are a bunch of more advanced ways to customize objects to make them work well with Syrupy, but I haven't dived into those yet.

First impressions are that this looks like exactly the snapshot tool I've been waiting for.

Created 2023-09-26T16:39:12-07:00 · Edit