pytest-xdist workers race on writing .ambr files, silently destroying snapshots
When --snapshot-update runs under pytest-xdist and more than one worker has updates for the same amber file, AmberDataSerializer.write_file performs an unsynchronized read-modify-write on that file. Workers read the file while another worker is streaming it out, get a truncated parse, merge their own snapshots into that partial content, and write it back. Snapshots are lost.
pytest exits 0 and the summary reports every snapshot as successfully updated, so nothing signals the loss. It surfaces later as a "deleted snapshot" in the diff, which reads like an unused-snapshot removal but is not one.
This is not the unused-snapshot detection added in #1132. In the failing runs SnapshotSession.remove_unused_snapshots is never called, the report's unused set is empty. The data is lost in the writes.
To reproduce
You'll need this file:
test_race.py
import pytest
VALUE = "v1" # bump to v2 to make every snapshot need an update
@pytest.mark.parametrize("i", range(200)) # increase this number if needed on your machine
def test_many(snapshot, i):
assert f"{VALUE}-{i}" == snapshot
Then, assuming you have uv installed on your machine:
$ uv init
$ uv add pytest syrupy pytest-xdist
$ uv run pytest test_race.py --snapshot-update -q
200 snapshots generated.
$ grep -c '^# name:' __snapshots__/test_race.ambr # just to check that .ambr file is consistent
200
$ sed -i 's/VALUE = "v1"/VALUE = "v2"/' test_race.py # replace v1 by v2
$ uv run pytest test_race.py --snapshot-update -q -n auto # or -n number_of_workers
200 snapshots updated. # reports full success, exits 0
$ grep -c '^# name:' __snapshots__/test_race.ambr
44 # 156 snapshots destroyed on my side... 🥲
The serial control (same command without -n auto) leaves 200/200 every time.
Note that snapshots which merely passed are the most vulnerable: a worker only holds the snapshots it rewrote in its own queue, so everything else survives purely through the read-modify-write base. On a large suite where only a handful of snapshots change, a torn base read discards the untouched majority.
Environment
- syrupy 6.0.0 (latest)
- pytest 9.1.1 (latest)
- pytest-xdist 3.8.0 (latest)
- Python 3.13.9
- Linux 7.0.0, 22 logical CPUs (
-n auto ➜ 22 workers)
pytest-xdistworkers race on writing .ambr files, silently destroying snapshotsWhen
--snapshot-updateruns underpytest-xdistand more than one worker has updates for the same amber file,AmberDataSerializer.write_fileperforms an unsynchronized read-modify-write on that file. Workers read the file while another worker is streaming it out, get a truncated parse, merge their own snapshots into that partial content, and write it back. Snapshots are lost.pytest exits 0 and the summary reports every snapshot as successfully updated, so nothing signals the loss. It surfaces later as a "deleted snapshot" in the diff, which reads like an unused-snapshot removal but is not one.
This is not the unused-snapshot detection added in #1132. In the failing runs
SnapshotSession.remove_unused_snapshotsis never called, the report'sunusedset is empty. The data is lost in the writes.To reproduce
You'll need this file:
test_race.pyThen, assuming you have
uvinstalled on your machine:The serial control (same command without
-n auto) leaves 200/200 every time.Note that snapshots which merely passed are the most vulnerable: a worker only holds the snapshots it rewrote in its own queue, so everything else survives purely through the read-modify-write base. On a large suite where only a handful of snapshots change, a torn base read discards the untouched majority.
Environment
-n auto➜ 22 workers)