Skip to content

Lack of robustness in test_transform_identity #1238

Description

@jscanvic

Environment

Latest

Describe the bug

In #1236, the way shifts are performed is fixed so that x_shift corresponds to horizontal displacements and y_shift to vertical displacement, instead of the other way around. It turns out that this makes test_transform_identity fail for unrelated reasons. Indeed, the test is inherently unstable and fails for certain RNG states.

See e.g. this CI failure: https://github.com/deepinv/deepinv/actions/runs/27545402944/job/81421195001

We should make it more stable.

Reproducing

  1. Checkout commit 089abf1
  2. Verify that python -m pytest deepinv/tests/test_transform.py -k "test_transform_identity" passes:
=================================================================== 100 passed, 104 deselected, 41 warnings in 11.38s ===================================================================
  1. Apply this diff changing the RNG state in the test
diff --git a/deepinv/tests/test_transform.py b/deepinv/tests/test_transform.py
index 479b4dc04..223429ebe 100644
--- a/deepinv/tests/test_transform.py
+++ b/deepinv/tests/test_transform.py
@@ -224,6 +224,8 @@ def test_transform_identity(
         # more reliable with a cpu rng here
         rng = torch.Generator().manual_seed(0)
 
+    rng = torch.Generator().manual_seed(3)
+
     if transform_name in ("randomnoise", "randomphaseerror"):
         # Random noise or phase error is not invertible
         return
  1. Verify that python -m pytest deepinv/tests/test_transform.py -k "test_transform_identity" now fails:
================================================================================ short test summary info ================================================================================
FAILED deepinv/tests/test_transform.py::test_transform_identity[device0-True-homography] - AssertionError: assert False
FAILED deepinv/tests/test_transform.py::test_transform_identity[device0-True-affine] - AssertionError: assert False
FAILED deepinv/tests/test_transform.py::test_transform_identity[device0-False-homography] - AssertionError: assert False
FAILED deepinv/tests/test_transform.py::test_transform_identity[device0-False-affine] - AssertionError: assert False
FAILED deepinv/tests/test_transform.py::test_transform_identity[device1-True-homography] - AssertionError: assert False
FAILED deepinv/tests/test_transform.py::test_transform_identity[device1-True-affine] - AssertionError: assert False
FAILED deepinv/tests/test_transform.py::test_transform_identity[device1-False-homography] - AssertionError: assert False
FAILED deepinv/tests/test_transform.py::test_transform_identity[device1-False-affine] - AssertionError: assert False
=============================================================== 8 failed, 92 passed, 104 deselected, 41 warnings in 9.54s ===============================================================

Steps to reproduce

/

Expected behavior

/

Full traceback (if applicable)

No response

Additional context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: highNeeds prompt attention from contributors or maintainers.type: bugSomething isn't working

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions