Skip to content

Set up framebuffer testing #7

@rallyplot

Description

@rallyplot

At present, the main way to run the tests is locally in the GUI, checking the plots render correct as prompted. Obviously, this needs automating, and the best way to do this is to store the visually confirmed 'correct' framebuffer. Then regression tests can test the current framebuffer against these canonically correct version. The test is here.

The infrastructure for testing the CI by comparing framebuffer of known correct results vs test run exists, but is not currently used. The main issue will be 1) dealing with some stochasticity in the results across platform / display system 2) getting the headless CI to render the framebuffer in a reasonable way for testing.

Steps to complete are:

  1. get the CI to same some framebuffers from the lined script, dealing with the headless issue with a virtual display
  2. store as artefact and inspect locally, to see how they look. If they are okay, find a reasonable threshold by which to compare them to the locally generated framebuffers
  3. run full tests in CI

It would be cool to run this cross platform, but linux only would be sufficient as I fear this will be very difficult.

As part of this issue, tidy up the tests too.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions