Skip to content

SYNERGY dataset (Leenaars_2019) gives "group should not contain conflicting labels" error in simulation mode #2528

Description

@timovdk

Describe the bug
When setting up a simulation with the SYNERGY dataset Leenaars_2019, ASReview gives the following error. I highly suspect there are more SYNERGY datasets that have the same issue: for a duplicate (as defined by the new grouping system) there are conflicting labels. I propose that within a group we can simply take the True value and override the uploaded data (the title and abstract are the same, so the label should be too). But, in the export it should probably retain the original (False) label.

This will also be an issue for users uploading their own labelled data, as these can contain these mismatches on purpose: maybe they only include the journal publication, not the preprint (even though the title and abstract are the same).

Traceback (most recent call last):
  File "/Users/___/Documents/venvs/asreview-2/lib/python3.13/site-packages/asreview/webapp/_task_manager/task_manager.py", line 73, in run
    self.func(*self.args)
    ~~~~~~~~~^^^^^^^^^^^^
  File "/Users/___/Documents/venvs/asreview-2/lib/python3.13/site-packages/asreview/webapp/_tasks.py", line 31, in run_task
    run_simulation(project)
    ~~~~~~~~~~~~~~^^^^^^^^^
  File "/Users/___/Documents/venvs/asreview-2/lib/python3.13/site-packages/asreview/webapp/_tasks.py", line 104, in run_simulation
    sim = Simulate(
        project.db.input.get_df(),
    ...<3 lines>...
        groups=project.db.input.get_groups(),
    )
  File "/Users/___/Documents/venvs/asreview-2/lib/python3.13/site-packages/asreview/simulation/simulate.py", line 210, in __init__
    raise ValueError(
        f"Groups should not contain conflicting labels: {e}"
    ) from e
ValueError: Groups should not contain conflicting labels: Group 1648 contains conflicting labels.
ERROR:asreview.task_manager:Failed to train model for project 023aee4779074391b2c846514f2f1362

To Reproduce
Create a simulation with the SYNERGY dataset Leenaars_2019

Version information

  • OS: MacOS
  • Browser: Safari
  • ASReview version: 3.0.5

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions