Skip to content

HybridMaxRandom/HybridMaxUncertainty ignore probability when given an integer random_state #2601

Description

@timovdk

Describe the bug
_mix_indices (asreview/models/queriers.py:38), used by both HybridMaxRandom.query() (line 187) and HybridMaxUncertainty.query() (line 220), calls check_random_state(random_state) inside its loop, once per item:

def _mix_indices(query_idx_1, query_idx_2, mix_probability=0.95, random_state=None):
    query_idx_mix = []
    i = 0
    j = 0
    while i < len(query_idx_1) and j < len(query_idx_2):
        if check_random_state(random_state).rand() < mix_probability:
            query_idx_mix.append(query_idx_1[i])
            i = i + 1
        else:
            query_idx_mix.append(query_idx_2[j])
            j = j + 1
    ...

When random_state is a plain int (as the HybridMaxRandom docstrings allow: random_state: int, RandomState), check_random_state(int) constructs a brand-new np.random.RandomState(int) on every call. This means that .rand() returns the exact same value every single iteration, for the entire query batch. This means that the "probabilistic" branch is decided once, not per item: either every item comes from query_idx_1 (Max) or every item comes from query_idx_2 (Random/Uncertainty).

To Reproduce
Minimal example:

import numpy as np
from sklearn.utils import check_random_state

# Same "random" draw every time an int seed is (re)wrapped — this is the root cause.
for _ in range(5):
    print(check_random_state(0).rand())
# 0.5488135039273248
# 0.5488135039273248
# 0.5488135039273248
# 0.5488135039273248
# 0.5488135039273248

# Compare to the correct behavior when passing an actual RandomState instance:
rng = np.random.RandomState(0)
for _ in range(5):
    print(check_random_state(rng).rand())
# 0.5488135039273248
# 0.7151893663724195
# 0.6027633760716439
# 0.5448831829968969
# 0.4236547993389047

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions