Skip to content

[DQ Threshold W2.2] Sampling/partition provenance + rewritten result messages #33234

Description

@TeddyCr

D3: under sampling the threshold is evaluated on the sample, and that is surfaced loudly — no extrapolation, no blocking.

Provenance — reuse the existing mechanism, do not invent one

An earlier draft proposed an EvaluationScope value object. Not needed: RuntimeParameterSetter + get_runtime_parameters() (base_test_handler.py:708) already injects a typed model into a validator, and tableDiff already uses it to reach the table's profiler config (table_diff_params_setter.py:139models.py:49).

Add a setter carrying the resolved sample config + partition details and render it into the message.

Messages must carry the full picture

Today's "Found nullCount=3. It should be 0" does not say what it was measured against.

Found 120 null rows out of 9,981 evaluated (1.20%). Threshold is 1.00%, so this test failed.
Evaluated on a 10% sample of the table, partitioned on event_date >= 2026-09-10.
Mean of `amount` is 87.4. Expected between 90 and 110, widened by a 5% tolerance to 85.5-115.5,
so this test passed. Evaluated on the full table.

Always state the threshold actually applied — an older ingestion agent against a newer server ignores the parameter, and the message is the only way that becomes visible rather than inferred.

Notes

  • PERCENTAGE is sample-stable; ABSOLUTE is not (100 nulls in a 10% sample implies ~1,000).
  • Statistical tests: location statistics (mean, median) survive sampling; MIN/MAX are biased inward and SUM scales with the sample — a deviation threshold on columnValuesSumToBeBetween under sampling is close to meaningless.
  • Rule-library SQL bypasses the sampler; say so.
  • These strings are English-only free text in TestCaseResult.result. The UI must re-derive its sentence from the parameters, never parse this string.

Acceptance

  • Message names violation count, evaluated population, threshold, verdict, and sampling/partition scope
  • Same table with and without a sample: PERCENTAGE agrees within sampling error, ABSOLUTE differs, both messages state the population

Depends on the row-tolerance and deviation sub-tasks.

Design spec: docs/plans/2026-09-11-dq-failure-thresholds.md. Parent epic: #16301.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

  • Status
    Release Backlog 🚧

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions