Skip to content

complexity guard: two maxGrowthRatio controls sit under the 1.5x margin convention on macos, and the linear control's 6.40x is unexplained #649

Description

@sectoramen

Two of the four maxGrowthRatio controls sit under this repo's own ≥1.5x margin convention on macos-latest. Neither is failing today; both are the next failure in the same class as #620's and #643's.

Measured, non-race step (the path that still asserts after #648)

test                                     platform  reading  bound            margin
TestGrowthRatioStaysLowOnLinearCode      macos      6.40x   must stay <=8.0  1.25x
TestGrowthRatioMissesASparseQuadratic    macos      6.04x   must stay <=8.0  1.32x
TestGrowthRatioCatchesAnEmitPerMatch     ubuntu    14.25x   must exceed 8.0  1.78x
TestGrowthRatioStillCatchesAGenuineQuad  macos     15.69x   must exceed 8.0  1.96x

The two "must stay low" controls are the thin ones, and the linear control is the outlier worth understanding: it reads 6.40x on macos-latest against ~4.0x everywhere else.

The linear control's 6.40x is not noise, and not a size effect

Its per-pair samples give the mechanism away:

macos-latest  6.40x  min base= 7.316ms  big=46.788ms  per-pair [3.87x 6.40x]
ubuntu-latest 3.99x  min base=11.147ms  big=44.508ms  per-pair [3.98x 3.99x]
darwin (dev)  3.97x  min base= 4.627ms  big=18.377ms  per-pair [4.00x 3.97x]

Reconstructing macos's two pairs from min base, min big and the pair ratios: the bases were ~12.5ms and 7.32ms — a 1.7x spread inside one run — while the two bigs were within 3% of each other. Ratio = min(big)/min(base) therefore divides the cleanest base into a big that both samples agree on, and reports 6.40x.

There is a second inconsistency in the same numbers. macos's base is 1.6x the dev machine's (7.32 vs 4.63ms), so a uniform slowdown predicts a big of ~29ms. It measured 46.8ms — 1.6x above prediction, on both samples. So the big readings look uniformly inflated on that runner, which a minimum over two samples cannot remove.

A size/memory threshold is refuted. detector.Match is 232 bytes and the big side emits 160,000 of them (~35 MB) with GC off, so memory pressure was the obvious candidate. Measured on darwin/arm64 across a size sweep:

reps=5000   3.09x   big ~4 MB      (base 989µs — too small to measure, noisy)
reps=10000  4.34x   big ~8 MB      (base 1.30ms — same)
reps=20000  3.91x   big ~17 MB
reps=40000  3.97x   big ~35 MB     <- the shipped size
reps=80000  4.03x   big ~70 MB

Flat at 3.91–4.03x from 20k to 80k reps. No upward trend, so the 4x-larger allocation is not what produces 6.40x.

Why "use more pairs" is not obviously the fix

Tempting, and it may be wrong. Ratio = min(big)/min(base), so adding samples shrinks both minima: a smaller min(base) pushes the ratio UP, a smaller min(big) pushes it DOWN. The net direction is not determined by the design.

Measured on darwin/arm64, worst of 4 trials at each pair count:

pairs=2   worst 4.61x   (margin 1.74x)
pairs=4   worst 4.13x   (margin 1.94x)
pairs=8   worst 4.39x   (margin 1.82x)

Four is tightest, eight is worse than four, and the spread between them is inside the run-to-run noise. On an idle 14-core machine that reproduces none of the problem, this says nothing about macos-latest.

What would settle it, and why I did not just fix it

The dev machine cannot reproduce the failure mode: it reads a stable ~4.0x where macos-latest reads 6.40x, is 3.6x faster on these fixtures, and has 14 cores against that runner's 3–4. Tuning a bound or a pair count against local numbers is what produced #620's and #643's failures in the first place, and then #648's second round.

The evidence that would settle it has to come from CI:

  1. Log the per-pair samples for both controls on all three runners in the non-race step at pairs=2 and pairs=4, and compare the base spread. If macos's base spread collapses at 4 pairs, more pairs is the fix and the direction question is answered empirically.
  2. If the base spread persists, the instability is in the runner rather than in the sample count, and the honest options are a same-run relative bound (linear control versus the quadratic control, which is scale-invariant) or a documented per-platform population.

Note that option 2's relative form is already available and already measured: the quadratic control read 15.69x in the same macos run, so quadratic/linear = 2.45x there against 3.95x on ubuntu. That statistic is machine-independent by construction, and pinning it is probably the real answer — but it needs its own margin study before it replaces a shipped bound.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions