Skip to content

fix: ExpHullAbrham drops significant figures for results below 1#429

Open
gaoflow wants to merge 1 commit into
shopspring:masterfrom
gaoflow:fix-exphullabrham-significant-figures
Open

fix: ExpHullAbrham drops significant figures for results below 1#429
gaoflow wants to merge 1 commit into
shopspring:masterfrom
gaoflow:fix-exphullabrham-significant-figures

Conversation

@gaoflow

@gaoflow gaoflow commented Jul 18, 2026

Copy link
Copy Markdown

ExpHullAbrham(overallPrecision) is documented to return overallPrecision
significant figures ("integer part + decimal part"), and its own doc example
confirms it: NewFromFloat(26.1).ExpHullAbrham(2) gives "220000000000", i.e.
2 significant figures rather than 2 decimal places. That contract holds for
positive arguments but breaks for negative ones:

decimal.RequireFromString("-50").ExpHullAbrham(10) // "0"            (true value 1.93e-22)
decimal.RequireFromString("-30").ExpHullAbrham(10) // "0"
decimal.RequireFromString("-20").ExpHullAbrham(10) // "0.0000000021" (2 sig figs, wanted 10)

All of these are well inside the function's own valid range (|x| < 23*overallPrecision).

The final rounding splits into two branches:

if resNumDigits > abs(res.exp) {                     // result >= 1
    roundDigits = int32(currentPrecision) - resNumDigits - res.exp
} else {                                             // result < 1
    roundDigits = int32(currentPrecision)            // decimal places, not sig figs
}

The else branch rounds to overallPrecision decimal places, discarding the
leading-zero positions of a sub-1 result. So the delivered precision decays as
the argument gets more negative (exp(-5) keeps 8 of 10 figures, exp(-20) keeps 2)
and reaches literal 0 once the leading-zero count exceeds overallPrecision.

The >=1 branch's formula currentPrecision - resNumDigits - res.exp is the
significant-figure-correct rounding position for both cases (the most
significant digit of res sits at power resNumDigits + res.exp - 1 regardless
of sign), so the branch is unnecessary and dropping it fixes the whole negative
half-line. Positive arguments and results in [0.1, 1) are unchanged.

Test changes: several negative rows of TestDecimal_ExpHullAbrham encoded the
truncated outputs (including exp(-50.1591).ExpHullAbrham(10) == "0"); I
corrected them to the full-precision values and added a significant-figure sweep
over x in {-2.5,-5,-10,-20,-30,-50} x prec in {10,20,30}, checked against
exp(x) computed to 40+ digits. go test ./... passes.

ExpHullAbrham is a niche entry point (ExpTaylor, whose precision is decimal
places by contract, is used for the large-precision/Pow paths and is left
untouched), but the significant-figure contract this function documents is
currently wrong for every negative argument below 0.1.

ExpHullAbrham is documented to return overallPrecision significant
figures, but its final rounding used a separate branch when the result
was < 1 and rounded to overallPrecision decimal places instead. That
discarded the leading-zero positions, so results below 0.1 lost
significant figures and collapsed to 0 once the leading-zero count
reached overallPrecision -- e.g. exp(-50).ExpHullAbrham(10) returned 0
(true value 1.93e-22).

The >=1 branch formula currentPrecision - resNumDigits - res.exp is the
significant-figure-correct rounding position in both cases, so drop the
branch. Positive arguments and results in [0.1,1) are unchanged.

Correct the negative rows of the ExpHullAbrham table (they encoded the
truncated values) and add a significant-figure sweep over the negative
half-line, checked against exp(x) to 40+ digits.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant