fix: ExpHullAbrham drops significant figures for results below 1#429
Open
gaoflow wants to merge 1 commit into
Open
fix: ExpHullAbrham drops significant figures for results below 1#429gaoflow wants to merge 1 commit into
gaoflow wants to merge 1 commit into
Conversation
ExpHullAbrham is documented to return overallPrecision significant figures, but its final rounding used a separate branch when the result was < 1 and rounded to overallPrecision decimal places instead. That discarded the leading-zero positions, so results below 0.1 lost significant figures and collapsed to 0 once the leading-zero count reached overallPrecision -- e.g. exp(-50).ExpHullAbrham(10) returned 0 (true value 1.93e-22). The >=1 branch formula currentPrecision - resNumDigits - res.exp is the significant-figure-correct rounding position in both cases, so drop the branch. Positive arguments and results in [0.1,1) are unchanged. Correct the negative rows of the ExpHullAbrham table (they encoded the truncated values) and add a significant-figure sweep over the negative half-line, checked against exp(x) to 40+ digits.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ExpHullAbrham(overallPrecision)is documented to returnoverallPrecisionsignificant figures ("integer part + decimal part"), and its own doc example
confirms it:
NewFromFloat(26.1).ExpHullAbrham(2)gives"220000000000", i.e.2 significant figures rather than 2 decimal places. That contract holds for
positive arguments but breaks for negative ones:
All of these are well inside the function's own valid range (
|x| < 23*overallPrecision).The final rounding splits into two branches:
The
elsebranch rounds tooverallPrecisiondecimal places, discarding theleading-zero positions of a sub-1 result. So the delivered precision decays as
the argument gets more negative (exp(-5) keeps 8 of 10 figures, exp(-20) keeps 2)
and reaches literal
0once the leading-zero count exceedsoverallPrecision.The
>=1branch's formulacurrentPrecision - resNumDigits - res.expis thesignificant-figure-correct rounding position for both cases (the most
significant digit of
ressits at powerresNumDigits + res.exp - 1regardlessof sign), so the branch is unnecessary and dropping it fixes the whole negative
half-line. Positive arguments and results in
[0.1, 1)are unchanged.Test changes: several negative rows of
TestDecimal_ExpHullAbrhamencoded thetruncated outputs (including
exp(-50.1591).ExpHullAbrham(10) == "0"); Icorrected them to the full-precision values and added a significant-figure sweep
over
x in {-2.5,-5,-10,-20,-30,-50}xprec in {10,20,30}, checked againstexp(x)computed to 40+ digits.go test ./...passes.ExpHullAbrhamis a niche entry point (ExpTaylor, whose precision is decimalplaces by contract, is used for the large-precision/
Powpaths and is leftuntouched), but the significant-figure contract this function documents is
currently wrong for every negative argument below 0.1.