On Wed, 11 Apr 2012 06:23:56 +0200, Sébastien Brisard wrote:
Hi,
Hi Sébastien,
I'm currently working on MATH-753, and the underlying accuracy
issues.
I therefore implemented the Gamma function for Dfps, and came accross
a few strange things. For example, althoug the javadoc states that
the
maximum exponent is 32768, there is no systematic check on this
limit.
I suppose overflow should happen in the following case
{code}
DfpField field = new DfpField(40);
final Dfp x = field.getOne().power10K(Integer.MAX_VALUE);
System.out.println(x);
System.out.println(x.isInfinite());
{code}
Instead, what happens is
{console}
1.000000000000000000000000000000000000e-4
false
{console}
The surprising answer comes from the fact that power10K actually
increments its integer argument. I do not think the above printout is
the expected behavior, am I wrong? I think this is potentially a vast
issue, because a quick check seems to indicate that "this.nans ==
INFINITE" is often checked in class Dfp, but I don't think the
statement "this.nans = INFINITE" ever occurs, which would mean that
overflow is never dealt with properly. Again, I'm probably wrong, I'd
like to have the feeling of someone more used to this class than I
am.
I obviously agree this result is wrong ...
Is William (the creator of both Dfp and FastMath) still around to deal
with this issue ?
Luc
Sébastien
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
