Issue 242633
Summary [APFloat] opUnderflow is not returned when a tiny result rounds up to the smallest normal number
Labels new issue
Assignees
Reporter lmtlssss
    `IEEEFloat::normalize` returns `opInexact` alone when a result below the smallest normal number is rounded up to it. `APFloat.h` says "underflow tininess is detected after rounding". By IEEE 754-2019 7.5 a result is then tiny when, rounded to the precision of the format as though the exponent range were unbounded, it lies strictly between ±2^emin, and a tiny result that is inexact raises underflow.

```c++
APFloat P(0x1.fffffffffffffp-1); // 1 - 2^-53
APFloat::opStatus St =
    P.multiply(APFloat(0x1p-1022), APFloat::rmNearestTiesToEven);
// P is 0x1p-1022, St is opInexact. Expected opUnderflow | opInexact.
```

The exact product is `0x1.fffffffffffffp-1023`. It fits in 53 bits, so rounding it with an unbounded exponent leaves it below 2^-1022. An x86-64 processor (tininess after rounding) and an AArch64 one (before rounding) both raise underflow for this multiplication.

The cause is the last test of `normalize` (`APFloat.cpp:1773` at 11a7e80b89a8):

```c++
  /* The normal case - we were and are not denormal, and any
     significand increment above didn't overflow.  */
  if (omsb == semantics->precision)
    return opInexact;
```

A denormal that the increment has just carried up to the smallest normal number arrives here too. It is "not tiny" only when the rounding to full precision reaches 2^emin as well, as for `0x1.fffffffffffff8p-1023` converted to binary64.

`multiply`, `divide`, `fusedMultiplyAdd`, `convert` and `convertFromString` show it, in every format. Over every operand pair of the twelve 8-, 6- and 4-bit formats in the five rounding modes, on main, 12,660 products, 11,044 quotients and 8,149 conversions return `opInexact` where underflow is due. The values are right.

I have a patch with a unit test, 42 lines added to `APFloat.cpp` and 21 removed:
https://github.com/lmtlssss/speedrunpapi/blob/main/hardware/audit/apfloat/underflow.patch

With it the statuses of that sweep agree with the rule (the overflow flag of #220018 aside), a line-by-line comparison with main shows no other change in 23,687,600 results, and the tests of `APFloatTest.cpp` pass. It does not change `convertFromString` on a decimal string: `roundSignificandWithExponent` cuts the denormal down before `normalize` sees it. I can open a PR if this is wanted.

The sweep and the exact reference it is compared against:
https://github.com/lmtlssss/speedrunpapi/tree/main/hardware/audit/apfloat

The harness and the patch were written with an AI model, and I reviewed this report before posting it.

_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to