https://gcc.gnu.org/bugzilla/show_bug.cgi?id=127923
Bug ID: 127923
Summary: x86 -mavx: if-converted `a < b ? c : d` loses
FE_INVALID on a quiet NaN (LT reversed to quiet UNGE,
emitted as vcmpnlt_uqsd)
Product: gcc
Version: 17.0
Status: UNCONFIRMED
Keywords: wrong-code
Severity: normal
Priority: P3
Component: target
Assignee: unassigned at gcc dot gnu.org
Reporter: kyuwoncho18 at gmail dot com
Target Milestone: ---
Target: x86_64-pc-linux-gnu
godbolt: https://godbolt.org/z/vY677T58f
GCC trunk drops the FE_INVALID that `*d < 0.5` must raise when `*d` is a quiet
NaN, at -O1 and above with -mavx:
```c
#include <fenv.h>
#include <stdio.h>
double arr[1];
__attribute__((noipa))
static void sel (double *d, double *e)
{
for (; d < e; ++d)
*d = (*d < 0.5) ? 0.7 : 0.0;
}
int main (void)
{
arr[0] = __builtin_nan ("");
feclearexcept (FE_ALL_EXCEPT);
sel (arr, arr + 1);
printf ("invalid=%d result=%g\n", fetestexcept (FE_INVALID) != 0, arr[0]);
return 0;
}
```
```console
$ gcc -O2 -mavx -frounding-math t.c -lm && ./a.out
invalid=0 result=0
$ gcc -O2 -mavx -frounding-math -fno-if-conversion t.c -lm && ./a.out
invalid=1 result=0
$ gcc -O0 -mavx -frounding-math t.c -lm && ./a.out
invalid=1 result=0
```
Expected `invalid=1 result=0`: `<` is a signaling comparison, so a quiet-NaN
operand raises FE_INVALID (C17 F.3, F.9.3; 7.12.14 footnote), and
`-frounding-math` with the default `-ftrapping-math` makes FENV_ACCESS "on"
(implement-c.texi). -O1 and -O3 fail too; without -mavx it passes. Clang prints
`invalid=1`.
-O0 emits the signaling `vcomisd`; -O2 emits `vcmpnlt_uqsd` (quiet). In ce1,
`noce_try_cmove_arith` reverses the condition through
`reversed_comparison_code`. For CCFPmode `REVERSIBLE_CC_MODE` is 1, so
`ix86_reverse_condition` returns `reverse_condition_maybe_unordered (LT)` =
quiet `UNGE` and never checks `flag_trapping_math`/`HONOR_NANS`.
`sse_comparison_operator` then accepts `unge` for `setcc_<mode>_sse`, and `'D'`
prints it as `nlt_uq`.
GCC 16.2 is correct: it already reverses LT to UNGE but still prints it as the
signaling `vcmpnltsd`. r17-4240 (PR127177) added the quiet `nlt_uq` printing
for UNGE under AVX, which exposed the pre-existing signaling-to-quiet reversal.
This is not the vector LTGT issue fixed for PR127649, and the PR122307 patch
only touches LTGT/UNEQ (in `reversed_comparison_code_parts` and
`emit_store_flag`); the generic `reversed_comparison_code_parts` cases are
bypassed here anyway because `REVERSIBLE_CC_MODE` makes the i386 hook answer
first.
Disclaimer: This test case was generated by a fuzzer, and the report was
written with LLM assistance.
