Issue 243120
Summary [nsan] False positive warnings when a floating-point struct is passed by value (byval)
Labels new issue
Assignees
Reporter callelj
    # Description
With -fsanitize=numerical, an aggregate passed using LLVM's byval parameter attribute does not appear to get its shadow value copied to the callee. The callee can therefore observe stale shadow data associated with the stack memory used for the byval copy, causing a false positive.

Reproduced with x86_64 Linux:

Clang 20.1.7 locally
Clang 23.1.0 on Compiler Explorer
trunk cbfac21cd384 (24.0.0git) on Compiler Explorer

# Reproducer
With Clang 20.1.7, the issue reproduces at -O0, -O1, -O2, and -O3.

```c
typedef struct { double x, y, z; } S;

// volatile keeps the optimizer from removing the (otherwise dead) stores at -O1 and above
__attribute__((noinline)) void dirty(double v) { volatile double a[24]; for (int i = 0; i < 24; i++) a[i] = v; }
__attribute__((noinline)) double use(S a) { return a.x; }
__attribute__((noinline)) double test(void) { S a = {0.0, 0.0, 0.0}; return use(a); }

int main(void) { dirty(4.0); return test() != 0.0; }
```

Compiler Explorer (trunk, -O0, executed with `NSAN_OPTIONS=halt_on_error=0`): https://godbolt.org/z/5vqbTdf3Y

```
$ clang -O0 -fsanitize=numerical r.c -o r && NSAN_OPTIONS=halt_on_error=0 ./r
WARNING: NumericalStabilitySanitizer: inconsistent shadow results while checking return value
double       precision  (native): dec: 0.00000000000000000000  hex: 0x0.00000000000000000000p+0
__float128 precision  (shadow): dec: 4.00000000000000000000  hex: 0x8.00000000000000000000p-1
shadow truncated to double      : dec: 4.00000000000000000000  hex: 0x1.00000000000000000000p+2
Relative error: 100.00000000000000000000% (2^52 epsilons)
```

The program is correct: `use` returns 0.0, and every field of `a` is written before use. The shadow value 4.0 is what `dirty` left on the stack.

# Observations
- Passing `const S *` instead of `S` gives no warnings at -O0 through -O3.
- With a 16-byte struct (`double x, y`) there are no warnings. It is passed in two registers (`define double @use(double %0, double %1)`).
- The 24-byte struct is passed in memory (`define double @use(ptr noundef byval(%struct.S) align 8 %0)`).
- It also reproduces with `float` members (e.g. 6 x float, 24 bytes) and with a mixed struct (`double, int, double`). It needs the struct to be passed in memory (`byval`), not in registers. Not every `byval` struct warns: a 64-byte `double[8]` struct did not at -O0, and a `long double` struct did not at -O0 or -O2.
- Other memory-passed arguments (Clang 20.1.7, same `dirty`/`use`/`test` shape, one run per level):
  - Warns at -O0 and -O2: a union (`double[3]` / `float[6]`), a nested struct, a call through a function pointer, and a struct passed through `...` (`va_arg(ap, S)`).
  - Warns at -O2 only: `_Complex long double` (32 bytes, `byval`) and a 256-bit `double` vector (`byval`, built without AVX).
  - No warning: `_Complex double` and a 128-bit `double` vector (both passed in registers), a struct returned by value (`sret`), and a C++ class with a user-written copy constructor (built without the C++ standard library).
- Calling `use` directly from `main` instead of through `test` gives no warning. The `byval` copy is then placed above the stack region `dirty` wrote, so no stale shadow overlaps it (shadow dump at entry of `use`: none). Through `test` the copy lands inside that region and `use` sees `dirty`'s shadow (4.0). Checked at -O0.
- The warning count was the same on 5 runs at each optimization level.
- From reading `NumericalStabilitySanitizer.cpp`, `createShadowArguments` and `populateShadowStack` skip arguments whose type is not a float or float vector. I did not find handling of `byval` arguments.

The same missing shadow may also hide real errors. In one unreduced test at -O0, an accumulated float rounding error was reported when a 24-byte struct was passed by pointer but not when it was passed by value. I have not reduced that case or checked it at other optimization levels.

## Expected result
No warning, or shadow copied for `byval` arguments.

Assisted-by: Claude Code
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to