Issue 243130
Summary [SLPVectorizer] Copyable elements pad lanes with `fmul x, 1.0` under `denormal_fpenv(preservesign)`, flushing denormal lanes
Labels new issue
Assignees
Reporter fredrikkz
    With copyable elements (`-slp-copyable-elements`, on by default in LLVM 23), SLP vectorizes a group in which only some lanes are computed, padding the others with `fmul x, 1.0`. In a function with `denormal_fpenv(preservesign)`, that multiply is not an identity: a denormal lane becomes ±0. The source only copied that value; after SLP it is changed.

Repro (`opt -S -passes=slp-vectorizer`, LLVM 23.1.3):

```llvm
target datalayout = "e-m:w-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
target triple = "x86_64-w64-windows-gnu"

; Only lane 2 is multiplied; lanes 0, 1, 3 are just passed on.
define void @f(ptr %p, ptr %q, i8 %k) #0 {
  %a = load float, ptr %p, align 4
  %p1 = getelementptr i8, ptr %p, i64 4
  %b = load float, ptr %p1, align 4
  %p2 = getelementptr i8, ptr %p, i64 8
  %c = load float, ptr %p2, align 4
  %p3 = getelementptr i8, ptr %p, i64 12
  %d = load float, ptr %p3, align 4
  switch i8 %k, label %join [
    i8 3, label %mul
    i8 4, label %mul
  ]

mul:
  %c2 = fmul float %c, 2.000000e+00
  br label %join

join:
  %ra = phi float [ %a, %0 ], [ %a, %mul ]
  %rb = phi float [ %b, %0 ], [ %b, %mul ]
  %rc = phi float [ %c, %0 ], [ %c2, %mul ]
  %rd = phi float [ %d, %0 ], [ %d, %mul ]
  store float %ra, ptr %q, align 4
  %q1 = getelementptr i8, ptr %q, i64 4
  store float %rb, ptr %q1, align 4
  %q2 = getelementptr i8, ptr %q, i64 8
  store float %rc, ptr %q2, align 4
  %q3 = getelementptr i8, ptr %q, i64 12
  store float %rd, ptr %q3, align 4
  ret void
}

attributes #0 = { denormal_fpenv(preservesign) "target-cpu"="x86-64-v3" }
```

Output:

```llvm
%2 = fmul <4 x float> %1, <float 1.000000e+00, float 1.000000e+00, float 2.000000e+00, float 1.000000e+00>
```

Lanes 0, 1 and 3 used to be plain copies (load, phi, store), which preserve every bit. Now they go through `fmul`, which flushes denormal inputs to ±0 under the declared mode (x86 MXCSR DAZ). `-slp-copyable-elements=false` avoids it.

Impact: we hit this in real code. SROA had split a C union into `float`s, and one lane held an integer byte plus padding, which read as a denormal float. The process runs with FTZ/DAZ and is compiled with `-ffast-math -fdenormal-fp-math=preserve-sign`, so the vectorized copy zeroed that byte. That was found in a win64 UBSan build, where inlining and a by-value struct copy produced the shape above.

Possible fixes: when the function's denormal mode isn't IEEE, use no arithmetic identity for copyable float lanes (blend or shuffle the copied lanes in instead), or skip copyable elements for FP ops. An `fmul x, 1.0` identity also quiets signaling NaNs, so the same concern applies to IEEE mode when bits must be preserved.

Related: #231510, #231525.

_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to