Issue 244208
Summary [VectorCombine] shuffle(fop(x)) -> fop(shuffle(x)) duplicates the FP op when the mask repeats a lane or the op has other uses
Labels new issue
Assignees
Reporter he-weiwen
    (The following analysis is heavily LLM assisted, I'm relatively confident that the phenomenon described is real, but less so on its severity and appropriate solution.)

Several VectorCombine folds rewrite `shuffle(fop(x))` into `fop(shuffle(x))`. In two situations the rewritten IR computes the same FP result more than once, where the original computed it once and copied it: when the shuffle mask uses the same lane of `fop` more than once (A below), and when `fop` has another user and is kept for it (B below). Per LangRef ([Behavior of Floating-Point NaN values](https://llvm.org/docs/LangRef.html#behavior-of-floating-point-nan-values)), each computation that returns a NaN may pick its sign and payload independently. So lanes (or uses) that were copies of one result, and therefore bitwise equal, may differ after the fold. That is not a refinement, and Alive2 reports a value mismatch. When the mask doesn't repeat a lane and `fop` has no other uses, the fold is fine.

@ParkHanbum found this in the review of #228472 ([Alive2 log](https://github.com/llvm/llvm-project/pull/228472#discussion_r4226574797)), where the new `MergePermutes` path in `foldShuffleOfBinops` does it. It already happens on main, in two ways:

**A. The shuffle mask repeats a lane.** Moving the shuffle above the operation computes the repeated lane once per output lane instead of once and copying it:

```llvm
define <4 x float> @a_shuffle_of_binops(<2 x float> %x, <2 x float> %z) {
  %l = fadd <2 x float> %x, splat (float 1.0)
  %r = fadd <2 x float> %z, splat (float 1.0)
  %s = shufflevector <2 x float> %l, <2 x float> %r, <4 x i32> <i32 0, i32 0, i32 2, i32 2>
  ret <4 x float> %s
}
; -passes=vector-combine:
; %1 = shufflevector <2 x float> %x, <2 x float> %z, <4 x i32> <i32 0, i32 0, i32 2, i32 2>
;   %s = fadd <4 x float> %1, splat (float 1.000000e+00)
```
Alive2 (`<2 x half>` version): https://alive2.llvm.org/ce/z/6oKb3L

**B. The original operation has another use.** It is kept for that use, and the folded code computes the same lanes again, so the two uses can see different NaNs:

```llvm
define <4 x float> @b_shuffle_of_binops(<2 x float> %x, <2 x float> %z) {
  %l = fadd <2 x float> %x, splat (float 1.0)
  %r = fadd <2 x float> %z, splat (float 2.0)
  call void @use2f(<2 x float> %l)
  %s = shufflevector <2 x float> %l, <2 x float> %r, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
  ret <4 x float> %s
}
; -passes=vector-combine -mtriple=x86_64--:
;   %l = fadd <2 x float> %x, splat (float 1.000000e+00)
;   %1 = shufflevector <2 x float> %x, <2 x float> %z, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
;   %s = fadd <4 x float> %1, <float 1.000000e+00, float 1.000000e+00, float 2.000000e+00, float 2.000000e+00>
```

Affected folds on main (2517f566169a), with `opt -passes=vector-combine` on the repros below:

| Fold | A: repeated lane | B: original kept for another use |
|---|---|---|
| `foldShuffleOfBinops` | SSE2, AVX2, AArch64, no triple | SSE2, AVX2, AArch64 |
| `foldPermuteOfBinops` | SSE2, AVX2, AArch64, no triple | no (requires one use) |
| `foldShuffleOfIntrinsics` | SSE2, AVX2 (fma; minnum also with no triple) | SSE2, AVX2, AArch64, no triple (sqrt) |
| `foldPermuteOfIntrinsic` | SSE2, AVX2, AArch64, no triple (sqrt) | not seen: the kept intrinsic's cost is added, and my test did not fold |
| `foldShuffleOfCastops` | SSE2, AVX2, AArch64, no triple (fpext) | AVX2, no triple (fpext) |
| `foldShuffleToIdentity` | SSE2, AVX2, AArch64, no triple (when the operands are splats) | no (requires one user) |

<details><summary>All repros</summary>

```llvm
declare void @use2f(<2 x float>)
declare void @use2d(<2 x double>)

; A: foldPermuteOfBinops
define <4 x float> @a_permute_of_binops(<4 x float> %a, <4 x float> %b) {
  %a0 = shufflevector <4 x float> %a, <4 x float> poison, <4 x i32> <i32 1, i32 0, i32 3, i32 2>
 %b0 = shufflevector <4 x float> %b, <4 x float> poison, <4 x i32> <i32 1, i32 0, i32 3, i32 2>
  %d = fdiv <4 x float> %a0, %b0
  %s = shufflevector <4 x float> %d, <4 x float> poison, <4 x i32> <i32 3, i32 3, i32 2, i32 2>
  ret <4 x float> %s
}

; A: foldShuffleOfIntrinsics
define <4 x float> @a_shuffle_of_intrinsics(<4 x float> %a, <4 x float> %b, <4 x float> %c, <4 x float> %d, <4 x float> %e, <4 x float> %f) {
  %l = call <4 x float> @llvm.fma.v4f32(<4 x float> %a, <4 x float> %b, <4 x float> %c)
  %r = call <4 x float> @llvm.fma.v4f32(<4 x float> %d, <4 x float> %e, <4 x float> %f)
 %s = shufflevector <4 x float> %l, <4 x float> %r, <4 x i32> <i32 0, i32 0, i32 4, i32 4>
  ret <4 x float> %s
}

; A: foldPermuteOfIntrinsic
define <4 x float> @a_permute_of_intrinsic(<4 x float> %a) {
  %f = call <4 x float> @llvm.sqrt.v4f32(<4 x float> %a)
  %s = shufflevector <4 x float> %f, <4 x float> poison, <4 x i32> <i32 1, i32 1, i32 0, i32 0>
  ret <4 x float> %s
}

; A: foldShuffleOfCastops
define <4 x double> @a_shuffle_of_castops(<4 x float> %x) {
  %e = fpext <4 x float> %x to <4 x double>
  %s = shufflevector <4 x double> %e, <4 x double> poison, <4 x i32> <i32 1, i32 1, i32 0, i32 0>
  ret <4 x double> %s
}

; A: foldShuffleToIdentity
define <4 x float> @a_shuffle_to_identity(<4 x float> %a) {
  %xs = shufflevector <4 x float> %a, <4 x float> poison, <4 x i32> zeroinitializer
  %m = fmul <4 x float> %xs, splat (float 3.0)
  %d = fadd <4 x float> %m, splat (float 1.0)
  %s = shufflevector <4 x float> %d, <4 x float> poison, <4 x i32> <i32 0, i32 0, i32 1, i32 1>
  ret <4 x float> %s
}

; B: foldShuffleOfIntrinsics
define <4 x float> @b_shuffle_of_intrinsics(<2 x float> %x, <2 x float> %z) {
  %l = call <2 x float> @llvm.sqrt.v2f32(<2 x float> %x)
  %r = call <2 x float> @llvm.sqrt.v2f32(<2 x float> %z)
  call void @use2f(<2 x float> %l)
  %s = shufflevector <2 x float> %l, <2 x float> %r, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
  ret <4 x float> %s
}

; B: foldShuffleOfCastops
define <4 x double> @b_shuffle_of_castops(<2 x float> %x, <2 x float> %y) {
  %ex = fpext <2 x float> %x to <2 x double>
  %ey = fpext <2 x float> %y to <2 x double>
  call void @use2d(<2 x double> %ex)
  %s = shufflevector <2 x double> %ex, <2 x double> %ey, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
  ret <4 x double> %s
}
```
</details>
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to