Issue 244178
Summary [LoopVectorize] Scalarized predicated instruction feeding a widened GEP keeps only lane 0
Labels new issue
Assignees
Reporter maleadt
    When a predicated instruction is scalarized and its result is used as a vector by a widened GEP, only lane 0's result is kept. Every lane of the gather then uses lane 0's index.

Reproducer (`opt -mcpu=skylake-avx512 -passes=loop-vectorize`, on main and on 23.1.x):

```llvm
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

define void @f(ptr %out, ptr %xs, i64 %n, i64 %len) {
entry:
  br label %loop
loop:
  %i = phi i64 [ 0, %entry ], [ %i.next, %latch ]
  %i1 = add i64 %i, 1
  %c = icmp sgt i64 %i1, %len
  br i1 %c, label %latch, label %body, !prof !3
body:
  %d = sdiv i64 %i, %n
  %p = getelementptr inbounds float, ptr %xs, i64 %d
  %v = load float, ptr %p, align 4, !llvm.access.group !2
  %q = getelementptr inbounds float, ptr %out, i64 %i
  store float %v, ptr %q, align 4, !llvm.access.group !2
  br label %latch
latch:
  %i.next = add nuw nsw i64 %i, 1
  %e = icmp eq i64 %i.next, 32
  br i1 %e, label %exit, label %loop, !llvm.loop !0
exit:
  ret void
}
!0 = distinct !{!0, !1}
!1 = !{!"llvm.loop.parallel_accesses", !2}
!2 = distinct !{}
!3 = !{!"branch_weights", i32 1000, i32 1}
```

The vectorized loop computes one `sdiv` per vector iteration instead of eight. Calling `@f(out, xs, 2, 20)` with `xs[i] = i` gives:

```
0 0 0 0 0 0 0 0 4 4 4 4 4 4 4 4 8 8 8 8
```

The expected output is `0 0 1 1 2 2 …`. `udiv`, `srem` and `urem` fail the same way.

The branch weights make the division scalarized rather than lowered to `llvm.masked.sdiv`. Without them, main masks it and the result is correct; 23.x scalarizes it either way.

**Cause.** `dissolveReplicateRegion` (`VPlanUnroll.cpp`) only packs a predicated phi's lanes into a vector if the phi has users that need more than the first lane. It asks this after `convertRecipesInRegionBlocksToSingleScalar` has already replaced the `VPPredInstPHIRecipe` with a scalar `VPPhi`. `VPWidenGEPRecipe::usesFirstLaneOnly` returns `isSingleScalar(Op)`, which is true for that scalar phi. So no `BuildVector` is created, and lanes 1–7 are dropped.

_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to