| Issue |
244178
|
| Summary |
[LoopVectorize] Scalarized predicated instruction feeding a widened GEP keeps only lane 0
|
| Labels |
new issue
|
| Assignees |
|
| Reporter |
maleadt
|
When a predicated instruction is scalarized and its result is used as a vector by a widened GEP, only lane 0's result is kept. Every lane of the gather then uses lane 0's index.
Reproducer (`opt -mcpu=skylake-avx512 -passes=loop-vectorize`, on main and on 23.1.x):
```llvm
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"
define void @f(ptr %out, ptr %xs, i64 %n, i64 %len) {
entry:
br label %loop
loop:
%i = phi i64 [ 0, %entry ], [ %i.next, %latch ]
%i1 = add i64 %i, 1
%c = icmp sgt i64 %i1, %len
br i1 %c, label %latch, label %body, !prof !3
body:
%d = sdiv i64 %i, %n
%p = getelementptr inbounds float, ptr %xs, i64 %d
%v = load float, ptr %p, align 4, !llvm.access.group !2
%q = getelementptr inbounds float, ptr %out, i64 %i
store float %v, ptr %q, align 4, !llvm.access.group !2
br label %latch
latch:
%i.next = add nuw nsw i64 %i, 1
%e = icmp eq i64 %i.next, 32
br i1 %e, label %exit, label %loop, !llvm.loop !0
exit:
ret void
}
!0 = distinct !{!0, !1}
!1 = !{!"llvm.loop.parallel_accesses", !2}
!2 = distinct !{}
!3 = !{!"branch_weights", i32 1000, i32 1}
```
The vectorized loop computes one `sdiv` per vector iteration instead of eight. Calling `@f(out, xs, 2, 20)` with `xs[i] = i` gives:
```
0 0 0 0 0 0 0 0 4 4 4 4 4 4 4 4 8 8 8 8
```
The expected output is `0 0 1 1 2 2 …`. `udiv`, `srem` and `urem` fail the same way.
The branch weights make the division scalarized rather than lowered to `llvm.masked.sdiv`. Without them, main masks it and the result is correct; 23.x scalarizes it either way.
**Cause.** `dissolveReplicateRegion` (`VPlanUnroll.cpp`) only packs a predicated phi's lanes into a vector if the phi has users that need more than the first lane. It asks this after `convertRecipesInRegionBlocksToSingleScalar` has already replaced the `VPPredInstPHIRecipe` with a scalar `VPPhi`. `VPWidenGEPRecipe::usesFirstLaneOnly` returns `isSingleScalar(Op)`, which is true for that scalar phi. So no `BuildVector` is created, and lanes 1–7 are dropped.
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs