Issue 244597
Summary [SLPVectorizer] Live vector lanes silently turned into poison in processBuildVector when isGatherShuffledEntry collapses a multi-part gather
Labels new issue
Assignees alexey-bataev
Reporter alexfh
    ### Summary

`SLPVectorizer` silently turns live vector lanes into `poison` in an emitted `<16 x i32>` `shufflevector`, causing a silent miscompile.

- **Regressed in:** `63b4d988d9812c16ef319b67adfba551126f295d` (`[SLP]Fix undef/poison matching in TreeEntry::isSame`)
- **Last good tested:** `4717a10901d02b84ab65516a1ff79c9262215fa3`
- **Still reproduces on `main`:** `5492a3514d05aca14bbd32397f02a80fad001617`

### Reproducer

Run `opt -passes=slp-vectorizer -S repro.ll` on:

```llvm
target triple = "x86_64-unknown-linux-gnu"

define void @repro(ptr %p, ptr %out) #0 {
entry:
  %p1 = getelementptr inbounds i32, ptr %p, i64 1
  %p2 = getelementptr inbounds i32, ptr %p, i64 2
  %p3 = getelementptr inbounds i32, ptr %p, i64 3
  %a = load i32, ptr %p, align 4
  %b = load i32, ptr %p1, align 4
  %c = load i32, ptr %p2, align 4
  %d = load i32, ptr %p3, align 4

  %r0 = lshr i32 %b, 16
  %r1 = lshr i32 %b, 15
  %r2 = lshr i32 %a, 16
  %r3 = lshr i32 %a, 15
  %r4 = lshr i32 %d, 16
  %r5 = lshr i32 %d, 15
  %r6 = lshr i32 %c, 16
  %r7 = lshr i32 %c, 15

  %s0 = shl i32 %d, 1
  %s1 = shl i32 %c, 1
  %s2 = shl i32 %b, 1
  %s3 = shl i32 %a, 1

  %u0  = xor i32 %b, 1
  %v0  = add i32 %u0, %r0
  %t0  = trunc i32 %v0 to i8
  %u1  = xor i32 %b, 2
  %v1  = add i32 %u1, %r1
  %t1  = trunc i32 %v1 to i8
  %u2  = xor i32 %d, 3
  %v2  = add i32 %u2, %d
  %t2  = trunc i32 %v2 to i8
  %u3  = xor i32 %d, 4
  %v3  = add i32 %u3, %s0
  %t3  = trunc i32 %v3 to i8
  %u4  = xor i32 %a, 5
  %v4  = add i32 %u4, %r2
  %t4  = trunc i32 %v4 to i8
  %u5  = xor i32 %a, 6
  %v5  = add i32 %u5, %r3
  %t5  = trunc i32 %v5 to i8
  %u6  = xor i32 %c, 7
  %v6  = add i32 %u6, %c
  %t6  = trunc i32 %v6 to i8
  %u7  = xor i32 %c, 8
  %v7  = add i32 %u7, %s1
  %t7  = trunc i32 %v7 to i8
  %u8  = xor i32 %d, 9
  %v8  = add i32 %u8, %r4
  %t8  = trunc i32 %v8 to i8
  %u9  = xor i32 %d, 10
  %v9  = add i32 %u9, %r5
  %t9  = trunc i32 %v9 to i8
  %u10 = xor i32 %b, 11
  %v10 = add i32 %u10, %b
  %t10 = trunc i32 %v10 to i8
  %u11 = xor i32 %b, 12
  %v11 = add i32 %u11, %s2
  %t11 = trunc i32 %v11 to i8
  %u12 = xor i32 %c, 13
  %v12 = add i32 %u12, %r6
  %t12 = trunc i32 %v12 to i8
  %u13 = xor i32 %c, 14
  %v13 = add i32 %u13, %r7
  %t13 = trunc i32 %v13 to i8
  %u14 = xor i32 %a, 15
  %v14 = add i32 %u14, %a
  %t14 = trunc i32 %v14 to i8
  %u15 = xor i32 %a, 16
  %v15 = add i32 %u15, %s3
  %t15 = trunc i32 %v15 to i8

  %o1  = getelementptr inbounds i8, ptr %out, i64 1
  %o2  = getelementptr inbounds i8, ptr %out, i64 2
  %o3  = getelementptr inbounds i8, ptr %out, i64 3
  %o4  = getelementptr inbounds i8, ptr %out, i64 4
  %o5  = getelementptr inbounds i8, ptr %out, i64 5
  %o6  = getelementptr inbounds i8, ptr %out, i64 6
  %o7  = getelementptr inbounds i8, ptr %out, i64 7
  %o8  = getelementptr inbounds i8, ptr %out, i64 8
  %o9  = getelementptr inbounds i8, ptr %out, i64 9
  %o10 = getelementptr inbounds i8, ptr %out, i64 10
  %o11 = getelementptr inbounds i8, ptr %out, i64 11
  %o12 = getelementptr inbounds i8, ptr %out, i64 12
  %o13 = getelementptr inbounds i8, ptr %out, i64 13
  %o14 = getelementptr inbounds i8, ptr %out, i64 14
  %o15 = getelementptr inbounds i8, ptr %out, i64 15
  store i8 %t0,  ptr %out, align 1
  store i8 %t1,  ptr %o1,  align 1
  store i8 %t2,  ptr %o2,  align 1
  store i8 %t3,  ptr %o3,  align 1
  store i8 %t4,  ptr %o4,  align 1
  store i8 %t5,  ptr %o5,  align 1
  store i8 %t6,  ptr %o6,  align 1
  store i8 %t7,  ptr %o7,  align 1
  store i8 %t8,  ptr %o8,  align 1
  store i8 %t9,  ptr %o9,  align 1
  store i8 %t10, ptr %o10, align 1
  store i8 %t11, ptr %o11, align 1
  store i8 %t12, ptr %o12, align 1
  store i8 %t13, ptr %o13, align 1
  store i8 %t14, ptr %o14, align 1
  store i8 %t15, ptr %o15, align 1
  ret void
}

attributes #0 = { "target-features"="+avx" }
```

### Expected vs. Actual Output

**Expected** (before `63b4d988d981`):
```llvm
  %11 = shufflevector <16 x i32> %10, <16 x i32> %1, <16 x i32> <i32 0, i32 1, i32 18, i32 12, i32 2, i32 3, i32 22, i32 13, i32 4, i32 5, i32 26, i32 14, i32 6, i32 7, i32 30, i32 15>
  %12 = add <16 x i32> %6, %11
  %13 = trunc <16 x i32> %12 to <16 x i8>
  store <16 x i8> %13, ptr %out, align 1
```

**Actual** (`63b4d988d981` through `main`):
```llvm
  %11 = shufflevector <16 x i32> %10, <16 x i32> %1, <16 x i32> <i32 0, i32 1, i32 18, i32 12, i32 2, i32 3, i32 22, i32 13, i32 4, i32 5, i32 poison, i32 14, i32 6, i32 7, i32 poison, i32 15>
  %12 = add <16 x i32> %6, %11
  %13 = trunc <16 x i32> %12 to <16 x i8>
  store <16 x i8> %13, ptr %out, align 1
```

Lanes 10 and 14 (`i32 26` and `i32 30`, selecting `%1[10]` = `%b` and `%1[14]` = `%a`) are replaced with `poison`, causing `out[10]` and `out[14]` to store `poison` instead of `trunc((%b ^ 11) + %b)` and `trunc((%a ^ 15) + %a)`.

### Root Cause

In `llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp` (line numbers at `5492a3514d05`):

1. In `BoUpSLP::processBuildVector` (line 23801), for the 16-lane (`VF = 16`, `NumParts = 2`, `SliceSize = 8`) gather bundle `E`, `tryToGatherExtractElements` satisfies 12 lanes (`ExtractShuffles`) and marks them `poison` in `GatheredScalars`, leaving 4 live scalars at lanes 2, 6, 10, 14. `processBuildVector` passes `GatheredScalars` as `VL` to `isGatherShuffledEntry`.
2. In `BoUpSLP::isGatherShuffledEntry` (lines 22426–22442), on `Part = 0`, `isGatherShuffledSingleRegisterEntry` matches the 16-lane entry `%1` (which has `ReuseShuffleIndices` of length 16) with `SubEntries.size() == 1`, `*SubRes == TTI::SK_PermuteSingleSrc`, and `getVectorFactor() == VL.size()`.
3. Since `63b4d988d981`, `TreeEntry::isSame` (lines 2747–2757) returns `true` for any `PoisonValue` lane when `VL.size() == Mask.size()`. Because the 4 non-poison lanes of `VL` (`GatheredScalars`) match `%1` at indices 2, 6, 10, 14 and the other 12 lanes are `poison`, `SubEntries.front()->isSame(VL)` (line 22429) evaluates to `true`.
4. `isGatherShuffledEntry` therefore takes the early-return collapse (lines 22430–22441): it populates all 16 lanes of `Mask` (`Mask[2]=2, Mask[6]=6, Mask[10]=10, Mask[14]=14`, `PoisonMaskElem` elsewhere), **collapses `Entries` and `Res` to size 1** despite `NumParts == 2`, and returns early.
5. Back in `BoUpSLP::processBuildVector`:
   - The perfect-diamond fast path (lines 23833–23876) is **not** taken because it checks `isSame` against the full unmasked `E->Scalars`, which does not match `%1`.
   - Lines 23888–23891 clear **all** non-poison lanes of `Mask` (including lanes 10 and 14) from `GatheredScalars`.
   - The gather-shuffle emission loop (lines 24034–24071) computes `SliceSize = getPartNumElems(VF, NumParts)` (`= 8`) and iterates `for (const auto [I, TEs] : enumerate(Entries))`. Because `Entries.size() == 1`, only `I = 0` executes, slicing `Mask[0..7]` and **never visiting `Mask[8..15]`**. Lanes 10 and 14 are silently dropped as `poison`.

Note that the only other caller, `BoUpSLP::findReusedOrderedScalars` (lines 5964–5970), already has a special case for this (`if (GatherShuffles.size() == 1 && NumParts != 1) { ... PartSz = NumScalars; NumParts = 1; }`), while `processBuildVector` does not.

### Potential Fixes

1. In `BoUpSLP::processBuildVector` (lines 24034–24040), handle the collapsed full-width single-source case (mirroring `findReusedOrderedScalars` lines 5964–5970): when `Entries.size() == 1` with `Entries.front().size() == 1` and `Entries.front().front()->getVectorFactor() == Mask.size()` (or `GatherShuffles.size() == 1 && NumParts != 1`), set `SliceSize = Mask.size()` so the full `Mask` is emitted in one pass.
2. Or in `BoUpSLP::isGatherShuffledEntry` (lines 22426–22442), only collapse `Entries`/`Res` to size 1 when `SubEntries.front()->isSame(TE->Scalars)` (or populate all `NumParts` entries instead of collapsing to size 1 when only `isSame(VL)` holds).

(bug prepared using Gemini)
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to