[Bug tree-optimization/119181] Missed vectorization due to imperfect SLP discovery for 2 grouped load with same base pointer (taken as 1 interleaved load)
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119181 Hongtao Liu changed: What|Removed |Added Resolution|--- |FIXED Status|NEW |RESOLVED Known to work||16.0 --- Comment #14 from Hongtao Liu --- Fixed in GCC16.
[Bug tree-optimization/119181] Missed vectorization due to imperfect SLP discovery for 2 grouped load with same base pointer (taken as 1 interleaved load)
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119181
--- Comment #13 from GCC Commits ---
The master branch has been updated by hongtao Liu :
https://gcc.gnu.org/g:1bc5b47f5b06dc4e8d2e7b622a7100b40b8e6b27
commit r16-924-g1bc5b47f5b06dc4e8d2e7b622a7100b40b8e6b27
Author: liuhongt
Date: Tue Mar 11 18:40:07 2025 -0700
For datarefs with big gap, split them into different groups.
The patch tries to solve miss vectorization for below case.
void
foo (int* a, int* restrict b)
{
b[0] = a[0] * a[64];
b[1] = a[65] * a[1];
b[2] = a[2] * a[66];
b[3] = a[67] * a[3];
b[4] = a[68] * a[4];
b[5] = a[69] * a[5];
b[6] = a[6] * a[70];
b[7] = a[7] * a[71];
}
In vect_analyze_data_ref_accesses, a[0], a[1], .. a[7], a[64], ...,
a[71] are in same group with size of 71. It caused vectorization
unprofitable.
gcc/ChangeLog:
PR tree-optimization/119181
* tree-vect-data-refs.cc (vect_analyze_data_ref_accesses):
Split datarefs when there's a gap bigger than
MAX_BITSIZE_MODE_ANY_MODE.
gcc/testsuite/ChangeLog:
* gcc.dg/vect/bb-slp-pr119181.c: New test.
[Bug tree-optimization/119181] Missed vectorization due to imperfect SLP discovery for 2 grouped load with same base pointer (taken as 1 interleaved load)
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119181
--- Comment #12 from Richard Biener ---
(In reply to Hongtao Liu from comment #10)
> But it still can't fix the issue with
>
> void
> foo (int* a, int* restrict b)
> {
> b[0] = a[0] * a[8];
> b[1] = a[1] * a[9];
> b[2] = a[2] * a[10];
> b[3] = a[11] * a[3];
> b[4] = a[12] * a[4];
> b[5] = a[5] * a[13];
> b[6] = a[6] * a[14];
> b[7] = a[7] * a[15];
> }
>
> -O2 -mavx2
>
> foo:
> vmovdqu ymm0, YMMWORD PTR [rdi]
> vmovdqu ymm2, YMMWORD PTR [rdi+32]
> vpblenddymm1, ymm2, ymm0, 231
> vpblenddymm0, ymm0, ymm2, 231
> vpmulld ymm0, ymm1, ymm0
> vmovdqu YMMWORD PTR [rsi], ymm0
> vzeroupper
> ret
>
> There's 2 redundant vpblendd here.
Yes, which is why I didn't try splitting groups - the most practical cases
will not have a large constant gap. Instead this asks for a optimization
phase on the SLP tree, possibly part of permute optimizations.
For vector code as in comment #11 this could be optimized by either a
match.pd pattern or by forwprop. Note it could be deeper in an
expression tree, like permute * (x + permute), where eliding two
permutes in exchange for an additional permute on 'x' might pay off,
this shouldn't be done with match.pd or simple pattern matching but
would ask for some kind of propagation pass (like we do in SLP permute
optimization).
[Bug tree-optimization/119181] Missed vectorization due to imperfect SLP discovery for 2 grouped load with same base pointer (taken as 1 interleaved load)
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119181
--- Comment #11 from Hongtao Liu ---
More common case is
typedef int v8si __attribute__((vector_size(32)));
v8si
foo1 (v8si a, v8si b)
{
v8si c = __builtin_shufflevector (a, b, 0, 1, 2, 11, 4, 5, 6, 15);
v8si d = __builtin_shufflevector (b, a, 0, 1, 2, 11, 4, 5, 6, 15);
return c * d;
}
Redudant vector permutation is not optimized off.
[Bug tree-optimization/119181] Missed vectorization due to imperfect SLP discovery for 2 grouped load with same base pointer (taken as 1 interleaved load)
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119181
--- Comment #10 from Hongtao Liu ---
But it still can't fix the issue with
void
foo (int* a, int* restrict b)
{
b[0] = a[0] * a[8];
b[1] = a[1] * a[9];
b[2] = a[2] * a[10];
b[3] = a[11] * a[3];
b[4] = a[12] * a[4];
b[5] = a[5] * a[13];
b[6] = a[6] * a[14];
b[7] = a[7] * a[15];
}
-O2 -mavx2
foo:
vmovdqu ymm0, YMMWORD PTR [rdi]
vmovdqu ymm2, YMMWORD PTR [rdi+32]
vpblenddymm1, ymm2, ymm0, 231
vpblenddymm0, ymm0, ymm2, 231
vpmulld ymm0, ymm1, ymm0
vmovdqu YMMWORD PTR [rsi], ymm0
vzeroupper
ret
There's 2 redundant vpblendd here.
[Bug tree-optimization/119181] Missed vectorization due to imperfect SLP discovery for 2 grouped load with same base pointer (taken as 1 interleaved load)
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119181 --- Comment #9 from rguenther at suse dot de --- On Tue, 11 Mar 2025, liuhongt at gcc dot gnu.org wrote: > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119181 > > --- Comment #8 from Hongtao Liu --- > (In reply to Richard Biener from comment #7) > > The issue is we detect this as a single interleaving group: > > > > t.c:12:1: note: Detected interleaving load of size 264 > > t.c:12:1: note: _1 = *a_26(D); > > t.c:12:1: note: _5 = MEM[(double *)a_26(D) + 8B]; > > t.c:12:1: note: _7 = MEM[(double *)a_26(D) + 16B]; > > t.c:12:1: note: _11 = MEM[(double *)a_26(D) + 24B]; > > t.c:12:1: note: _14 = MEM[(double *)a_26(D) + 32B]; > > t.c:12:1: note: _17 = MEM[(double *)a_26(D) + 40B]; > > t.c:12:1: note: _19 = MEM[(double *)a_26(D) + 48B]; > > t.c:12:1: note: _22 = MEM[(double *)a_26(D) + 56B]; > > t.c:12:1: note: > > t.c:12:1: note: _2 = MEM[(double *)a_26(D) + 2048B]; > > t.c:12:1: note: _4 = MEM[(double *)a_26(D) + 2056B]; > > t.c:12:1: note: _8 = MEM[(double *)a_26(D) + 2064B]; > > t.c:12:1: note: _10 = MEM[(double *)a_26(D) + 2072B]; > > t.c:12:1: note: _13 = MEM[(double *)a_26(D) + 2080B]; > > t.c:12:1: note: _16 = MEM[(double *)a_26(D) + 2088B]; > > t.c:12:1: note: _20 = MEM[(double *)a_26(D) + 2096B]; > > t.c:12:1: note: _23 = MEM[(double *)a_26(D) + 2104B]; > > > > so the heuristic to swap operands to get a single group in leafs doesn't > > work. Instead you get offsetting costs to avoid runaway with very large > > gaps: > Thanks for pointing this. > > > > *a_26(D) 132 times unaligned_load (misalign -1) costs 1584 in body > > > > and that makes it unprofitable. > > > > There is indeed some better heuristic needed where to split groups - gaps > > bigger than the biggest vector size might be a good candidate. Note > > when two different interleaving groups are used in the same SLP leaf > > we fail as we don't support that yet. > > A simple hack like below works, But I guess we may need better heuristic. Esp. since you are not supposed to get at a vector type - the dataref analysis is shared between the iteration through vector types. The heuristic should probably be based on MAX_BITSIZE_MODE_ANY_MODE, also instead of checking init_b - init_a I'd check init_b - init_prev, otherwise we risk breaking a contiguous set of DRs when the gap is placed oddly around MAX_BITSIZE_MODE_ANY_MODE.
[Bug tree-optimization/119181] Missed vectorization due to imperfect SLP discovery for 2 grouped load with same base pointer (taken as 1 interleaved load)
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119181
--- Comment #8 from Hongtao Liu ---
(In reply to Richard Biener from comment #7)
> The issue is we detect this as a single interleaving group:
>
> t.c:12:1: note: Detected interleaving load of size 264
> t.c:12:1: note: _1 = *a_26(D);
> t.c:12:1: note: _5 = MEM[(double *)a_26(D) + 8B];
> t.c:12:1: note: _7 = MEM[(double *)a_26(D) + 16B];
> t.c:12:1: note: _11 = MEM[(double *)a_26(D) + 24B];
> t.c:12:1: note: _14 = MEM[(double *)a_26(D) + 32B];
> t.c:12:1: note: _17 = MEM[(double *)a_26(D) + 40B];
> t.c:12:1: note: _19 = MEM[(double *)a_26(D) + 48B];
> t.c:12:1: note: _22 = MEM[(double *)a_26(D) + 56B];
> t.c:12:1: note:
> t.c:12:1: note: _2 = MEM[(double *)a_26(D) + 2048B];
> t.c:12:1: note: _4 = MEM[(double *)a_26(D) + 2056B];
> t.c:12:1: note: _8 = MEM[(double *)a_26(D) + 2064B];
> t.c:12:1: note: _10 = MEM[(double *)a_26(D) + 2072B];
> t.c:12:1: note: _13 = MEM[(double *)a_26(D) + 2080B];
> t.c:12:1: note: _16 = MEM[(double *)a_26(D) + 2088B];
> t.c:12:1: note: _20 = MEM[(double *)a_26(D) + 2096B];
> t.c:12:1: note: _23 = MEM[(double *)a_26(D) + 2104B];
>
> so the heuristic to swap operands to get a single group in leafs doesn't
> work. Instead you get offsetting costs to avoid runaway with very large
> gaps:
Thanks for pointing this.
>
> *a_26(D) 132 times unaligned_load (misalign -1) costs 1584 in body
>
> and that makes it unprofitable.
>
> There is indeed some better heuristic needed where to split groups - gaps
> bigger than the biggest vector size might be a good candidate. Note
> when two different interleaving groups are used in the same SLP leaf
> we fail as we don't support that yet.
A simple hack like below works, But I guess we may need better heuristic.
diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc
index c9395e33fcd..d9d55ff4a3e 100644
--- a/gcc/tree-vect-data-refs.cc
+++ b/gcc/tree-vect-data-refs.cc
@@ -3567,6 +3567,12 @@ vect_analyze_data_ref_accesses (vec_info *vinfo,
&& init_a <= init_prev
&& init_prev <= init_b);
+ tree vectype = get_vectype_for_scalar_type (vinfo, TREE_TYPE (DR_REF
(dra)));
+ unsigned HOST_WIDE_INT vf;
+ if (vectype
+ && TYPE_VECTOR_SUBPARTS (vectype).is_constant (&vf)
+ && (unsigned HOST_WIDE_INT)(init_b - init_a) > vf * tree_to_uhwi
(sza))
+ break;
/* Do not place the same access in the interleaving chain twice. */
if (init_b == init_prev)
{
[Bug tree-optimization/119181] Missed vectorization due to imperfect SLP discovery for 2 grouped load with same base pointer (taken as 1 interleaved load)
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119181 Richard Biener changed: What|Removed |Added Last reconfirmed||2025-03-10 Summary|Missed vectorization due to |Missed vectorization due to |imperfect SLP discovery for |imperfect SLP discovery for |2 grouped load with same|2 grouped load with same |base pointer(taken as 1 |base pointer (taken as 1 |interleaved load) |interleaved load) Status|UNCONFIRMED |NEW Ever confirmed|0 |1 --- Comment #7 from Richard Biener --- The issue is we detect this as a single interleaving group: t.c:12:1: note: Detected interleaving load of size 264 t.c:12:1: note: _1 = *a_26(D); t.c:12:1: note: _5 = MEM[(double *)a_26(D) + 8B]; t.c:12:1: note: _7 = MEM[(double *)a_26(D) + 16B]; t.c:12:1: note: _11 = MEM[(double *)a_26(D) + 24B]; t.c:12:1: note: _14 = MEM[(double *)a_26(D) + 32B]; t.c:12:1: note: _17 = MEM[(double *)a_26(D) + 40B]; t.c:12:1: note: _19 = MEM[(double *)a_26(D) + 48B]; t.c:12:1: note: _22 = MEM[(double *)a_26(D) + 56B]; t.c:12:1: note: t.c:12:1: note: _2 = MEM[(double *)a_26(D) + 2048B]; t.c:12:1: note: _4 = MEM[(double *)a_26(D) + 2056B]; t.c:12:1: note: _8 = MEM[(double *)a_26(D) + 2064B]; t.c:12:1: note: _10 = MEM[(double *)a_26(D) + 2072B]; t.c:12:1: note: _13 = MEM[(double *)a_26(D) + 2080B]; t.c:12:1: note: _16 = MEM[(double *)a_26(D) + 2088B]; t.c:12:1: note: _20 = MEM[(double *)a_26(D) + 2096B]; t.c:12:1: note: _23 = MEM[(double *)a_26(D) + 2104B]; so the heuristic to swap operands to get a single group in leafs doesn't work. Instead you get offsetting costs to avoid runaway with very large gaps: *a_26(D) 132 times unaligned_load (misalign -1) costs 1584 in body and that makes it unprofitable. There is indeed some better heuristic needed where to split groups - gaps bigger than the biggest vector size might be a good candidate. Note when two different interleaving groups are used in the same SLP leaf we fail as we don't support that yet.
