https://gcc.gnu.org/bugzilla/show_bug.cgi?id=127946
Bug ID: 127946
Summary: Missing destination-dependency breaking for an
all-ones-masked AVX-512 gather in a loop
Product: gcc
Version: 16.2.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: c++
Assignee: unassigned at gcc dot gnu.org
Reporter: z00823823 at outlook dot com
Target Milestone: ---
```
#include <immintrin.h>
#include <cstddef>
extern "C" __attribute__((noinline))
void gather_polynomial(const double* __restrict table,
const long long* __restrict indices,
double* __restrict output, std::size_t count) {
const __m512d a = _mm512_set1_pd(0.123);
const __m512d b = _mm512_set1_pd(0.456);
for (std::size_t i = 0; i < count; i += 8) {
const __m512i index = _mm512_loadu_si512(indices + i);
__m512d result = _mm512_i64gather_pd(index, table, 8);
for (int j = 0; j < 8; ++j)
result = _mm512_fmadd_pd(result, a, b);
_mm512_storeu_pd(output + i, result);
}
}
```
online version: https://godbolt.org/z/efE45aha1
compile with -O2 -march=tigerlake
```
.L4:
vmovdqu64 zmm3, ZMMWORD PTR [rsi+rdx*8]
kxnorb k2, k2, k2
mov eax, 8
vgatherqpd zmm0{k2}, [rdi+zmm3*8]
.L3:
vfmadd132pd zmm0, zmm1, zmm2
vfmadd132pd zmm0, zmm1, zmm2
sub eax, 2
jne .L3
vmovupd ZMMWORD PTR [r8+rdx*8], zmm0
add rdx, 8
cmp rdx, rcx
jb .L4
```
This example gather data then use result to do FMA, then write to result. GCC
reuse the gather destination register zmm0 between loops, but the
`vgatherqpd zmm0{k2}, [rdi+zmm3*8]`
actually cause a dependent on the zmm0 of previous iteraion: CPU do not take
advantage of the fact that k2 is all true, assume some lanes might use data
from previous iter's zmm0, and create an implicit dependent, prevents register
rename and ILP. there is a related discussion in llvm:
https://groups.google.com/g/llvm-dev/c/KiydZ5id0UI?pli=1
clang fixed this problem by zero out gather destination before gather:
```
vxorpd xmm3, xmm3, xmm3
vgatherqpd zmm3 {k1}, qword ptr [rdi + 8*zmm2]
```
