https://gcc.gnu.org/bugzilla/show_bug.cgi?id=127773
Bug ID: 127773
Summary: [x86] vzeroupper missing before call to IPA-RA-known
non-AVX callee, causing SSE/AVX transition assists
Product: gcc
Version: 13.3.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: target
Assignee: unassigned at gcc dot gnu.org
Reporter: michiel.van.slobbe at gmail dot com
Target Milestone: ---
When an AVX2 function calls a function that GCC compiled without AVX, and
-fipa-ra knows the callee's register usage, GCC can emit the vzeroupper after
the call instead of before it. The callee then runs legacy SSE with dirty upper
YMM state. On Intel P-cores from Golden Cove on, that costs two SSE/AVX-mix
assists per call.
Testcase:
typedef char v32 __attribute__((vector_size(32)));
static __attribute__((noinline)) int f(void) {
int x;
__asm__ volatile("pxor %%xmm0, %%xmm0" : "=r"(x) :: "xmm0"); /* legacy SSE
*/
return x;
}
__attribute__((target("avx2"))) int g(v32 *p) { *p += *p; return f() + 1; }
int main(void) {
static v32 buf;
int sink = 0;
for (int i = 0; i < 100000000; i++) {
v32 *p = &buf;
__asm__ volatile("" : "+r"(p));
sink += g(p);
}
return sink == 42;
}
gcc -O2 -S -masm=intel pr.c, trunk (17.0.0 20261007), function g:
push rbp
vmovdqa ymm0, YMMWORD PTR [rdi]
vpaddb ymm0, ymm0, ymm0
mov rbp, rsp
and rsp, -32
vmovdqa YMMWORD PTR [rdi], ymm0
call f
add eax, 1
vzeroupper
leave
ret
With -fno-ipa-ra, and with Clang, vzeroupper comes before the call. Same code
with GCC 11 to 16.
gcc -O2 pr.c && perf stat -e task-clock,cpu_core/event=0xc1,umask=0x10/ taskset
-c 0 ./a.out
(100M calls, i7-12700H P-core; 0xc1/0x10 = ASSISTS.SSE_AVX_MIX):
build order in g time assists
gcc 15.2 -O2 call, vzeroupper 13478 ms 200.0M
gcc 16.0.1 -O2 call, vzeroupper 13370 ms 200.0M
gcc 15.2 -fno-ipa-ra vzeroupper, call 229 ms 0
clang 22 -O2 vzeroupper, call 243 ms 0
That is about 134 ns vs 2.3 ns per call.
It only happens when every path to the call has dirty upper state. If any path
reaches the call clean, vzeroupper is placed on the dirty edges.
This looks like the check added in r10-3477 (PR 91994 follow-up) in
ix86_avx_u128_mode_needed: if the callee ABI does not clobber all SSE registers
in V4DImode, the call needs AVX_U128_ANY instead of AVX_U128_CLEAN. With
IPA-RA, a non-AVX callee that touches only a few XMM registers takes this path,
even though nothing 256-bit is live across the call. PR 112891 (r14-6447) fixed
the state after such a call, which is why vzeroupper now appears after it, but
not the missing one before it.
Possible fix: return AVX_U128_CLEAN when no 256/512-bit value is live across
the call, or when the callee is known to be compiled without AVX.
Real-world impact: simdutf's AVX2/AVX-512 kernels call scalar tail routines in
the same TU. With GCC 14 to 16 at -O2/-O3, convert_utf8_to_utf16le on 80 bytes
of ASCII takes 111.8 ns instead of 16.1 ns (2 assists per call); Clang takes
10.0 ns. Workaround with explicit _mm256_zeroupper():
https://github.com/simdutf/simdutf/pull/1049
I did this investigation with Claude (an AI assistant). I reviewed the text and
checked the results.
