https://gcc.gnu.org/bugzilla/show_bug.cgi?id=128111
--- Comment #15 from GCC Commits <cvs-commit at gcc dot gnu.org> --- The master branch has been updated by Kyrylo Tkachov <[email protected]>: https://gcc.gnu.org/g:f819469be78f6e4b408e04ff35e8c5781ec416f4 commit r17-6104-gf819469be78f6e4b408e04ff35e8c5781ec416f4 Author: Kyrylo Tkachov <[email protected]> Date: Mon Nov 9 11:14:12 2026 +0100 aarch64: Use SVE2 PMULLB for clmul in streaming mode [PR128111] Advanced SIMD PMULL is not available in streaming mode, so the clmul expanders are limited to non-streaming mode and __builtin_clmulg falls back to a multiply-based sequence there. SVE2 has PMULLB, which is available in streaming mode. Its pair form writes the least and most significant halves of each product to adjacent elements, which is the shape of clmul_widen: PMULLB Zd.H, Zn.B, Zm.B 8x8->16 PMULLB Zd.D, Zn.S, Zm.S 32x32->64 PMULLB Zd.Q, Zn.D, Zm.D 64x64->128 (+sve-aes, or +ssve-aes in streaming mode) Add aarch64_expand_sve_clmul. For 8-bit and 32-bit inputs it uses a scalar pattern for PMULLB that names the overlapping scalar registers with the %Z modifier. The product is one element of twice the width, which is split with a shift. For 64-bit inputs it loads the operands into element 0 of zeroed vectors with FMOV, uses the pair form and moves element 1 to element 0 with UZP2. Use it for clmuldi3 and clmul_widendi3 when the Advanced SIMD PMULL is not usable but TARGET_SVE_AES is, for clmul_widenqi3 when TARGET_SIMD is not available, and for a new clmul_widensi3. The 32-bit pattern is also used in non-streaming mode with +sve2 and without +aes, where it replaces the multiply-based fallback. Advanced SIMD PMULL stays preferred when available. For example, in a streaming function: unsigned int f (unsigned int a, unsigned int b) { return __builtin_clmulg (a, b); } now generates: f: fmov s31, w0 fmov s30, w1 pmullb z30.d, z31.s, z30.s fmov w0, s30 ret instead of a 44 instruction sequence with 16 multiplies. Bootstrapped and tested on aarch64-linux-gnu. gcc/ChangeLog: PR middle-end/128111 * config/aarch64/aarch64-protos.h (aarch64_expand_sve_clmul): Declare. * config/aarch64/aarch64.cc (aarch64_expand_sve_clmul): New function. * config/aarch64/aarch64-sve2.md (aarch64_sve_pmullb_scalar<mode>): New pattern. * config/aarch64/iterators.md (SVE_PMULL_SCALAR, Vpmull_etype) (Vpmull_wetype): New iterator and attributes. * config/aarch64/aarch64-simd.md (clmuldi3, clmul_widendi3): Also enable for TARGET_SVE_AES and use aarch64_expand_sve_clmul when Advanced SIMD PMULL is not usable. (clmul_widensi3): New expander. (clmul_widenqi3): Also enable for TARGET_SVE2. gcc/testsuite/ChangeLog: PR middle-end/128111 * gcc.target/aarch64/sme/clmul_1.c: Replace. * gcc.target/aarch64/sme/clmul_2.c: New test. * gcc.target/aarch64/sve2/clmul_1.c: Likewise. Signed-off-by: Kyrylo Tkachov <[email protected]>
