[Bug middle-end/128111] Add support for __builtin_clmulg

[cvs-commit at gcc dot gnu.org via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=128111

--- Comment #14 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Kyrylo Tkachov <ktkac...@gcc.gnu.org>:

https://gcc.gnu.org/g:ee2f4aeb3c55f2603e41cf4772275660f987f046

commit r17-6103-gee2f4aeb3c55f2603e41cf4772275660f987f046
Author: Kyrylo Tkachov <ktkac...@nvidia.com>
Date:   Mon Nov 9 10:53:43 2026 +0100

    aarch64: Implement clmul and clmul_widen optabs [PR128111]

    __builtin_clmulg expands through the clmul and clmul_widen optabs, and
    without them it falls back to a multiply-based software sequence.
    AArch64 has carry-less multiplication in PMULL: the 64x64->128 form
    with +aes (FEAT_PMULL) and the 8x8->16 form in base AdvSIMD.

    Add clmuldi3 and clmul_widendi3 for TARGET_AES on top of
    aarch64_crypto_pmulldi.  The middle-end zero-extends narrower operands
    to DImode, so these also cover 16-bit and 32-bit clmul and the 32-bit
    widening form.  The high half is extracted as a lane of the V2DI view
    of the product.  A highpart subreg of the TImode product is spilled
    to the stack when it is the only half used.

    These are not available in streaming mode, so the expanders require
    TARGET_NON_STREAMING.

    Add clmul_widenqi3 for TARGET_SIMD using the 8x8->16 PMULL, so 8-bit
    clmul does not need +aes.

    For example, with +aes the low half of a 64x64->128 carry-less product
    (clmul) and both halves (clmul_widen):

      unsigned long long clmul (unsigned long long a, unsigned long long b)
      {
        return __builtin_clmulg (a, b);
      }

      unsigned long long clmul_widen (unsigned long long a, unsigned long long
b,
                                      unsigned long long *hi)
      {
        return __builtin_clmulg (a, b, hi);
      }

    generate:

      clmul:
            fmov    d31, x0
            fmov    d30, x1
            pmull   v30.1q, v31.1d, v30.1d
            fmov    x0, d30
            ret

      clmul_widen:
            fmov    d31, x0
            fmov    d30, x1
            pmull   v30.1q, v31.1d, v30.1d
            fmov    x0, d30
            st1     {v30.d}[1], [x2]
            ret

    and without +aes the 8-bit clmul_widen generates:

      clmul8_widen:
            dup     v31.8b, w0
            dup     v30.8b, w1
            pmull   v30.8h, v31.8b, v30.8b
            ushr    v31.2s, v30.2s, 8
            umov    w0, v30.b[0]
            str     b31, [x2]
            ret

    The runtime test clmul_4.c needs FEAT_PMULL, so it requires the new
    effective target aarch64_pmull_hw.

    Bootstrapped and tested on aarch64-linux-gnu.

    gcc/ChangeLog:

            PR middle-end/128111
            * config/aarch64/aarch64-simd.md (clmuldi3, clmul_widendi3): New
            expanders, enabled for TARGET_AES in non-streaming mode.
            (clmul_widenqi3): New expander.

    gcc/testsuite/ChangeLog:

            PR middle-end/128111
            * lib/target-supports.exp
(check_effective_target_aarch64_pmull_hw):
            New.
            * gcc.target/aarch64/clmul_1.c: New test.
            * gcc.target/aarch64/clmul_2.c: Likewise.
            * gcc.target/aarch64/clmul_3.c: Likewise.
            * gcc.target/aarch64/clmul_4.c: Likewise.
            * gcc.target/aarch64/clmul_5.c: Likewise.
            * gcc.target/aarch64/clmul_6.c: Likewise.
            * gcc.target/aarch64/sme/clmul_1.c: Likewise.

    Signed-off-by: Kyrylo Tkachov <ktkac...@nvidia.com>