On Sat, Jun 8, 2024 at 3:41 PM Richard Sandiford <richard.sandif...@arm.com> wrote:
> Mariam Arutunian <mariamarutun...@gmail.com> writes: > > This patch introduces two new expanders for the aarch64 backend, > > dedicated to generate optimized code for CRC computations. > > The new expanders are designed to leverage specific hardware capabilities > > to achieve faster CRC calculations, > > particularly using the pmul or crc32 instructions when supported by the > > target architecture. > > Thanks for porting this to aarch64! > > > Expander 1: Bit-Forward CRC (crc<ALLI:mode><ALLX:mode>4) > > For targets that support pmul instruction (TARGET_AES), > > the expander will generate code that uses the pmul (crypto_pmulldi) > > instruction for CRC computation. > > > > Expander 2: Bit-Reversed CRC (crc_rev<ALLI:mode><ALLX:mode>4) > > The expander first checks if the target supports the CRC32 instruction > set > > (TARGET_CRC32) > > and the polynomial in use is 0x1EDC6F41 (iSCSI). If the conditions are > met, > > it emits calls to the corresponding crc32 instruction (crc32b, crc32h, > > crc32w, or crc32x depending on the data size). > > If the target does not support crc32 but supports pmul, it then uses the > > pmul (crypto_pmulldi) instruction for bit-reversed CRC computation. > > > > Otherwise table-based CRC is generated. > > > > gcc/config/aarch64/ > > > > * aarch64-protos.h (aarch64_expand_crc_using_clmul): New extern > > function declaration. > > (aarch64_expand_reversed_crc_using_clmul): Likewise. > > * aarch64.cc (aarch64_expand_crc_using_clmul): New function. > > (aarch64_expand_reversed_crc_using_clmul): Likewise. > > * aarch64.md (UNSPEC_CRC, UNSPEC_CRC_REV): New unspecs. > > (crc_rev<ALLI:mode><ALLX:mode>4): New expander for reversed CRC. > > (crc<ALLI:mode><ALLX:mode>4): New expander for reversed CRC. > > * iterators.md (crc_data_type): New mode attribute. > > > > gcc/testsuite/gcc.target/aarch64/ > > > > * crc-1-pmul.c: Likewise. > > * crc-10-pmul.c: Likewise. > > * crc-12-pmul.c: Likewise. > > * crc-13-pmul.c: Likewise. > > * crc-14-pmul.c: Likewise. > > * crc-17-pmul.c: Likewise. > > * crc-18-pmul.c: Likewise. > > * crc-21-pmul.c: Likewise. > > * crc-22-pmul.c: Likewise. > > * crc-23-pmul.c: Likewise. > > * crc-4-pmul.c: Likewise. > > * crc-5-pmul.c: Likewise. > > * crc-6-pmul.c: Likewise. > > * crc-7-pmul.c: Likewise. > > * crc-8-pmul.c: Likewise. > > * crc-9-pmul.c: Likewise. > > * crc-CCIT-data16-pmul.c: Likewise. > > * crc-CCIT-data8-pmul.c: Likewise. > > * crc-coremark-16bitdata-pmul.c: Likewise. > > * crc-crc32-data16.c: New test. > > * crc-crc32-data32.c: Likewise. > > * crc-crc32-data8.c: Likewise. > > > > Signed-off-by: Mariam Arutunian <mariamarutun...@gmail.com > > diff --git a/gcc/config/aarch64/aarch64-protos.h > b/gcc/config/aarch64/aarch64-protos.h > > index 1d3f94c813e..167e1140f0d 100644 > > --- a/gcc/config/aarch64/aarch64-protos.h > > +++ b/gcc/config/aarch64/aarch64-protos.h > > @@ -1117,5 +1117,8 @@ extern void mingw_pe_encode_section_info (tree, > rtx, int); > > > > bool aarch64_optimize_mode_switching (aarch64_mode_entity); > > void aarch64_restore_za (rtx); > > +void aarch64_expand_crc_using_clmul (rtx *); > > +void aarch64_expand_reversed_crc_using_clmul (rtx *); > > + > > > > #endif /* GCC_AARCH64_PROTOS_H */ > > diff --git a/gcc/config/aarch64/aarch64.cc > b/gcc/config/aarch64/aarch64.cc > > index ee12d8897a8..05cd0296d38 100644 > > --- a/gcc/config/aarch64/aarch64.cc > > +++ b/gcc/config/aarch64/aarch64.cc > > @@ -30265,6 +30265,135 @@ aarch64_retrieve_sysreg (const char *regname, > bool write_p, bool is128op) > > return sysreg->encoding; > > } > > > > +/* Generate assembly to calculate CRC > > + using carry-less multiplication instruction. > > + OPERANDS[1] is input CRC, > > + OPERANDS[2] is data (message), > > + OPERANDS[3] is the polynomial without the leading 1. */ > > + > > +void > > +aarch64_expand_crc_using_clmul (rtx *operands) > > This should probably be pmul rather than clmul. > > > +{ > > + /* Check and keep arguments. */ > > + gcc_assert (!CONST_INT_P (operands[0])); > > + gcc_assert (CONST_INT_P (operands[3])); > > + rtx crc = operands[1]; > > + rtx data = operands[2]; > > + rtx polynomial = operands[3]; > > + > > + unsigned HOST_WIDE_INT > > + crc_size = GET_MODE_BITSIZE (GET_MODE (operands[0])).to_constant > (); > > + gcc_assert (crc_size <= 32); > > + unsigned HOST_WIDE_INT > > + data_size = GET_MODE_BITSIZE (GET_MODE (data)).to_constant (); > > We could instead make the interface: > > void > aarch64_expand_crc_using_pmul (scalar_mode crc_mode, scalar_mode data_mode, > rtx *operands) > > so that the lines above don't need the to_constant. This should "just > work" on the .md file side, since the modes being passed are naturally > scalar_mode. > > I think it'd be worth asserting also that data_size <= crc_size. > (Although we could handle any MAX (data_size, crc_size) <= 32 > with some adjustment.) > > > + > > + /* Calculate the quotient. */ > > + unsigned HOST_WIDE_INT > > + q = gf2n_poly_long_div_quotient (UINTVAL (polynomial), crc_size + > 1); > > + > > + /* CRC calculation's main part. */ > > + if (crc_size > data_size) > > + crc = expand_shift (RSHIFT_EXPR, DImode, crc, crc_size - data_size, > > + NULL_RTX, 1); > > + > > + rtx t0 = gen_reg_rtx (DImode); > > + aarch64_emit_move (t0, gen_int_mode (q, DImode)); > > It's only a minor simplification, but this could instead be: > > rtx t0 = force_reg (DImode, gen_int_mode (q, DImode)); > > > + rtx t1 = gen_reg_rtx (DImode); > > + aarch64_emit_move (t1, polynomial); > > If polynomial is a constant operand of mode crc_mode, GCC's standard > CONST_INT representation is to sign-extend the constant to 64 bits. > E.g. a QImode value of 0b1000_0000 would be represented as -128. > > I think here we want the zero-extended form, so it might be safer to do: > > polynomial = simplify_gen_unary (ZERO_EXTEND, DImode, polynomial, > crc_mode); > rtx t1 = force_reg (DImode, polynomial); > > > + > > + rtx a0 = expand_binop (DImode, xor_optab, crc, data, NULL_RTX, 1, > > + OPTAB_WIDEN); > > + > > + rtx clmul_res = gen_reg_rtx (TImode); > > + emit_insn (gen_aarch64_crypto_pmulldi (clmul_res, a0, t0)); > > + a0 = gen_lowpart (DImode, clmul_res); > > + > > + a0 = expand_shift (RSHIFT_EXPR, DImode, a0, crc_size, NULL_RTX, 1); > > + > > + emit_insn (gen_aarch64_crypto_pmulldi (clmul_res, a0, t1)); > > + a0 = gen_lowpart (DImode, clmul_res); > > + > > + if (crc_size > data_size) > > + { > > + rtx crc_part = expand_shift (LSHIFT_EXPR, DImode, operands[1], > data_size, > > + NULL_RTX, 0); > > + a0 = expand_binop (DImode, xor_optab, a0, crc_part, NULL_RTX, 1, > > + OPTAB_DIRECT); > > Formatting nit: extra space after "a0 = " > > > + } > > + /* Zero upper bits beyond crc_size. */ > > + rtx num_shift = gen_int_mode (64 - crc_size, DImode); > > + a0 = expand_shift (LSHIFT_EXPR, DImode, a0, 64 - crc_size, NULL_RTX, > 0); > > + a0 = expand_shift (RSHIFT_EXPR, DImode, a0, 64 - crc_size, NULL_RTX, > 1); > > Rather than shift left and then right, I think we should just AND: > > rtx mask = gen_int_mode (GET_MODE_MASK (crc_mode), DImode); > a0 = expand_binop (DImode, and_optab, a0, mask, NULL_RTX, 1, > OPTAB_DIRECT); > > That said, it looks like operands[0] has crc_mode. The register bits > above crc_size therefore shouldn't matter, since they're undefined on read. > E.g. even though (reg:SI R) is stored in an X register, only the low 32 > bits are defined; the upper 32 bits can be any value. > > So I'd expect we could replace this and... > > > + > > + rtx tgt = simplify_gen_subreg (DImode, operands[0], > > + GET_MODE (operands[0]), 0); > > + aarch64_emit_move (tgt, a0); > > ...this with just: > > aarch64_emitmove (operands[0], gen_lowpart (crc_mode, a0)); > > Perhaps that would break down if operands[0] is a subreg with > SUBREG_PROMOTED_VAR_P set, but I think it's up to target-independent > code to handle that case. > > > @@ -4543,6 +4545,63 @@ > > [(set_attr "type" "crc")] > > ) > > > > +;; Reversed CRC > > +(define_expand "crc_rev<ALLI:mode><ALLX:mode>4" > > + ;; return value (calculated CRC) > > + [(set (match_operand:ALLX 0 "register_operand" "=r") > > + ;; initial CRC > > + (unspec:ALLX [(match_operand:ALLX 1 "register_operand" "r") > > + ;; data > > + (match_operand:ALLI 2 "register_operand" "r") > > + ;; polynomial without leading 1 > > + (match_operand:ALLX 3)] > > + UNSPEC_CRC_REV))] > > Since we (rightly) never generate the RTL above, I think this can just be: > > (define_expand "crc_rev<ALLI:mode><ALLX:mode>4" > [;; return value (calculated CRC) > (match_operand:ALLX 0 "register_operand") > ;; initial CRC > (match_operand:ALLX 1 "register_operand") > ;; data > (match_operand:ALLI 2 "register_operand") > ;; polynomial without leading 1 > (match_operand:ALLX 3)] > > without the unspec and constraints. > > > + "" > > + { > > + /* If the polynomial is the same as the polynomial of crc32 > instruction, > > + put that instruction. crc32 uses iSCSI polynomial > (0x1EDC6F41). */ > > + if (TARGET_CRC32 && INTVAL (operands[3]) == 517762881) > > The hex constant feels a little easier to read. I think it'd also > be worth checking <ALLX:MODE>mode == SImode, even though it's currently > redundant (given that no other choice would allow that polynomial). > > > + { > > + rtx crc_result = gen_reg_rtx (SImode); > > + rtx crc = operands[1]; > > + rtx data = operands[2]; > > + emit_insn (gen_aarch64_crc32c<ALLI:crc_data_type> (crc_result, crc, > > + data)); > > + emit_move_insn (operands[0], > > + gen_lowpart (GET_MODE (operands[0]), crc_result)); > > If operands[0] has ALLX mode (== SImode), it looks like we should be > able to use operands[0] directly as the result of the CRC32C. > > FWIW, there's also CRC32 for the HDLC etc. polynomial 0x04C11DB7. > > > + } > > + else if (TARGET_AES) > > I think we also need to check <ALLI:sizen> <= <ALLX:sizen> for this. > Similarly for the unreversed CRC pattern. > > Thanks again for doing this. I realise RISC-V is the lead target for > this work, so you've gone above and beyond by doing a full AArch64 > port too. It'd be perfectly valid to ask Arm developers to deal > with the comments above, so please let me know if you'd prefer that. > The patch looks close to ready to me though. > Thanks for your suggestions and explanations, and thank you for recognizing my work. I'll resolve all the comments. Best regards, Mariam > Richard > > > + aarch64_expand_reversed_crc_using_clmul (operands); > > + else > > + { > > + /* Otherwise, generate table-based CRC. */ > > + expand_reversed_crc_table_based (operands[0], operands[1], > operands[2], > > + operands[3], GET_MODE > (operands[2]), > > + > generate_reflecting_code_standard); > > + } > > + DONE; > > + } > > +) > > + > > +;; Bit-forward CRC > > +(define_expand "crc<ALLI:mode><ALLX:mode>4" > > + ;; return value (calculated CRC) > > + [(set (match_operand:ALLX 0 "register_operand" "=r") > > + ;; initial CRC > > + (unspec:ALLX [(match_operand:ALLX 1 "register_operand" "r") > > + ; data > > + (match_operand:ALLI 2 "register_operand" "r") > > + ;; polynomial without leading 1 > > + (match_operand:ALLX 3)] > > + UNSPEC_CRC))] > > + "TARGET_AES" > > + { > > + aarch64_expand_crc_using_clmul (operands); > > + DONE; > > + } > > +) > > + > > + > > (define_insn "*csinc2<mode>_insn" > > [(set (match_operand:GPI 0 "register_operand" "=r") > > (plus:GPI (match_operand 2 "aarch64_comparison_operation" "") > > diff --git a/gcc/config/aarch64/iterators.md > b/gcc/config/aarch64/iterators.md > > index 99cde46f1ba..86e4863d684 100644 > > --- a/gcc/config/aarch64/iterators.md > > +++ b/gcc/config/aarch64/iterators.md > > @@ -1276,6 +1276,10 @@ > > ;; Map a mode to a specific constraint character. > > (define_mode_attr cmode [(QI "q") (HI "h") (SI "s") (DI "d")]) > > > > +;; Map a mode to a specific constraint character for calling > > +;; appropriate version of crc. > > +(define_mode_attr crc_data_type [(QI "b") (HI "h") (SI "w") (DI "x")]) > > + > > ;; Map modes to Usg and Usj constraints for SISD right shifts > > (define_mode_attr cmode_simd [(SI "g") (DI "j")]) > > > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-1-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-1-pmul.c > > new file mode 100644 > > index 00000000000..2bea6280762 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-1-pmul.c > > @@ -0,0 +1,8 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details -fdisable-tree-phiopt2 -fdisable-tree-phiopt3" } */ > > + > > +#include "../../gcc.c-torture/execute/crc-1.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > \ No newline at end of file > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-10-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-10-pmul.c > > new file mode 100644 > > index 00000000000..846eecbaa85 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-10-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-10.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-12-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-12-pmul.c > > new file mode 100644 > > index 00000000000..0eea6aa6741 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-12-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details -fdisable-tree-phiopt2 -fdisable-tree-phiopt3" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-12.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-13-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-13-pmul.c > > new file mode 100644 > > index 00000000000..7ff8fbcb665 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-13-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-13.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-14-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-14-pmul.c > > new file mode 100644 > > index 00000000000..80766daf487 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-14-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-14.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-17-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-17-pmul.c > > new file mode 100644 > > index 00000000000..0e32fffa0b6 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-17-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-17.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-18-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-18-pmul.c > > new file mode 100644 > > index 00000000000..87f4c63b5ea > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-18-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-18.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-21-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-21-pmul.c > > new file mode 100644 > > index 00000000000..6eeac8cf97f > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-21-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-21.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-22-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-22-pmul.c > > new file mode 100644 > > index 00000000000..76e3c00ce9f > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-22-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-22.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-23-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-23-pmul.c > > new file mode 100644 > > index 00000000000..e3a5e99ffba > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-23-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-23.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-4-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-4-pmul.c > > new file mode 100644 > > index 00000000000..528006c0099 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-4-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-4.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-5-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-5-pmul.c > > new file mode 100644 > > index 00000000000..41e1f8202bc > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-5-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -w -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-5.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > \ No newline at end of file > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-6-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-6-pmul.c > > new file mode 100644 > > index 00000000000..83db99ccb8b > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-6-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-6.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > \ No newline at end of file > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-7-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-7-pmul.c > > new file mode 100644 > > index 00000000000..7ad777aac8c > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-7-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-7.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-8-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-8-pmul.c > > new file mode 100644 > > index 00000000000..da1b619c418 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-8-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-8.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-9-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-9-pmul.c > > new file mode 100644 > > index 00000000000..33bbe0bfb26 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-9-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-9.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-CCIT-data16-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-CCIT-data16-pmul.c > > new file mode 100644 > > index 00000000000..0c452c1c0f4 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-CCIT-data16-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-w -march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-CCIT-data16.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > \ No newline at end of file > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-CCIT-data8-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-CCIT-data8-pmul.c > > new file mode 100644 > > index 00000000000..87a0b4489a2 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-CCIT-data8-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-w -march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > > + > > +#include "../../gcc.c-torture/execute/crc-CCIT-data8.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > \ No newline at end of file > > diff --git > a/gcc/testsuite/gcc.target/aarch64/crc-coremark-16bitdata-pmul.c > b/gcc/testsuite/gcc.target/aarch64/crc-coremark-16bitdata-pmul.c > > new file mode 100644 > > index 00000000000..75ed5aff80b > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-coremark-16bitdata-pmul.c > > @@ -0,0 +1,9 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-w -march=armv8-a+crypto -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include "../../gcc.c-torture/execute/crc-coremark16-data16.c" > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "pmull" "dfinish"} } */ > > \ No newline at end of file > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-crc32-data16.c > b/gcc/testsuite/gcc.target/aarch64/crc-crc32-data16.c > > new file mode 100644 > > index 00000000000..d5aeee7c0c4 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-crc32-data16.c > > @@ -0,0 +1,53 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crc -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include <stdint.h> > > +#include <stdlib.h> > > + > > +__attribute__ ((noinline,optimize(0))) > > +uint32_t _crc32_O0 (uint32_t crc, uint16_t data) { > > + int i; > > + crc = crc ^ data; > > + > > + for (i = 0; i < 8; i++) { > > + if (crc & 1) > > + crc = (crc >> 1) ^ 0x82F63B78; > > + else > > + crc = (crc >> 1); > > + } > > + > > + return crc; > > +} > > + > > +uint32_t _crc32 (uint32_t crc, uint16_t data) { > > + int i; > > + crc = crc ^ data; > > + > > + for (i = 0; i < 8; i++) { > > + if (crc & 1) > > + crc = (crc >> 1) ^ 0x82F63B78; > > + else > > + crc = (crc >> 1); > > + } > > + > > + return crc; > > +} > > + > > +int main () > > +{ > > + uint32_t crc = 0x0D800D80; > > + for (uint16_t i = 0; i < 0xffff; i++) > > + { > > + uint32_t res1 = _crc32_O0 (crc, i); > > + uint32_t res2 = _crc32 (crc, i); > > + if (res1 != res2) > > + abort (); > > + crc = res1; > > + } > > +} > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "UNSPEC_CRC32" "dfinish"} } */ > > +/* { dg-final { scan-rtl-dump-times "pmull" 0 "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-crc32-data32.c > b/gcc/testsuite/gcc.target/aarch64/crc-crc32-data32.c > > new file mode 100644 > > index 00000000000..f0e319b3ab8 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-crc32-data32.c > > @@ -0,0 +1,52 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crc -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include <stdint.h> > > +#include <stdlib.h> > > +__attribute__ ((noinline,optimize(0))) > > +uint32_t _crc32_O0 (uint32_t crc, uint32_t data) { > > + int i; > > + crc = crc ^ data; > > + > > + for (i = 0; i < 32; i++) { > > + if (crc & 1) > > + crc = (crc >> 1) ^ 0x82F63B78; > > + else > > + crc = (crc >> 1); > > + } > > + > > + return crc; > > +} > > + > > +uint32_t _crc32 (uint32_t crc, uint32_t data) { > > + int i; > > + crc = crc ^ data; > > + > > + for (i = 0; i < 32; i++) { > > + if (crc & 1) > > + crc = (crc >> 1) ^ 0x82F63B78; > > + else > > + crc = (crc >> 1); > > + } > > + > > + return crc; > > +} > > + > > +int main () > > +{ > > + uint32_t crc = 0x0D800D80; > > + for (uint8_t i = 0; i < 0xff; i++) > > + { > > + uint32_t res1 = _crc32_O0 (crc, i); > > + uint32_t res2 = _crc32 (crc, i); > > + if (res1 != res2) > > + abort (); > > + crc = res1; > > + } > > +} > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "UNSPEC_CRC32" "dfinish"} } */ > > +/* { dg-final { scan-rtl-dump-times "pmull" 0 "dfinish"} } */ > > diff --git a/gcc/testsuite/gcc.target/aarch64/crc-crc32-data8.c > b/gcc/testsuite/gcc.target/aarch64/crc-crc32-data8.c > > new file mode 100644 > > index 00000000000..95ffde6a9d2 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/aarch64/crc-crc32-data8.c > > @@ -0,0 +1,53 @@ > > +/* { dg-do run } */ > > +/* { dg-options "-march=armv8-a+crc -O2 -fdump-rtl-dfinish > -fdump-tree-crc-details" } */ > > +/* { dg-skip-if "" { *-*-* } { "-flto"} } */ > > + > > +#include <stdint.h> > > +#include <stdlib.h> > > + > > +__attribute__ ((noinline,optimize(0))) > > +uint32_t _crc32_O0 (uint32_t crc, uint8_t data) { > > + int i; > > + crc = crc ^ data; > > + > > + for (i = 0; i < 8; i++) { > > + if (crc & 1) > > + crc = (crc >> 1) ^ 0x82F63B78; > > + else > > + crc = (crc >> 1); > > + } > > + > > + return crc; > > +} > > + > > +uint32_t _crc32 (uint32_t crc, uint8_t data) { > > + int i; > > + crc = crc ^ data; > > + > > + for (i = 0; i < 8; i++) { > > + if (crc & 1) > > + crc = (crc >> 1) ^ 0x82F63B78; > > + else > > + crc = (crc >> 1); > > + } > > + > > + return crc; > > +} > > + > > +int main () > > +{ > > + uint32_t crc = 0x0D800D80; > > + for (uint8_t i = 0; i < 0xff; i++) > > + { > > + uint32_t res1 = _crc32_O0 (crc, i); > > + uint32_t res2 = _crc32 (crc, i); > > + if (res1 != res2) > > + abort (); > > + crc = res1; > > + } > > +} > > + > > +/* { dg-final { scan-tree-dump "calculates CRC!" "crc"} } */ > > +/* { dg-final { scan-tree-dump-times "Couldn't generate faster CRC > code." 0 "crc"} } */ > > +/* { dg-final { scan-rtl-dump "UNSPEC_CRC32" "dfinish"} } */ > > +/* { dg-final { scan-rtl-dump-times "pmull" 0 "dfinish"} } */ >