Hi Kyrill, On 12/11/2019 09:40, Kyrill Tkachov wrote: > Hi Dennis, > > On 11/7/19 1:48 PM, Dennis Zhang wrote: >> Hi Kyrill, >> >> I have rebased the patch on top of current truck. >> For resolve_overloaded, I redefined my memtag overloading function to >> fit the latest resolve_overloaded_builtin interface. >> >> Regression tested again and survived for aarch64-none-linux-gnu. > > Please reply inline rather than top-posting on gcc-patches. > > >> Cheers >> Dennis >> >> Changelog is updated as following: >> >> gcc/ChangeLog: >> >> 2019-11-07 Dennis Zhang <dennis.zh...@arm.com> >> >> * config/aarch64/aarch64-builtins.c (enum aarch64_builtins): Add >> AARCH64_MEMTAG_BUILTIN_START, AARCH64_MEMTAG_BUILTIN_IRG, >> AARCH64_MEMTAG_BUILTIN_GMI, AARCH64_MEMTAG_BUILTIN_SUBP, >> AARCH64_MEMTAG_BUILTIN_INC_TAG, AARCH64_MEMTAG_BUILTIN_SET_TAG, >> AARCH64_MEMTAG_BUILTIN_GET_TAG, and AARCH64_MEMTAG_BUILTIN_END. >> (aarch64_init_memtag_builtins): New. >> (AARCH64_INIT_MEMTAG_BUILTINS_DECL): New macro. >> (aarch64_general_init_builtins): Call aarch64_init_memtag_builtins. >> (aarch64_expand_builtin_memtag): New. >> (aarch64_general_expand_builtin): Call aarch64_expand_builtin_memtag. >> (AARCH64_BUILTIN_SUBCODE): New macro. >> (aarch64_resolve_overloaded_memtag): New. >> (aarch64_resolve_overloaded_builtin_general): New hook. Call >> aarch64_resolve_overloaded_memtag to handle overloaded MTE builtins. >> * config/aarch64/aarch64-c.c (aarch64_update_cpp_builtins): Define >> __ARM_FEATURE_MEMORY_TAGGING when enabled. >> (aarch64_resolve_overloaded_builtin): Call >> aarch64_resolve_overloaded_builtin_general. >> * config/aarch64/aarch64-protos.h >> (aarch64_resolve_overloaded_builtin_general): New declaration. >> * config/aarch64/aarch64.h (AARCH64_ISA_MEMTAG): New macro. >> (TARGET_MEMTAG): Likewise. >> * config/aarch64/aarch64.md (define_c_enum "unspec"): Add >> UNSPEC_GEN_TAG, UNSPEC_GEN_TAG_RND, and UNSPEC_TAG_SPACE. >> (irg, gmi, subp, addg, ldg, stg): New instructions. >> * config/aarch64/arm_acle.h (__arm_mte_create_random_tag): New macro. >> (__arm_mte_exclude_tag, __arm_mte_increment_tag): Likewise. >> (__arm_mte_ptrdiff, __arm_mte_set_tag, __arm_mte_get_tag): Likewise. >> * config/aarch64/predicates.md (aarch64_memtag_tag_offset): New. >> (aarch64_granule16_uimm6, aarch64_granule16_simm9): New. >> * config/arm/types.md (memtag): New. >> * doc/invoke.texi (-memtag): Update description. >> >> gcc/testsuite/ChangeLog: >> >> 2019-11-07 Dennis Zhang <dennis.zh...@arm.com> >> >> * gcc.target/aarch64/acle/memtag_1.c: New test. >> * gcc.target/aarch64/acle/memtag_2.c: New test. >> * gcc.target/aarch64/acle/memtag_3.c: New test. >> >> >> On 04/11/2019 16:40, Kyrill Tkachov wrote: >>> Hi Dennis, >>> >>> On 10/17/19 11:03 AM, Dennis Zhang wrote: >>>> Hi, >>>> >>>> Arm Memory Tagging Extension (MTE) is published with Armv8.5-A. >>>> It can be used for spatial and temporal memory safety detection and >>>> lightweight lock and key system. >>>> >>>> This patch enables new intrinsics leveraging MTE instructions to >>>> implement functionalities of creating tags, setting tags, reading tags, >>>> and manipulating tags. >>>> The intrinsics are part of Arm ACLE extension: >>>> https://developer.arm.com/docs/101028/latest/memory-tagging-intrinsics >>>> The MTE ISA specification can be found at >>>> https://developer.arm.com/docs/ddi0487/latest chapter D6. >>>> >>>> Bootstraped and regtested for aarch64-none-linux-gnu. >>>> >>>> Please help to check if it's OK for trunk. >>>> >>> This looks mostly ok to me but for further review this needs to be >>> rebased on top of current trunk as there are some conflicts with the SVE >>> ACLE changes that recently went in. Most conflicts looks trivial to >>> resolve but one that needs more attention is the definition of the >>> TARGET_RESOLVE_OVERLOADED_BUILTIN hook. >>> >>> Thanks, >>> >>> Kyrill >>> >>>> Many Thanks >>>> Dennis >>>> >>>> gcc/ChangeLog: >>>> >>>> 2019-10-16 Dennis Zhang <dennis.zh...@arm.com> >>>> >>>> * config/aarch64/aarch64-builtins.c (enum >>>> aarch64_builtins): Add >>>> AARCH64_MEMTAG_BUILTIN_START, AARCH64_MEMTAG_BUILTIN_IRG, >>>> AARCH64_MEMTAG_BUILTIN_GMI, AARCH64_MEMTAG_BUILTIN_SUBP, >>>> AARCH64_MEMTAG_BUILTIN_INC_TAG, >>>> AARCH64_MEMTAG_BUILTIN_SET_TAG, >>>> AARCH64_MEMTAG_BUILTIN_GET_TAG, and >>>> AARCH64_MEMTAG_BUILTIN_END. >>>> (aarch64_init_memtag_builtins): New. >>>> (AARCH64_INIT_MEMTAG_BUILTINS_DECL): New macro. >>>> (aarch64_general_init_builtins): Call >>>> aarch64_init_memtag_builtins. >>>> (aarch64_expand_builtin_memtag): New. >>>> (aarch64_general_expand_builtin): Call >>>> aarch64_expand_builtin_memtag. >>>> (AARCH64_BUILTIN_SUBCODE): New macro. >>>> (aarch64_resolve_overloaded_memtag): New. >>>> (aarch64_resolve_overloaded_builtin): New hook. Call >>>> aarch64_resolve_overloaded_memtag to handle overloaded MTE >>>> builtins. >>>> * config/aarch64/aarch64-c.c (aarch64_update_cpp_builtins): >>>> Define >>>> __ARM_FEATURE_MEMORY_TAGGING when enabled. >>>> * config/aarch64/aarch64-protos.h >>>> (aarch64_resolve_overloaded_builtin): >>>> Add declaration. >>>> * config/aarch64/aarch64.c >>>> (TARGET_RESOLVE_OVERLOADED_BUILTIN): >>>> New hook. >>>> * config/aarch64/aarch64.h (AARCH64_ISA_MEMTAG): New macro. >>>> (TARGET_MEMTAG): Likewise. >>>> * config/aarch64/aarch64.md (define_c_enum "unspec"): Add >>>> UNSPEC_GEN_TAG, UNSPEC_GEN_TAG_RND, and UNSPEC_TAG_SPACE. >>>> (irg, gmi, subp, addg, ldg, stg): New instructions. >>>> * config/aarch64/arm_acle.h (__arm_mte_create_random_tag): New >>>> macro. >>>> (__arm_mte_exclude_tag, __arm_mte_increment_tag): Likewise. >>>> (__arm_mte_ptrdiff, __arm_mte_set_tag, __arm_mte_get_tag): >>>> Likewise. >>>> * config/aarch64/predicates.md (aarch64_memtag_tag_offset): >>>> New. >>>> (aarch64_granule16_uimm6, aarch64_granule16_simm9): New. >>>> * config/arm/types.md (memtag): New. >>>> * doc/invoke.texi (-memtag): Update description. >>>> >>>> gcc/testsuite/ChangeLog: >>>> >>>> 2019-10-16 Dennis Zhang <dennis.zh...@arm.com> >>>> >>>> * gcc.target/aarch64/acle/memtag_1.c: New test. >>>> * gcc.target/aarch64/acle/memtag_2.c: New test. >>>> * gcc.target/aarch64/acle/memtag_3.c: New test. > > > +/* Expand an expression EXP that calls a MEMTAG built-in FCODE > + with result going to TARGET. */ > +static rtx > +aarch64_expand_builtin_memtag (int fcode, tree exp, rtx target) > +{ > + rtx pat = NULL; > + enum insn_code icode = aarch64_memtag_builtin_data[fcode - > + AARCH64_MEMTAG_BUILTIN_START - 1].icode; > + > + rtx op0 = expand_normal (CALL_EXPR_ARG (exp, 0)); > + machine_mode mode0 = GET_MODE (op0); > + op0 = force_reg (mode0 == VOIDmode ? DImode : mode0, op0); > + op0 = convert_to_mode (DImode, op0, true); > + > + switch (fcode) > + { > + case AARCH64_MEMTAG_BUILTIN_IRG: > + case AARCH64_MEMTAG_BUILTIN_GMI: > + case AARCH64_MEMTAG_BUILTIN_SUBP: > + case AARCH64_MEMTAG_BUILTIN_INC_TAG: > + { > + if (! target > + || GET_MODE (target) != DImode > + || ! (*insn_data[icode].operand[0].predicate) (target, DImode)) > + target = gen_reg_rtx (DImode); > + > + if (fcode == AARCH64_MEMTAG_BUILTIN_INC_TAG) > + { > + rtx op1 = expand_normal (CALL_EXPR_ARG (exp, 1)); > + > + if ((*insn_data[icode].operand[3].predicate) (op1, QImode)) > + { > + pat = GEN_FCN (icode) (target, op0, const0_rtx, op1); > + break; > + } > + error ("%Kargument %d must be a constant immediate " > + "in range [0,15]", exp, 2); > + return target; > + } > + else > + { > + rtx op1 = expand_normal (CALL_EXPR_ARG (exp, 1)); > + machine_mode mode1 = GET_MODE (op1); > + op1 = force_reg (mode1 == VOIDmode ? DImode : mode1, op1); > + op1 = convert_to_mode (DImode, op1, true); > + pat = GEN_FCN (icode) (target, op0, op1); > + } > + break; > + } > + case AARCH64_MEMTAG_BUILTIN_GET_TAG: > + target = op0; > + pat = GEN_FCN (icode) (target, op0, const0_rtx); > + break; > + case AARCH64_MEMTAG_BUILTIN_SET_TAG: > + pat = GEN_FCN (icode) (op0, op0, const0_rtx); > + break; > + default: > + gcc_unreachable(); > + } > + > + if (!pat) > + return NULL_RTX; > + > + emit_insn (pat); > + return target; > +} > + > > I think we want to error out when using -mabi=ilp32. > Can you please add an error message when they are used with TARGET_ILP32.
Done. And it's tested in testsuite memtag_3.c. > > diff --git a/gcc/testsuite/gcc.target/aarch64/acle/memtag_1.c > b/gcc/testsuite/gcc.target/aarch64/acle/memtag_1.c > new file mode 100644 > index 00000000000..a738f7e0b58 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/aarch64/acle/memtag_1.c > @@ -0,0 +1,61 @@ > +/* Test the MEMTAG ACLE intrinsic. */ > + > +/* { dg-do compile { target aarch64-*-* } } */ > +/* { dg-options "-O2 -march=armv8.5-a+memtag" } */ > + > > You shouldn't need the { target aarch64-*-* } here unless you're trying > to restrict this for big-endian? > Instead we should gate these on lp64 target to prevent these from > running on ILP32 targets. Fixed. And regtested again for aarch64-none-linux-gnu. Thanks a lot for the feedback. > > Ok with those two changes. > Thanks for the patch! > Kyrill > > +#include "arm_acle.h" > + > Cheers Dennis
diff --git a/gcc/config/aarch64/aarch64-builtins.c b/gcc/config/aarch64/aarch64-builtins.c index 83d6f75ae07..2f4f564cb32 100644 --- a/gcc/config/aarch64/aarch64-builtins.c +++ b/gcc/config/aarch64/aarch64-builtins.c @@ -448,6 +448,15 @@ enum aarch64_builtins /* Armv8.5-a RNG instruction builtins. */ AARCH64_BUILTIN_RNG_RNDR, AARCH64_BUILTIN_RNG_RNDRRS, + /* MEMTAG builtins. */ + AARCH64_MEMTAG_BUILTIN_START, + AARCH64_MEMTAG_BUILTIN_IRG, + AARCH64_MEMTAG_BUILTIN_GMI, + AARCH64_MEMTAG_BUILTIN_SUBP, + AARCH64_MEMTAG_BUILTIN_INC_TAG, + AARCH64_MEMTAG_BUILTIN_SET_TAG, + AARCH64_MEMTAG_BUILTIN_GET_TAG, + AARCH64_MEMTAG_BUILTIN_END, AARCH64_BUILTIN_MAX }; @@ -1130,6 +1139,51 @@ aarch64_init_rng_builtins (void) AARCH64_BUILTIN_RNG_RNDRRS); } +/* Initialize the memory tagging extension (MTE) builtins. */ +struct +{ + tree ftype; + enum insn_code icode; +} aarch64_memtag_builtin_data[AARCH64_MEMTAG_BUILTIN_END - + AARCH64_MEMTAG_BUILTIN_START - 1]; + +static void +aarch64_init_memtag_builtins (void) +{ + tree fntype = NULL; + +#define AARCH64_INIT_MEMTAG_BUILTINS_DECL(F, N, I, T) \ + aarch64_builtin_decls[AARCH64_MEMTAG_BUILTIN_##F] \ + = aarch64_general_add_builtin ("__builtin_aarch64_memtag_"#N, \ + T, AARCH64_MEMTAG_BUILTIN_##F); \ + aarch64_memtag_builtin_data[AARCH64_MEMTAG_BUILTIN_##F - \ + AARCH64_MEMTAG_BUILTIN_START - 1] = \ + {T, CODE_FOR_##I}; + + fntype = build_function_type_list (ptr_type_node, ptr_type_node, + uint64_type_node, NULL); + AARCH64_INIT_MEMTAG_BUILTINS_DECL (IRG, irg, irg, fntype); + + fntype = build_function_type_list (uint64_type_node, ptr_type_node, + uint64_type_node, NULL); + AARCH64_INIT_MEMTAG_BUILTINS_DECL (GMI, gmi, gmi, fntype); + + fntype = build_function_type_list (ptrdiff_type_node, ptr_type_node, + ptr_type_node, NULL); + AARCH64_INIT_MEMTAG_BUILTINS_DECL (SUBP, subp, subp, fntype); + + fntype = build_function_type_list (ptr_type_node, ptr_type_node, + unsigned_type_node, NULL); + AARCH64_INIT_MEMTAG_BUILTINS_DECL (INC_TAG, inc_tag, addg, fntype); + + fntype = build_function_type_list (void_type_node, ptr_type_node, NULL); + AARCH64_INIT_MEMTAG_BUILTINS_DECL (SET_TAG, set_tag, stg, fntype); + + fntype = build_function_type_list (ptr_type_node, ptr_type_node, NULL); + AARCH64_INIT_MEMTAG_BUILTINS_DECL (GET_TAG, get_tag, ldg, fntype); + +#undef AARCH64_INIT_MEMTAG_BUILTINS_DECL +} /* Initialize all builtins in the AARCH64_BUILTIN_GENERAL group. */ @@ -1184,6 +1238,9 @@ aarch64_general_init_builtins (void) if (TARGET_TME) aarch64_init_tme_builtins (); + + if (TARGET_MEMTAG) + aarch64_init_memtag_builtins (); } /* Implement TARGET_BUILTIN_DECL for the AARCH64_BUILTIN_GENERAL group. */ @@ -1665,6 +1722,76 @@ aarch64_expand_rng_builtin (tree exp, rtx target, int fcode, int ignore) return target; } +/* Expand an expression EXP that calls a MEMTAG built-in FCODE + with result going to TARGET. */ +static rtx +aarch64_expand_builtin_memtag (int fcode, tree exp, rtx target) +{ + if (TARGET_ILP32) + error ("Memory Tagging Extension does not support '-mabi=ilp32'"); + + rtx pat = NULL; + enum insn_code icode = aarch64_memtag_builtin_data[fcode - + AARCH64_MEMTAG_BUILTIN_START - 1].icode; + + rtx op0 = expand_normal (CALL_EXPR_ARG (exp, 0)); + machine_mode mode0 = GET_MODE (op0); + op0 = force_reg (mode0 == VOIDmode ? DImode : mode0, op0); + op0 = convert_to_mode (DImode, op0, true); + + switch (fcode) + { + case AARCH64_MEMTAG_BUILTIN_IRG: + case AARCH64_MEMTAG_BUILTIN_GMI: + case AARCH64_MEMTAG_BUILTIN_SUBP: + case AARCH64_MEMTAG_BUILTIN_INC_TAG: + { + if (! target + || GET_MODE (target) != DImode + || ! (*insn_data[icode].operand[0].predicate) (target, DImode)) + target = gen_reg_rtx (DImode); + + if (fcode == AARCH64_MEMTAG_BUILTIN_INC_TAG) + { + rtx op1 = expand_normal (CALL_EXPR_ARG (exp, 1)); + + if ((*insn_data[icode].operand[3].predicate) (op1, QImode)) + { + pat = GEN_FCN (icode) (target, op0, const0_rtx, op1); + break; + } + error ("%Kargument %d must be a constant immediate " + "in range [0,15]", exp, 2); + return target; + } + else + { + rtx op1 = expand_normal (CALL_EXPR_ARG (exp, 1)); + machine_mode mode1 = GET_MODE (op1); + op1 = force_reg (mode1 == VOIDmode ? DImode : mode1, op1); + op1 = convert_to_mode (DImode, op1, true); + pat = GEN_FCN (icode) (target, op0, op1); + } + break; + } + case AARCH64_MEMTAG_BUILTIN_GET_TAG: + target = op0; + pat = GEN_FCN (icode) (target, op0, const0_rtx); + break; + case AARCH64_MEMTAG_BUILTIN_SET_TAG: + pat = GEN_FCN (icode) (op0, op0, const0_rtx); + break; + default: + gcc_unreachable(); + } + + if (!pat) + return NULL_RTX; + + emit_insn (pat); + return target; +} + /* Expand an expression EXP that calls built-in function FCODE, with result going to TARGET if that's convenient. IGNORE is true if the result of the builtin is ignored. */ @@ -1800,6 +1927,10 @@ aarch64_general_expand_builtin (unsigned int fcode, tree exp, rtx target, || fcode == AARCH64_TME_BUILTIN_TCANCEL) return aarch64_expand_builtin_tme (fcode, exp, target); + if (fcode >= AARCH64_MEMTAG_BUILTIN_START + && fcode <= AARCH64_MEMTAG_BUILTIN_END) + return aarch64_expand_builtin_memtag (fcode, exp, target); + gcc_unreachable (); } @@ -2151,6 +2282,106 @@ aarch64_atomic_assign_expand_fenv (tree *hold, tree *clear, tree *update) reload_fenv, restore_fnenv), update_call); } +/* Resolve overloaded MEMTAG build-in functions. */ +#define AARCH64_BUILTIN_SUBCODE(F) \ + (DECL_MD_FUNCTION_CODE (F) >> AARCH64_BUILTIN_SHIFT) + +static tree +aarch64_resolve_overloaded_memtag (location_t loc, + tree fndecl, void *pass_params) +{ + vec<tree, va_gc> *params = static_cast<vec<tree, va_gc> *> (pass_params); + unsigned param_num = params ? params->length() : 0; + unsigned int fcode = AARCH64_BUILTIN_SUBCODE (fndecl); + tree inittype = aarch64_memtag_builtin_data[ + fcode - AARCH64_MEMTAG_BUILTIN_START - 1].ftype; + unsigned arg_num = list_length (TYPE_ARG_TYPES (inittype)) - 1; + + if (param_num != arg_num) + { + TREE_TYPE (fndecl) = inittype; + return NULL_TREE; + } + tree retype = NULL; + + if (fcode == AARCH64_MEMTAG_BUILTIN_SUBP) + { + tree t0 = TREE_TYPE ((*params)[0]); + tree t1 = TREE_TYPE ((*params)[1]); + + if (t0 == error_mark_node || TREE_CODE (t0) != POINTER_TYPE) + t0 = ptr_type_node; + if (t1 == error_mark_node || TREE_CODE (t1) != POINTER_TYPE) + t1 = ptr_type_node; + + if (TYPE_MODE (t0) != DImode) + warning_at (loc, 1, "expected 64-bit address but argument 1 is %d-bit", + (int)tree_to_shwi (DECL_SIZE ((*params)[0]))); + + if (TYPE_MODE (t1) != DImode) + warning_at (loc, 1, "expected 64-bit address but argument 2 is %d-bit", + (int)tree_to_shwi (DECL_SIZE ((*params)[1]))); + + retype = build_function_type_list (ptrdiff_type_node, t0, t1, NULL); + } + else + { + tree t0 = TREE_TYPE ((*params)[0]); + + if (t0 == error_mark_node || TREE_CODE (t0) != POINTER_TYPE) + { + TREE_TYPE (fndecl) = inittype; + return NULL_TREE; + } + + if (TYPE_MODE (t0) != DImode) + warning_at (loc, 1, "expected 64-bit address but argument 1 is %d-bit", + (int)tree_to_shwi (DECL_SIZE ((*params)[0]))); + + switch (fcode) + { + case AARCH64_MEMTAG_BUILTIN_IRG: + retype = build_function_type_list (t0, t0, uint64_type_node, NULL); + break; + case AARCH64_MEMTAG_BUILTIN_GMI: + retype = build_function_type_list (uint64_type_node, t0, + uint64_type_node, NULL); + break; + case AARCH64_MEMTAG_BUILTIN_INC_TAG: + retype = build_function_type_list (t0, t0, unsigned_type_node, NULL); + break; + case AARCH64_MEMTAG_BUILTIN_SET_TAG: + retype = build_function_type_list (void_type_node, t0, NULL); + break; + case AARCH64_MEMTAG_BUILTIN_GET_TAG: + retype = build_function_type_list (t0, t0, NULL); + break; + default: + return NULL_TREE; + } + } + + if (!retype || retype == error_mark_node) + TREE_TYPE (fndecl) = inittype; + else + TREE_TYPE (fndecl) = retype; + + return NULL_TREE; +} + +/* Called at aarch64_resolve_overloaded_builtin in aarch64-c.c. */ +tree +aarch64_resolve_overloaded_builtin_general (location_t loc, tree function, + void *pass_params) +{ + unsigned int fcode = AARCH64_BUILTIN_SUBCODE (function); + + if (fcode >= AARCH64_MEMTAG_BUILTIN_START + && fcode <= AARCH64_MEMTAG_BUILTIN_END) + return aarch64_resolve_overloaded_memtag(loc, function, pass_params); + + return NULL_TREE; +} #undef AARCH64_CHECK_BUILTIN_MODE #undef AARCH64_FIND_FRINT_VARIANT diff --git a/gcc/config/aarch64/aarch64-c.c b/gcc/config/aarch64/aarch64-c.c index 7c322ca0813..7ffa4609be4 100644 --- a/gcc/config/aarch64/aarch64-c.c +++ b/gcc/config/aarch64/aarch64-c.c @@ -161,6 +161,7 @@ aarch64_update_cpp_builtins (cpp_reader *pfile) aarch64_def_or_undef (TARGET_FRINT, "__ARM_FEATURE_FRINT", pfile); aarch64_def_or_undef (TARGET_TME, "__ARM_FEATURE_TME", pfile); aarch64_def_or_undef (TARGET_RNG, "__ARM_FEATURE_RNG", pfile); + aarch64_def_or_undef (TARGET_MEMTAG, "__ARM_FEATURE_MEMORY_TAGGING", pfile); /* Not for ACLE, but required to keep "float.h" correct if we switch target between implementations that do or do not support ARMv8.2-A @@ -277,8 +278,8 @@ aarch64_resolve_overloaded_builtin (unsigned int uncast_location, switch (code & AARCH64_BUILTIN_CLASS) { case AARCH64_BUILTIN_GENERAL: - return NULL_TREE; - + return aarch64_resolve_overloaded_builtin_general (location, fndecl, + uncast_arglist); case AARCH64_BUILTIN_SVE: new_fndecl = aarch64_sve::resolve_overloaded_builtin (location, subcode, arglist); diff --git a/gcc/config/aarch64/aarch64-protos.h b/gcc/config/aarch64/aarch64-protos.h index 1d4f4fd4888..c60c841c60c 100644 --- a/gcc/config/aarch64/aarch64-protos.h +++ b/gcc/config/aarch64/aarch64-protos.h @@ -772,4 +772,6 @@ extern const atomic_ool_names aarch64_ool_ldset_names; extern const atomic_ool_names aarch64_ool_ldclr_names; extern const atomic_ool_names aarch64_ool_ldeor_names; +tree aarch64_resolve_overloaded_builtin_general (location_t, tree, void *); + #endif /* GCC_AARCH64_PROTOS_H */ diff --git a/gcc/config/aarch64/aarch64.h b/gcc/config/aarch64/aarch64.h index 425a36332cd..ee01909abb9 100644 --- a/gcc/config/aarch64/aarch64.h +++ b/gcc/config/aarch64/aarch64.h @@ -248,6 +248,7 @@ extern unsigned aarch64_architecture_version; #define AARCH64_ISA_RNG (aarch64_isa_flags & AARCH64_FL_RNG) #define AARCH64_ISA_V8_5 (aarch64_isa_flags & AARCH64_FL_V8_5) #define AARCH64_ISA_TME (aarch64_isa_flags & AARCH64_FL_TME) +#define AARCH64_ISA_MEMTAG (aarch64_isa_flags & AARCH64_FL_MEMTAG) /* Crypto is an optional extension to AdvSIMD. */ #define TARGET_CRYPTO (TARGET_SIMD && AARCH64_ISA_CRYPTO) @@ -304,6 +305,9 @@ extern unsigned aarch64_architecture_version; /* Random number instructions from Armv8.5-a. */ #define TARGET_RNG (AARCH64_ISA_RNG) +/* Memory Tagging instructions optional to Armv8.5 enabled through +memtag. */ +#define TARGET_MEMTAG (AARCH64_ISA_V8_5 && AARCH64_ISA_MEMTAG) + /* Make sure this is always defined so we don't have to check for ifdefs but rather use normal ifs. */ #ifndef TARGET_FIX_ERR_A53_835769_DEFAULT diff --git a/gcc/config/aarch64/aarch64.md b/gcc/config/aarch64/aarch64.md index f19e2272750..9987fb900b5 100644 --- a/gcc/config/aarch64/aarch64.md +++ b/gcc/config/aarch64/aarch64.md @@ -268,6 +268,9 @@ UNSPEC_SVE_PREFETCH_GATHER UNSPEC_SVE_COMPACT UNSPEC_SVE_SPLICE + UNSPEC_GEN_TAG ; Generate a 4-bit MTE tag. + UNSPEC_GEN_TAG_RND ; Generate a random 4-bit MTE tag. + UNSPEC_TAG_SPACE ; Translate address to MTE tag address space. ]) (define_c_enum "unspecv" [ @@ -7384,6 +7387,93 @@ [(set_attr "type" "mrs")] ) +;; Memory Tagging Extension (MTE) instructions. + +(define_insn "irg" + [(set (match_operand:DI 0 "register_operand" "=rk") + (ior:DI + (and:DI (match_operand:DI 1 "register_operand" "rk") + (const_int -1080863910568919041)) ;; 0xf0ff... + (ashift:DI (unspec:QI [(match_operand:DI 2 "register_operand" "r")] + UNSPEC_GEN_TAG_RND) + (const_int 56))))] + "TARGET_MEMTAG" + "irg\\t%0, %1, %2" + [(set_attr "type" "memtag")] +) + +(define_insn "gmi" + [(set (match_operand:DI 0 "register_operand" "=r") + (ior:DI (ashift:DI + (const_int 1) + (and:QI (lshiftrt:DI + (match_operand:DI 1 "register_operand" "rk") + (const_int 56)) (const_int 15))) + (match_operand:DI 2 "register_operand" "r")))] + "TARGET_MEMTAG" + "gmi\\t%0, %1, %2" + [(set_attr "type" "memtag")] +) + +(define_insn "addg" + [(set (match_operand:DI 0 "register_operand" "=rk") + (ior:DI + (and:DI (plus:DI (match_operand:DI 1 "register_operand" "rk") + (match_operand:DI 2 "aarch64_granule16_uimm6" "i")) + (const_int -1080863910568919041)) ;; 0xf0ff... + (ashift:DI + (unspec:QI + [(and:QI (lshiftrt:DI (match_dup 1) (const_int 56)) (const_int 15)) + (match_operand:QI 3 "aarch64_memtag_tag_offset" "i")] + UNSPEC_GEN_TAG) + (const_int 56))))] + "TARGET_MEMTAG" + "addg\\t%0, %1, #%2, #%3" + [(set_attr "type" "memtag")] +) + +(define_insn "subp" + [(set (match_operand:DI 0 "register_operand" "=r") + (minus:DI + (and:DI (match_operand:DI 1 "register_operand" "rk") + (const_int 72057594037927935)) ;; 0x00ff... + (and:DI (match_operand:DI 2 "register_operand" "rk") + (const_int 72057594037927935))))] ;; 0x00ff... + "TARGET_MEMTAG" + "subp\\t%0, %1, %2" + [(set_attr "type" "memtag")] +) + +;; LDG will use the 16-byte aligned value of the address. +(define_insn "ldg" + [(set (match_operand:DI 0 "register_operand" "+r") + (ior:DI + (and:DI (match_dup 0) (const_int -1080863910568919041)) ;; 0xf0ff... + (ashift:DI + (mem:QI (unspec:DI + [(and:DI (plus:DI (match_operand:DI 1 "register_operand" "rk") + (match_operand:DI 2 "aarch64_granule16_simm9" "i")) + (const_int -16))] UNSPEC_TAG_SPACE)) + (const_int 56))))] + "TARGET_MEMTAG" + "ldg\\t%0, [%1, #%2]" + [(set_attr "type" "memtag")] +) + +;; STG doesn't align the address but aborts with alignment fault +;; when the address is not 16-byte aligned. +(define_insn "stg" + [(set (mem:QI (unspec:DI + [(plus:DI (match_operand:DI 1 "register_operand" "rk") + (match_operand:DI 2 "aarch64_granule16_simm9" "i"))] + UNSPEC_TAG_SPACE)) + (and:QI (lshiftrt:DI (match_operand:DI 0 "register_operand" "rk") + (const_int 56)) (const_int 15)))] + "TARGET_MEMTAG" + "stg\\t%0, [%1, #%2]" + [(set_attr "type" "memtag")] +) + ;; AdvSIMD Stuff (include "aarch64-simd.md") diff --git a/gcc/config/aarch64/arm_acle.h b/gcc/config/aarch64/arm_acle.h index 2284e71648a..1dfac866e0a 100644 --- a/gcc/config/aarch64/arm_acle.h +++ b/gcc/config/aarch64/arm_acle.h @@ -209,6 +209,29 @@ __rndrrs (uint64_t *__res) #pragma GCC pop_options +#pragma GCC push_options +#pragma GCC target ("arch=armv8.5-a+memtag") + +#define __arm_mte_create_random_tag(__ptr, __u64_mask) \ + __builtin_aarch64_memtag_irg(__ptr, __u64_mask) + +#define __arm_mte_exclude_tag(__ptr, __u64_excluded) \ + __builtin_aarch64_memtag_gmi(__ptr, __u64_excluded) + +#define __arm_mte_ptrdiff(__ptr_a, __ptr_b) \ + __builtin_aarch64_memtag_subp(__ptr_a, __ptr_b) + +#define __arm_mte_increment_tag(__ptr, __u_offset) \ + __builtin_aarch64_memtag_inc_tag(__ptr, __u_offset) + +#define __arm_mte_set_tag(__tagged_address) \ + __builtin_aarch64_memtag_set_tag(__tagged_address) + +#define __arm_mte_get_tag(__address) \ + __builtin_aarch64_memtag_get_tag(__address) + +#pragma GCC pop_options + #ifdef __cplusplus } #endif diff --git a/gcc/config/aarch64/predicates.md b/gcc/config/aarch64/predicates.md index 2c5c53c716d..7eb49cfe66a 100644 --- a/gcc/config/aarch64/predicates.md +++ b/gcc/config/aarch64/predicates.md @@ -869,3 +869,17 @@ (define_predicate "aarch64_sve_any_binary_operator" (match_code "plus,minus,mult,div,udiv,smax,umax,smin,umin,and,ior,xor")) + +(define_predicate "aarch64_memtag_tag_offset" + (and (match_code "const_int") + (match_test "IN_RANGE (INTVAL (op), 0, 15)"))) + +(define_predicate "aarch64_granule16_uimm6" + (and (match_code "const_int") + (match_test "IN_RANGE (INTVAL (op), 0, 1008) + && !(INTVAL (op) & 0xf)"))) + +(define_predicate "aarch64_granule16_simm9" + (and (match_code "const_int") + (match_test "IN_RANGE (INTVAL (op), -4096, 4080) + && !(INTVAL (op) & 0xf)"))) diff --git a/gcc/config/arm/types.md b/gcc/config/arm/types.md index 60faad65979..df39522f2ad 100644 --- a/gcc/config/arm/types.md +++ b/gcc/config/arm/types.md @@ -1096,7 +1096,8 @@ crypto_sm3,\ crypto_sm4,\ coproc,\ - tme" + tme,\ + memtag" (const_string "untyped")) ; Is this an (integer side) multiply with a 32-bit (or smaller) result? diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index 00eb7e77808..11e093e3da6 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -16268,9 +16268,8 @@ Enable the Armv8.5-a Random Number instructions. This option is only to enable the extension at the assembler level and does not affect code generation. @item memtag -Enable the Armv8.5-a Memory Tagging Extensions. This option is only to -enable the extension at the assembler level and does not affect code -generation. +Enable the Armv8.5-a Memory Tagging Extensions. +Use of this option with architectures prior to Armv8.5-A is not supported. @item sb Enable the Armv8-a Speculation Barrier instruction. This option is only to enable the extension at the assembler level and does not affect code diff --git a/gcc/testsuite/gcc.target/aarch64/acle/memtag_1.c b/gcc/testsuite/gcc.target/aarch64/acle/memtag_1.c new file mode 100644 index 00000000000..f8368690032 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/acle/memtag_1.c @@ -0,0 +1,62 @@ +/* Test the MEMTAG ACLE intrinsic. */ + +/* { dg-do compile } */ +/* { dg-require-effective-target lp64 } */ +/* { dg-options "-O3 -march=armv8.5-a+memtag" } */ + +#include "arm_acle.h" + +/* irg */ + +void * +test_memtag_1 (void *p) +{ + return __arm_mte_create_random_tag (p, 0); +} + +/* gmi */ + +uint64_t +test_memtag_2 (void *p) +{ + return __arm_mte_exclude_tag (p, 0); +} + +/* addg */ + +void * +test_memtag_3 (void *p) +{ + return __arm_mte_increment_tag (p, 1); +} + +/* subp */ + +int64_t +test_memtag_4 (void *p, void *q) +{ + return __arm_mte_ptrdiff (p, q); +} + +/* ldg */ + +void * +test_memtag_5 (void *p) +{ + return __arm_mte_get_tag (p); +} + +/* stg */ + +void +test_memtag_6 (void *p) +{ + __arm_mte_set_tag (p); +} + +/* { dg-final { scan-assembler-times {irg\tx..?, x..?, x..?\n} 1 } } */ +/* { dg-final { scan-assembler-times {gmi\tx..?, x..?, x..?\n} 1 } } */ +/* { dg-final { scan-assembler-times {subp\tx..?, x..?, x..?\n} 1 } } */ +/* { dg-final { scan-assembler-times {addg\tx..?, x..?, #0, #1\n} 1 } } */ +/* { dg-final { scan-assembler-times {ldg\tx..?, \[x..?, #0\]\n} 1 } } */ +/* { dg-final { scan-assembler-times {stg\tx..?, \[x..?, #0\]\n} 1 } } */ \ No newline at end of file diff --git a/gcc/testsuite/gcc.target/aarch64/acle/memtag_2.c b/gcc/testsuite/gcc.target/aarch64/acle/memtag_2.c new file mode 100644 index 00000000000..fcab05b7abe --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/acle/memtag_2.c @@ -0,0 +1,70 @@ +/* Test the MEMTAG intrinsic qualifier warnings and argument errors. */ + +/* { dg-do compile } */ +/* { dg-require-effective-target lp64 } */ +/* { dg-options "-O3 -march=armv8.5-a+memtag" } */ + +#include "arm_acle.h" + +void +test_memtag_warning_return_qualifier (void) +{ + const char *c; + volatile char *v; + char *n; + int *i; + int64_t d; + + v = __arm_mte_get_tag(c); /* { dg-warning {assignment} } */ + n = __arm_mte_get_tag(c); /* { dg-warning {assignment} } */ + i = __arm_mte_get_tag(c); /* { dg-warning {assignment} } */ + c = __arm_mte_get_tag(v); /* { dg-warning {assignment} } */ + n = __arm_mte_get_tag(v); /* { dg-warning {assignment} } */ + + i = __arm_mte_create_random_tag (c, 0); /* { dg-warning {assignment} } */ + i = __arm_mte_increment_tag (c, 0); /* { dg-warning {assignment} } */ + + c = __arm_mte_get_tag(n); /* No warning. */ + d = __arm_mte_ptrdiff(c, i); /* No warning. */ +} + +void +test_memtag_warning_argument (void) +{ + const char *c; + uint64_t i; + __arm_mte_exclude_tag(i, 0); /* { dg-warning {argument} } */ + __arm_mte_create_random_tag (i, 0); /* { dg-warning {argument} } */ + __arm_mte_set_tag(i); /* { dg-warning {argument} } */ + __arm_mte_get_tag(i); /* { dg-warning {argument} } */ + __arm_mte_increment_tag (i, 15); /* { dg-warning {argument} } */ + __arm_mte_ptrdiff(c, i); /* { dg-warning {argument} } */ + __arm_mte_ptrdiff(i, c); /* { dg-warning {argument} } */ + + __arm_mte_exclude_tag(1, 0); /* { dg-warning {argument} } */ + __arm_mte_create_random_tag (1, 0); /* { dg-warning {argument} } */ + __arm_mte_set_tag(1); /* { dg-warning {argument} } */ + __arm_mte_get_tag(1); /* { dg-warning {argument} } */ + __arm_mte_increment_tag (1, 15); /* { dg-warning {argument} } */ + __arm_mte_ptrdiff(c, 1); /* { dg-warning {argument} } */ + __arm_mte_ptrdiff(1, c); /* { dg-warning {argument} } */ + + __arm_mte_exclude_tag(0, 0); /* No warning. */ + __arm_mte_create_random_tag (0, 0); /* No warning. */ + __arm_mte_set_tag(0); /* No warning. */ + __arm_mte_get_tag(0); /* No warning. */ + __arm_mte_increment_tag (0, 15); /* No warning. */ + __arm_mte_ptrdiff(c, 0); /* No warning. */ + __arm_mte_ptrdiff(0, c); /* No warning. */ +} + +void +test_memtag_error_argument (void) +{ + /* Produce errors properly for invalid arguments. */ + __arm_mte_exclude_tag(no_decl, 0); /* { dg-error {} } */ + __arm_mte_exclude_tag(); /* { dg-error {} } */ + __arm_mte_ptrdiff(no_decl2, 0); /* { dg-error {} } */ + __arm_mte_ptrdiff(0); /* { dg-error {} } */ + __arm_mte_ptrdiff(); /* { dg-error {} } */ +} \ No newline at end of file diff --git a/gcc/testsuite/gcc.target/aarch64/acle/memtag_3.c b/gcc/testsuite/gcc.target/aarch64/acle/memtag_3.c new file mode 100644 index 00000000000..fd2f6ea7a36 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/acle/memtag_3.c @@ -0,0 +1,17 @@ +/* Test the MEMTAG intrinsic expanding errors. */ + +/* { dg-do compile } */ +/* { dg-require-effective-target lp64 } */ +/* { dg-options "-O3 -march=armv8.5-a+memtag -mabi=ilp32" } */ + +#include "arm_acle.h" + +void +test_memtag_error_expand (int i) +{ + const char *p; + /* { dg-error "not support '-mabi=ilp32'" "" { target *-*-* } 0 } */ + p = __arm_mte_increment_tag (p, -1); /* { dg-error {in range \[0,15\]} } */ + p = __arm_mte_increment_tag (p, 16); /* { dg-error {in range \[0,15\]} } */ + p = __arm_mte_increment_tag (p, i); /* { dg-error {constant immediate} } */ +}