Hi David,
Thanks for the comments. I have just realized that I introduced several errors in the patch by mistake right before I sent it. Like the missing "*". Apologies, I will revise it and send a new patch in a moment. Thanks, Cupertino David Faust writes: > On 10/25/23 11:51, Cupertino Miranda wrote: >> Hi everyone, >> >> This patch contains some more recent improvements to BPF CO-RE builtins. >> Please find further details of the changes on the patch header. >> >> Looking forward for your review and comments. >> >> Best regards, >> Cupertino Miranda > > Hi Cupertino, > > Thanks for continuing to work on these BPF builtins. > > Comments inline below. > >> >> >> bpf_builtins_upstream.patch >> >> commit 6054209c0a8af9c3e6363550bf2ba4f4f2172eba >> Author: Cupertino Miranda <cupertino.mira...@oracle.com> >> Date: Tue Aug 8 09:22:41 2023 +0100 >> >> bpf: Improvements in CO-RE builtins implementation. >> >> This patch moved the processing of attribute preserve_access_index to >> its own independent pass in a gimple lowering pass. >> This approach is more consistent with the implementation of the CO-RE >> builtins when used explicitly in the code. The attributed type accesses >> are now early converted to __builtin_core_reloc builtin instead of being >> kept as an expression in code through out all of the middle-end. >> This disables the compiler to optimize out or manipulate the expression >> using the local defined type, instead of assuming nothing is known about >> this expression, as it should be the case in all of the CO-RE >> relocations. >> >> In the process, also the __builtin_preserve_access_index has been >> improved to generate code for more complex expressions that would >> require more then one CO-RE relocation. >> This turned out to be a requirement, since bpf-next selftests would rely >> on >> loop unrolling in order to convert an undefined index array access into a >> defined one. This seemed extreme to expect for the unroll to happen, and >> for >> that reason GCC still generates correct code in such scenarios, even >> when index >> access is never predictable or unrolling does not occur. >> >> gcc/ChangeLog: >> * config/bpf/bpf-passes.def (pass_lower_bpf_core): Added pass. > > It may only be due to how the patch is formatted in the attachment > (everything above the first diff seems to be indented?), but each entry > here should start with a tab rather than spaces. > > Please double-check with contrib/gcc-changelog/git_check_commit.py, it > will complain if the indentation is wrong. > >> * config/bpf/bpf-protos.h: Added prototype for new pass. >> * config/bpf/bpf.cc (bpf_const_not_ok_for_debug_p): New function. >> * config/bpf/bpf.md (mov_reloc_core<MM:mode>): Changed > > Looks like an entry along the lines of "Renamed to..." would be > more fitting. > >> * config/bpf/core-builtins.cc (cr_builtins, >> is_attr_preserve_access, >> core_field_info, bpf_core_get_index, compute_field_expr, >> process_field_expr, pack_type, make_core_relo, >> bpf_handle_plugin_finish_type, core_buintin_helpers, >> construct_builtin_core_reloc, >> bpf_resolve_overloaded_core_builtin, >> bpf_add_core_reloc): Changed. > > "Changed." as a ChangeLog entry is not useful. Please be at least a > little bit more descriptive here so someone only reading the ChangeLog > has an idea of what they're looking at. > > Also, I'm not sure this is a proper way to note multiple functions > with the same entry, usually this is done with a new pair of () for > each new line, like: > > * config/bpf/core-builtins.cc (cr_builtins, is_attr_preserve_access) > (core_field_info, bpf_core_get_index): Changed to do blah. > > contrib/gcc-changelog/git_check_commit.py might complain about this; > please check. > >> (root_for_core_field_info, pack_field_expr, >> core_expr_with_field_expr_plus_base, make_core_safe_access_index, >> replace_core_access_index_comp_expr, >> maybe_get_base_for_field_expr, >> core_access_clean, core_is_access_index, >> core_mark_as_access_index, >> make_gimple_core_safe_access_index, execute_lower_bpf_core, >> make_pass_lower_bpf_core): Added functions. > > Same thing about the multi-line entry within a single () pair. > >> (pass_data_lower_bpf_core): New pass struct. >> (pass_lower_bpf_core): New gimple_opt_pass class. >> (pack_field_expr_for_preserve_field, >> bpf_replace_core_move_operands): Removed function. >> (bpf_enum_value_kind): Added GTY(()). >> * config/bpf/core-builtins.h (bpf_field_info_kind, >> bpf_type_id_kind, >> bpf_type_info_kind, bpf_enum_value_kind): New enum. >> * config/bpf/t-bpf: Added pass bpf-passes.def to PASSES_EXTRA. >> >> gcc/testsuite/ChangeLog: >> * gcc.target/bpf/core-attr-5.c: New test. >> * gcc.target/bpf/core-attr-6.c: New test. >> * gcc.target/bpf/core-builtin-1.c: Corrected >> * gcc.target/bpf/core-builtin-enumvalue-opt.c: Corrected regular >> expression. >> * gcc.target/bpf/core-builtin-enumvalue.c: Corrected regular >> expression. >> * gcc.target/bpf/core-builtin-exprlist-1.c: New test. >> * gcc.target/bpf/core-builtin-exprlist-2.c: New test. >> * gcc.target/bpf/core-builtin-exprlist-3.c: New test. >> * gcc.target/bpf/core-builtin-exprlist-4.c: New test. >> * gcc.target/bpf/core-builtin-fieldinfo-offset-1.c: Extra tests >> >> diff --git a/gcc/config/bpf/bpf-passes.def b/gcc/config/bpf/bpf-passes.def >> new file mode 100644 >> index 000000000000..249c58e22067 >> --- /dev/null >> +++ b/gcc/config/bpf/bpf-passes.def >> @@ -0,0 +1,20 @@ >> +/* Declaration of target-specific passes for eBPF. >> + Copyright (C) 2021-2023 Free Software Foundation, Inc. > > This file is new, so just 2023 no? > >> + >> + This file is part of GCC. >> + >> + GCC is free software; you can redistribute it and/or modify it >> + under the terms of the GNU General Public License as published by >> + the Free Software Foundation; either version 3, or (at your option) >> + any later version. >> + >> + GCC is distributed in the hope that it will be useful, but >> + WITHOUT ANY WARRANTY; without even the implied warranty of >> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >> + General Public License for more details. >> + >> + You should have received a copy of the GNU General Public License >> + along with GCC; see the file COPYING3. If not see >> + <http://www.gnu.org/licenses/>. */ >> + >> +INSERT_PASS_BEFORE (pass_lower_omp, 1, pass_lower_bpf_core); >> diff --git a/gcc/config/bpf/bpf-protos.h b/gcc/config/bpf/bpf-protos.h >> index fbe0d8a0213f..ecbbd7870e13 100644 >> --- a/gcc/config/bpf/bpf-protos.h >> +++ b/gcc/config/bpf/bpf-protos.h >> @@ -31,6 +31,8 @@ extern void bpf_expand_prologue (void); >> extern void bpf_expand_epilogue (void); >> extern void bpf_expand_cbranch (machine_mode, rtx *); >> const char *bpf_add_core_reloc (rtx *operands, const char *templ); >> -void bpf_replace_core_move_operands (rtx *operands); >> + >> +class gimple_opt_pass; >> +gimple_opt_pass *make_pass_lower_bpf_core (gcc::context *ctxt); >> >> #endif /* ! GCC_BPF_PROTOS_H */ >> diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc >> index 437bd652de3e..a340cd742d54 100644 >> --- a/gcc/config/bpf/bpf.cc >> +++ b/gcc/config/bpf/bpf.cc >> @@ -1034,6 +1034,18 @@ bpf_resolve_overloaded_builtin (location_t loc, tree >> fndecl, void *arglist) >> #undef TARGET_RESOLVE_OVERLOADED_BUILTIN >> #define TARGET_RESOLVE_OVERLOADED_BUILTIN bpf_resolve_overloaded_builtin >> >> +static bool >> +bpf_const_not_ok_for_debug_p (rtx p) >> +{ >> + if (GET_CODE (p) == UNSPEC >> + && XINT (p, 1) == UNSPEC_CORE_RELOC) >> + return true; >> + >> + return false; >> +} >> + >> +#undef TARGET_CONST_NOT_OK_FOR_DEBUG_P >> +#define TARGET_CONST_NOT_OK_FOR_DEBUG_P bpf_const_not_ok_for_debug_p >> >> /* Initialize target-specific function library calls. This is mainly >> used to call library-provided soft-fp operations, since eBPF >> diff --git a/gcc/config/bpf/bpf.md b/gcc/config/bpf/bpf.md >> index e87d72182bb8..0e2ad8da5ace 100644 >> --- a/gcc/config/bpf/bpf.md >> +++ b/gcc/config/bpf/bpf.md >> @@ -374,8 +374,6 @@ >> "" >> " >> { >> - bpf_replace_core_move_operands (operands); >> - >> if (!register_operand(operands[0], <MM:MODE>mode) >> && !register_operand(operands[1], <MM:MODE>mode)) >> operands[1] = force_reg (<MM:MODE>mode, operands[1]); >> @@ -393,7 +391,7 @@ >> {st<mop>\t%0,%1|*(<smop> *) (%0) = %1}" >> [(set_attr "type" "ldx,alu,alu,stx,st")]) >> >> -(define_insn "mov_reloc_core<MM:mode>" >> +(define_insn "*mov_reloc_core<MM:mode>" >> [(set (match_operand:MM 0 "nonimmediate_operand" "=r,q,r") >> (unspec:MM [ >> (match_operand:MM 1 "immediate_operand" " I,I,B") >> diff --git a/gcc/config/bpf/core-builtins.cc >> b/gcc/config/bpf/core-builtins.cc >> index 60d21a8ae739..7a9488c27124 100644 >> --- a/gcc/config/bpf/core-builtins.cc >> +++ b/gcc/config/bpf/core-builtins.cc >> @@ -38,97 +38,96 @@ along with GCC; see the file COPYING3. If not see >> #include "gimple.h" >> #include "gimple-iterator.h" >> #include "gimple-walk.h" >> +#include "gimple-fold.h" >> #include "tree-pass.h" >> #include "plugin.h" >> +#include "gimplify.h" >> >> #include "ctfc.h" >> #include "btf.h" >> #include "coreout.h" >> #include "core-builtins.h" >> >> -/* >> - * BPF CO-RE builtins definition. >> - >> - The expansion of CO-RE builtins occur in three steps: >> - 1. - bpf_resolve_overloaded_core_builtin (pack step) >> - Right after the front-end, all of the CO-RE builtins are converted to >> an >> - internal builtin __builtin_core_reloc, which takes a single argument >> and >> - has polymorphic return value to fit the particular expected return type >> - from the original builtin. The first argument contains an index >> argument >> - which points to the information stored in a vec<struct cr_builtins> >> - which collects the required information from the original CO-RE >> builtin in >> - order to use it later on in the __builtin_core_reloc expansion (the >> next >> - step). >> - >> - 2. - bpf_expand_core_builtin >> - In this step, the __builtin_core_reloc is expanded to a >> unspec:UNSPEC_CORE_RELOC >> - with 3 operands, destination, source and the index. The index operand >> - is the index in the vec constructed in the previous step. >> - >> - 3. - final asm output (process step) >> - This is the output of the unspec:UNSPEC_CORE_RELOC. The index passed in >> - the third operand is read and extracted as a integer from the rtx node. >> - The data is collected from the vec and it is used to create >> - the proper CO-RE relocation as well as do the final assembly output. >> - It also creates a label to mark the location of the move instruction >> that >> - is used in the CO-RE relocation. >> - >> - The initialization of the CO-RE builtins infrastructure occurs in >> - bpf_is function. It creates a struct >> - builtin_helpers_t arrays which defines the kind argument position, >> - the callback helpers, kind, compare, pack and process, for each individual >> - type of builtin argument possible in the original CO-RE builtins. >> - >> - More precisely, field expression, type and enum value, used in the >> following >> - relocations: >> - - __builtin_core_preserve_access_index (<field_expr>) >> - - __builtin_core_field_info (<field_expr>, <kind>) >> - - __builtin_core_type_id (<type>, <kind>) >> - - __builtin_core_type_info (<type>, <kind>) >> - - __builtin_core_enum_value (<enum_value>, <kind>) >> - >> - The kind helper allows to identify the proper relocation for the builtin >> - call based on the value within the kind argument. >> - >> - The compare helper is used to identify if a new builtin call has similar >> - arguments to any other builtin call with the compiling unit. >> - This enables the possibility to optimize consecutive similar calls of the >> - builtins. >> - >> - The pack helper callbacks are suppose to decode the original CO-RE builtin >> - call arguments, verify that it is a valid tree node for the particular >> - builtin, allocate a struct cr_local in vector and write it with the >> - relevant data for the particular builtin type. >> - >> - The process helper should take the data constructed in the pack helper and >> - create a struct cr_final element which contains the essential >> - information to create a CO-RE relocation. >> - This information is further used by the final assembly output step to >> define >> - the CO-RE relocation and pass-through the default value for the original >> - CO-RE builtin. >> - >> - >> - BPF CO-RE preserve access is supported in two forms: >> - - A target builtin, __builtin_preserve_access_index >> - >> - This builtin accepts a single argument. Any access to an aggregate data >> - structure (struct, union or array) within the argument will be recorded >> by >> - the CO-RE machinery, resulting in a relocation record being placed in >> the >> - .BTF.ext section of the output. >> - >> - It is implemented in bpf_resolve_overloaded_builtin () and >> - bpf_expand_builtin (), using the supporting routines below. >> - >> - - An attribute, __attribute__((preserve_access_index)) >> - >> - This attribute can be applied to struct and union types. Any access to >> a >> - type with this attribute will be recorded by the CO-RE machinery. >> - In the expand, any move matching is checked if any of its operands is >> - an expression to an attributed type, and if so, the expand will emit a >> - unspec:UNSPEC_CORE_RELOC that later on, in final assembly output, will >> - create the CO-RE relocation, just like it would happen if it was defined >> - as a builtin. */ >> - >> +/* BPF CO-RE builtins definition. >> + >> + The expansion of CO-RE builtins occur in three steps: >> + 1. - bpf_resolve_overloaded_core_builtin (pack step) >> + Right after the front-end, all of the CO-RE builtins are converted to >> an >> + internal builtin __builtin_core_reloc, which takes a single argument >> and >> + has polymorphic return value to fit the particular expected return >> type >> + from the original builtin. The first argument contains an index >> argument >> + which points to the information stored in a vec<struct cr_builtins> >> which >> + collects the required information from the original CO-RE builtin in >> + order to use it later on in the __builtin_core_reloc expansion (the >> next >> + step). >> + >> + 2. - bpf_expand_core_builtin >> + In this step, the __builtin_core_reloc is expanded to a >> + unspec:UNSPEC_CORE_RELOC with 3 operands, destination, source and the >> + index. The index operand is the index in the vec constructed in the >> + previous step. >> + >> + 3. - final asm output (process step) >> + This is the output of the unspec:UNSPEC_CORE_RELOC. The index passed >> in >> + the third operand is read and extracted as a integer from the rtx >> node. >> + The data is collected from the vec and it is used to create the proper >> + CO-RE relocation as well as do the final assembly output. It also >> + creates a label to mark the location of the move instruction that is >> used >> + in the CO-RE relocation. >> + >> + The initialization of the CO-RE builtins infrastructure occurs in >> + bpf_init_core_builtins function. It creates a struct builtin_helpers_t >> + arrays which defines the kind argument position, the callback helpers, >> + kind, compare, pack and process, for each individual type of builtin >> + argument possible in the original CO-RE builtins. >> + >> + More precisely, field expression, type and enum value, used in the >> + following relocations: >> + - __builtin_core_field_info (<field_expr>, <kind>) >> + - __builtin_core_type_id (<type>, <kind>) >> + - __builtin_core_type_info (<type>, <kind>) >> + - __builtin_core_enum_value (<enum_value>, <kind>) >> + >> + The kind helper allows to identify the proper relocation for the builtin >> + call based on the value within the kind argument. >> + >> + The compare helper is used to identify if a new builtin call has similar >> + arguments to any other builtin call with the compiling unit. This >> enables >> + the possibility to optimize consecutive similar calls of the builtins. >> + >> + The pack helper callbacks are suppose to decode the original CO-RE >> builtin >> + call arguments, verify that it is a valid tree node for the particular >> + builtin, allocate a struct cr_local in vector and write it with the >> + relevant data for the particular builtin type. >> + >> + The process helper should take the data constructed in the pack helper >> and >> + create a struct cr_final element which contains the essential >> information >> + to create a CO-RE relocation. >> + This information is further used by the final assembly output step to >> + define the CO-RE relocation and pass-through the default value for the >> + original CO-RE builtin. >> + >> + BPF CO-RE preserve access is supported in two forms: >> + - A target builtin, __builtin_preserve_access_index >> + >> + This builtin accepts a single argument. Any access to an aggregate >> data >> + structure (struct, union or array), also refered through the document >> as > > typo: referred > >> + an field expression, within the argument will be recorded by the >> CO-RE >> + machinery, resulting in one or more relocations being inserted in the >> + .BTF.ext section of the output. >> + >> + - An attribute, __attribute__((preserve_access_index)) >> + >> + This attribute can be applied to struct and union types. Any access >> done >> + through a node typed with this attribute will be recorded by the CO-RE >> + machinery. This convertion is done in an independent gimple pass very > > typo: conversion > >> + early in compilation, making sure that the field expression is >> + originating from a tree node which his type is attributed. >> + >> + Both these variants of preserve_acces_index rely on a tree walker that > > typo: preserve_access_index > >> + identifies and converts any CO-RE valid field expressions. Apart from >> the >> + gimple specific requirements for the attribute implementation both >> builtin >> + and attribute implementations rely on the same mechanism. */ >> >> struct GTY(()) cr_builtins >> { >> @@ -139,12 +138,13 @@ struct GTY(()) cr_builtins >> enum btf_core_reloc_kind kind; >> enum bpf_builtins orig_builtin_code; >> tree orig_arg_expr; >> + tree access_node; >> }; >> typedef struct cr_builtins *cr_builtins_ref; >> >> #define CORE_BUILTINS_DATA_EMPTY \ >> { NULL_TREE, NULL_TREE, NULL_TREE, NULL_RTX, BPF_RELO_INVALID, \ >> - BPF_BUILTIN_UNUSED, NULL } >> + BPF_BUILTIN_UNUSED, NULL_TREE, NULL_TREE} >> >> /* Vector definition and its access function. */ >> static GTY(()) vec<cr_builtins_ref, va_gc> *builtins_data = NULL; >> @@ -189,7 +189,6 @@ search_builtin_data (builtin_local_data_compare_fn >> callback, >> enum cr_decision >> { >> FAILED_VALIDATION = 0, >> - KEEP_ORIGINAL_NO_RELOCATION, >> REPLACE_CREATE_RELOCATION, >> REPLACE_NO_RELOCATION >> }; >> @@ -286,7 +285,7 @@ compare_same_ptr_type (struct cr_builtins *a, struct >> cr_builtins *b) >> >> /* Handling for __attribute__((preserve_access_index)) for BPF CO-RE >> support. >> >> - This attribute marks a structure/union/array type as "preseve", so that >> + This attribute marks a structure/union/array type as "preserve", so that >> every access to that type should be recorded and replayed by the BPF >> loader; >> this is just the same functionality as __builtin_preserve_access_index, >> but in the form of an attribute for an entire aggregate type. >> @@ -300,7 +299,7 @@ compare_same_ptr_type (struct cr_builtins *a, struct >> cr_builtins *b) >> will record access all the way to 'a', even though struct X does not have >> the preserve_access_index attribute. >> >> - This is to follow LLVM behavior. */ >> + This is to follow LLVM behavior. */ >> >> /* True if tree T accesses any member of a struct/union/class which is >> marked >> with the PRESERVE_ACCESS_INDEX attribute. */ >> @@ -319,10 +318,16 @@ is_attr_preserve_access (tree t) >> tree base = get_inner_reference (t, &bitsize, &bitpos, &var_off, &mode, >> &sign, &reverse, &vol); >> >> + if (TREE_CODE (t) == SSA_NAME >> + || TREE_CODE (t) == VAR_DECL) >> + return lookup_attribute ("preserve_access_index", >> + TYPE_ATTRIBUTES (TREE_TYPE (base))); >> + >> if (TREE_CODE (base) == MEM_REF) >> { >> return lookup_attribute ("preserve_access_index", >> TYPE_ATTRIBUTES (TREE_TYPE (base))); >> + > > Extra blank newline. > >> } >> >> if (TREE_CODE (t) == COMPONENT_REF) >> @@ -336,10 +341,12 @@ is_attr_preserve_access (tree t) >> if (TREE_CODE (op) == COMPONENT_REF) >> return is_attr_preserve_access (op); >> >> + > > Likewise, spurious newline. > >> const tree container = DECL_CONTEXT (TREE_OPERAND (t, 1)); >> >> return lookup_attribute ("preserve_access_index", >> - TYPE_ATTRIBUTES (container)); >> + TYPE_ATTRIBUTES (container)) >> + || is_attr_preserve_access (op); >> } >> >> else if (TREE_CODE (t) == ADDR_EXPR) >> @@ -348,6 +355,25 @@ is_attr_preserve_access (tree t) >> return false; >> } >> >> +static tree >> +root_for_core_field_info (tree node) >> +{ >> + bool done = false; >> + while (!done) >> + { >> + switch (TREE_CODE (node)) >> + { >> + case ADDR_EXPR: >> + case NOP_EXPR: >> + node = TREE_OPERAND (node, 0); >> + break; >> + default: >> + done = true; >> + break; >> + } >> + } >> + return node; >> +} >> >> /* Expand a call to __builtin_preserve_field_info by evaluating the >> requested >> information about SRC according to KIND, and return a tree holding >> @@ -364,6 +390,8 @@ core_field_info (tree src, enum btf_core_reloc_kind kind) >> location_t loc = EXPR_LOCATION (src); >> tree type = TREE_TYPE (src); >> >> + src = root_for_core_field_info (src); >> + >> get_inner_reference (src, &bitsize, &bitpos, &var_off, &mode, &unsignedp, >> &reversep, &volatilep); >> >> @@ -499,7 +527,7 @@ core_field_info (tree src, enum btf_core_reloc_kind kind) >> NODE should be a FIELD_DECL (i.e. of struct or union), or an ARRAY_REF. >> */ >> >> static int >> -bpf_core_get_index (const tree node) >> +bpf_core_get_index (const tree node, bool *valid) >> { >> enum tree_code code = TREE_CODE (node); >> >> @@ -539,66 +567,105 @@ bpf_core_get_index (const tree node) >> } >> } >> >> - gcc_unreachable (); >> + if (valid != NULL) >> + *valid = false; >> return -1; >> } >> >> -/* This function takes a possible field expression (node) and verifies it is >> - valid, extracts what should be the root of the valid field expression and >> - composes the accessors array of indices. The accessors are later used >> in the >> - CO-RE relocation in the string field. */ >> +#define PREPARE_FAKE_PTR(P) \ >> + _fake_##P; \ >> + if (P == NULL) \ >> + P = &_fake_##P; \ >> + _fake_##P >> + >> +#define MAX_NR_ACCESSORS 100 >> + >> +/* This function validates and extracts information for CO-RE field >> expression. >> + Any parametric expression is allowed to be passed in argument NODE. >> + >> + If NODE is a valid CO-RE expression VALID boolean pointer would be set to >> + true. >> + >> + A CO-RE field expression is an expression accessing structures, arrays >> and >> + unions. >> + An examples of CO-RE valid expression is: >> + A->B[2].UNION_C.D >> + >> + This function traverses the tree structure to verify if the expression in >> + NODE is valid and extracts other characteristics of the expression, and >> + returns it by updating the pointer arguments: >> + >> + ACCESSORS: is an array with the indexes of the particular fields >> + within the expression. The indexes are related to actual index on the >> + struct/union type, or the array access index. The RETURN of the >> function >> + is the number of accessors required to represent this expression in >> CO-RE >> + access string. >> + VALID - boolean pointer that sets if expression is valid or not for >> CO-RE. >> + ACCESS_NODE - It is the base of the expression. Using the example below >> is >> + the node that represents the A in the expression. >> + >> + ALLOW_ENTRY_CAST is an input argumenta and specifies if the function >> should > > typo: argumenta -> arguments > >> + consider as valid expressions in which NODE entry is a cast expression >> (or >> + tree code nop_expr). >> + */ > > nit: this */ should be on the preceding line. > >> >> static unsigned char >> -compute_field_expr (tree node, unsigned int *accessors, bool *valid, >> - tree *root) >> +compute_field_expr (tree node, unsigned int *accessors, >> + bool *valid, >> + tree *access_node, >> + bool allow_entry_cast = true) >> { >> unsigned char n = 0; >> + unsigned int fake_accessors[MAX_NR_ACCESSORS]; >> + if (accessors == NULL) >> + accessors = fake_accessors; >> + bool PREPARE_FAKE_PTR (valid) = true; >> + tree PREPARE_FAKE_PTR (access_node) = NULL_TREE; >> + >> if (node == NULL_TREE) >> { >> *valid = false; >> return 0; >> } >> >> + *access_node = node; >> + >> switch (TREE_CODE (node)) >> { >> - case INDIRECT_REF: >> case ADDR_EXPR: >> + return 0; >> + case INDIRECT_REF: >> accessors[0] = 0; >> - n = compute_field_expr (TREE_OPERAND (node, 0), &accessors[0], valid, >> - root); >> - *root = node; >> - return n + 1; >> + return 1; >> case POINTER_PLUS_EXPR: >> - accessors[0] = bpf_core_get_index (node); >> - *root = node; >> + accessors[0] = bpf_core_get_index (node, valid); >> return 1; >> case COMPONENT_REF: >> - n = compute_field_expr (TREE_OPERAND (node, 0), accessors, valid, >> - root); >> - accessors[n] = bpf_core_get_index (TREE_OPERAND (node, 1)); >> - *root = node; >> + n = compute_field_expr (TREE_OPERAND (node, 0), accessors, >> + valid, >> + access_node, false); >> + accessors[n] = bpf_core_get_index (TREE_OPERAND (node, 1), valid); >> return n + 1; >> case ARRAY_REF: >> case ARRAY_RANGE_REF: >> case MEM_REF: >> - n = compute_field_expr (TREE_OPERAND (node, 0), accessors, valid, >> root); >> - accessors[n] = bpf_core_get_index (node); >> - *root = node; >> - return n + 1; >> + n = compute_field_expr (TREE_OPERAND (node, 0), accessors, >> + valid, >> + access_node, false); >> + accessors[n++] = bpf_core_get_index (node, valid); >> + return n; >> case NOP_EXPR: >> - n = compute_field_expr (TREE_OPERAND (node, 0), accessors, valid, >> root); >> - *root = node; >> + if (allow_entry_cast == true) >> + { >> + *valid = false; >> + return 0; >> + } >> + n = compute_field_expr (TREE_OPERAND (node, 0), accessors, >> + valid, >> + access_node, false); >> return n; >> - case TARGET_EXPR: >> - { >> - tree value = TREE_OPERAND (node, 1); >> - if (TREE_CODE (value) == BIND_EXPR >> - && TREE_CODE (value = BIND_EXPR_BODY (value)) == MODIFY_EXPR) >> - return compute_field_expr (TREE_OPERAND (value, 1), accessors, valid, >> - root); >> - } >> - *root = node; >> - return 0; >> + >> + case CALL_EXPR: >> case SSA_NAME: >> case VAR_DECL: >> case PARM_DECL: >> @@ -608,81 +675,46 @@ compute_field_expr (tree node, unsigned int >> *accessors, bool *valid, >> return 0; >> } >> } >> +#undef PREPARE_FAKE_PTR >> >> -static struct cr_local >> -pack_field_expr_for_access_index (tree *args, >> - enum btf_core_reloc_kind kind, >> - enum bpf_builtins code ATTRIBUTE_UNUSED) >> -{ >> - struct cr_local ret = CR_LOCAL_EMPTY; >> - ret.fail = false; >> - >> - tree arg = args[0]; >> - tree root = arg; >> - >> - /* Avoid double-recording information if argument is an access to >> - a struct/union marked __attribute__((preserve_access_index)). This >> - Will be handled by the attribute handling pass. */ >> - if (is_attr_preserve_access (arg)) >> - { >> - ret.reloc_decision = REPLACE_NO_RELOCATION; >> - ret.reloc_data.expr = arg; >> - } >> - else >> - { >> - ret.reloc_decision = REPLACE_CREATE_RELOCATION; >> >> - unsigned int accessors[100]; >> - bool valid = true; >> - compute_field_expr (arg, accessors, &valid, &root); >> - >> - if (valid == true) >> - ret.reloc_data.expr = root; >> - else >> - { >> - bpf_error_at (EXPR_LOC_OR_LOC (arg, UNKNOWN_LOCATION), >> - "Cannot compute index for field argument"); >> - ret.fail = true; >> - } >> - } >> - >> - /* Note: the type of default_value is used to define the return type of >> - __builtin_core_reloc in bpf_resolve_overloaded_core_builtin. */ >> - ret.reloc_data.type = TREE_TYPE (root); >> - ret.reloc_data.default_value = build_int_cst (ret.reloc_data.type, 0); >> - ret.reloc_data.kind = kind; >> - >> - if (TREE_CODE (ret.reloc_data.default_value) == ERROR_MARK) >> - ret.fail = true; >> - >> - return ret; >> -} >> +/* Pack helper for the __builtin_preserve_field_info. */ >> >> static struct cr_local >> -pack_field_expr_for_preserve_field (tree *args, >> - enum btf_core_reloc_kind kind, >> - enum bpf_builtins code ATTRIBUTE_UNUSED) >> +pack_field_expr (tree *args, >> + enum btf_core_reloc_kind kind, >> + enum bpf_builtins code ATTRIBUTE_UNUSED) >> { >> struct cr_local ret = CR_LOCAL_EMPTY; >> ret.fail = false; >> >> tree arg = args[0]; >> - tree tmp; >> tree root = arg; >> + tree access_node = NULL_TREE; >> + tree type = NULL_TREE; >> >> - /* Remove cast to void * created by front-end to fit builtin type, when >> passed >> - * a simple expression like f->u. */ >> - if (TREE_CODE (arg) == NOP_EXPR && (tmp = TREE_OPERAND (arg, 0)) >> - && TREE_CODE (tmp) == ADDR_EXPR && (tmp = TREE_OPERAND (tmp, 0)) >> - && arg != NULL_TREE) >> - arg = tmp; >> + ret.reloc_decision = REPLACE_CREATE_RELOCATION; >> >> unsigned int accessors[100]; >> bool valid = true; >> - compute_field_expr (arg, accessors, &valid, &root); >> + compute_field_expr (root, accessors, &valid, &access_node, false); >> + >> + type = TREE_TYPE (access_node); >> >> if (valid == true) >> - ret.reloc_data.expr = root; >> + { >> + ret.reloc_data.expr = root; >> + >> + /* Note: the type of default_value is used to define the return type >> of >> + __builtin_core_reloc in bpf_resolve_overloaded_core_builtin. */ >> + ret.reloc_data.access_node = access_node; >> + ret.reloc_data.type = type; >> + ret.reloc_data.default_value = core_field_info (root, kind); >> + ret.reloc_data.kind = kind; >> + >> + if (TREE_CODE (ret.reloc_data.default_value) == ERROR_MARK) >> + ret.fail = true; >> + } >> else >> { >> bpf_error_at (EXPR_LOC_OR_LOC (arg, UNKNOWN_LOCATION), >> @@ -690,17 +722,11 @@ pack_field_expr_for_preserve_field (tree *args, >> ret.fail = true; >> } >> >> - ret.reloc_decision = REPLACE_CREATE_RELOCATION; >> - ret.reloc_data.type = TREE_TYPE (root); >> - ret.reloc_data.default_value = core_field_info (root, kind); >> - ret.reloc_data.kind = kind; >> - >> - if (TREE_CODE (ret.reloc_data.default_value) == ERROR_MARK) >> - ret.fail = true; >> - >> return ret; >> } >> >> +/* Process helper for the __builtin_preserve_field_info. */ >> + >> static struct cr_final >> process_field_expr (struct cr_builtins *data) >> { >> @@ -711,19 +737,18 @@ process_field_expr (struct cr_builtins *data) >> || data->kind == BPF_RELO_FIELD_SIGNED >> || data->kind == BPF_RELO_FIELD_EXISTS); >> >> - unsigned int accessors[100]; >> + unsigned int accessors[MAX_NR_ACCESSORS]; >> unsigned char nr_accessors = 0; >> - bool valid = true; >> - tree root = NULL_TREE; >> tree expr = data->expr; >> - tree type = TREE_TYPE (data->expr); >> + tree type = data->type; >> >> if (TREE_CODE (expr) == ADDR_EXPR) >> expr = TREE_OPERAND (expr, 0); >> >> - nr_accessors = compute_field_expr (expr, accessors, &valid, &root); >> + expr = root_for_core_field_info (expr); >> + nr_accessors = compute_field_expr (expr, accessors, NULL, NULL, false); >> >> - struct cr_final ret = { NULL, type, data->kind}; >> + struct cr_final ret = { NULL, type, data->kind }; >> >> char str[100]; >> if (nr_accessors > 0) >> @@ -740,9 +765,11 @@ process_field_expr (struct cr_builtins *data) >> return ret; >> } >> >> -hash_map <tree, tree> bpf_enum_mappings; >> - >> +static GTY(()) hash_map<tree, tree> *bpf_enum_mappings; >> tree enum_value_type = NULL_TREE; >> + >> +/* Pack helper for the __builtin_preserve_enum_value. */ >> + >> static struct cr_local >> pack_enum_value (tree *args, enum btf_core_reloc_kind kind, >> enum bpf_builtins code ATTRIBUTE_UNUSED) >> @@ -757,7 +784,7 @@ pack_enum_value (tree *args, enum btf_core_reloc_kind >> kind, >> tree type = NULL_TREE; >> >> /* Deconstructing "*(typeof (enum_type) *) enum_value" to collect both the >> - * enum_type and enum_value. */ >> + enum_type and enum_value. */ >> if (TREE_CODE (tmp) != TARGET_EXPR >> || (type = TREE_TYPE (tmp)) == NULL_TREE >> || (TREE_CODE (type) != POINTER_TYPE) >> @@ -771,7 +798,7 @@ pack_enum_value (tree *args, enum btf_core_reloc_kind >> kind, >> if (TREE_CODE (enum_value) != INTEGER_CST) >> goto pack_enum_value_fail; >> >> - result = bpf_enum_mappings.get (enum_value); >> + result = bpf_enum_mappings->get (enum_value); >> if (result == NULL) >> goto pack_enum_value_fail; >> >> @@ -797,6 +824,8 @@ pack_enum_value_fail: >> return ret; >> } >> >> +/* Process helper for the __builtin_preserve_enum_value. */ >> + >> static struct cr_final >> process_enum_value (struct cr_builtins *data) >> { >> @@ -829,6 +858,8 @@ process_enum_value (struct cr_builtins *data) >> return ret; >> } >> >> +/* Pack helper for the __builtin_preserve_type_info. */ >> + >> static struct cr_local >> pack_type (tree *args, enum btf_core_reloc_kind kind, >> enum bpf_builtins code ATTRIBUTE_UNUSED) >> @@ -842,12 +873,12 @@ pack_type (tree *args, enum btf_core_reloc_kind kind, >> tree tmp = args[0]; >> HOST_WIDE_INT type_size_i; >> >> + if (TYPE_P (tmp)) >> + goto is_already_type; >> /* Typical structure to match: >> - * *({ extern typeof (TYPE) *<tmp_name>; <tmp_name>; }) >> - */ >> + *({ extern typeof (TYPE) *<tmp_name>; <tmp_name>; }) */ >> >> /* Extract Pointer dereference from the construct. */ >> - >> while (tmp != NULL_TREE >> && (TREE_CODE (tmp) == INDIRECT_REF >> || TREE_CODE (tmp) == NOP_EXPR)) >> @@ -865,6 +896,7 @@ pack_type (tree *args, enum btf_core_reloc_kind kind, >> >> tmp = TREE_TYPE (tmp); >> >> +is_already_type: >> if (TREE_CODE (tmp) == POINTER_TYPE) >> tmp = TREE_TYPE (tmp); >> >> @@ -884,7 +916,7 @@ pack_type (tree *args, enum btf_core_reloc_kind kind, >> ret.reloc_data.type = root_type; >> ret.reloc_decision = REPLACE_CREATE_RELOCATION; >> >> - /* Force this type to be marked as used in dwarf2out. */ >> + /* Force this type to be marked as used in dwarf2out. */ >> gcc_assert (cfun); >> if (cfun->used_types_hash == NULL) >> cfun->used_types_hash = hash_set<tree>::create_ggc (37); >> @@ -915,11 +947,13 @@ pack_type (tree *args, enum btf_core_reloc_kind kind, >> >> pack_type_fail: >> bpf_error_at (EXPR_LOC_OR_LOC (args[0], UNKNOWN_LOCATION), >> - "invelid first argument format for enum value builtin"); >> + "invalid first argument format for enum value builtin"); >> ret.fail = true; >> return ret; >> } >> >> +/* Process helper for the __builtin_preserve_type_info. */ >> + >> static struct cr_final >> process_type (struct cr_builtins *data) >> { >> @@ -976,7 +1010,7 @@ make_core_relo (struct cr_final *data, rtx_code_label >> *label) >> /* Determine what output section this relocation will apply to. >> If this function is associated with a section, use that. Otherwise, >> fall back on '.text'. */ >> - const char * section_name; >> + const char section_name; > > Why this change from char* to char? > > This change doesn't make sense given the expressions assigned to > the variable. In fact, this does not compile. > >> if (current_function_decl && DECL_SECTION_NAME (current_function_decl)) >> section_name = DECL_SECTION_NAME (current_function_decl); >> else >> @@ -1075,30 +1109,8 @@ kind_preserve_type_info (tree *args, int nargs) >> return BPF_RELO_INVALID; >> } >> >> - >> -/* Required to overcome having different return type builtins to avoid >> warnings >> - at front-end and be able to share the same builtin definition and >> permitting >> - the PURE attribute to work. */ >> -hash_map<tree, tree> core_builtin_type_defs; >> - >> -static tree >> -get_core_builtin_fndecl_for_type (tree ret_type) >> -{ >> - tree *def = core_builtin_type_defs.get (ret_type); >> - if (def) >> - return *def; >> - >> - tree rettype = build_function_type_list (ret_type, integer_type_node, >> NULL); >> - tree new_fndecl = add_builtin_function_ext_scope ("__builtin_core_reloc", >> - rettype, >> - BPF_BUILTIN_CORE_RELOC, >> - BUILT_IN_MD, NULL, NULL); >> - DECL_PURE_P (new_fndecl) = 1; >> - >> - core_builtin_type_defs.put (ret_type, new_fndecl); >> - >> - return new_fndecl; >> -} >> +/* Plugin handler used in the parser that allows to collect enum value >> + information that other wise would be folded and non recoverable. */ >> >> void >> bpf_handle_plugin_finish_type (void *event_data, >> @@ -1106,6 +1118,9 @@ bpf_handle_plugin_finish_type (void *event_data, >> { >> tree type = (tree) event_data; >> >> + if (bpf_enum_mappings == NULL) >> + bpf_enum_mappings = hash_map<tree, tree>::create_ggc (10); >> + >> if (TREE_CODE (type) == ENUMERAL_TYPE) >> for (tree l = TYPE_VALUES (type); l; l = TREE_CHAIN (l)) >> { >> @@ -1115,12 +1130,12 @@ bpf_handle_plugin_finish_type (void *event_data, >> initial = copy_node (initial); >> DECL_INITIAL (value) = initial; >> >> - bpf_enum_mappings.put (initial, value); >> + tree *found = bpf_enum_mappings->get (initial); >> + if (found == NULL) >> + bpf_enum_mappings->put (initial, value); >> } >> } >> >> -/* -- Header file exposed functions -- */ >> - >> /* Initializes support information to process CO-RE builtins. >> Defines information for the builtin processing, such as helper functions >> to >> support the builtin convertion. */ >> @@ -1133,13 +1148,13 @@ bpf_init_core_builtins (void) >> core_builtin_helpers[BPF_BUILTIN_PRESERVE_ACCESS_INDEX] = >> BPF_CORE_HELPER_SET (kind_access_index, >> NULL, >> - pack_field_expr_for_access_index, >> - process_field_expr, >> + NULL, >> + NULL, >> true); >> core_builtin_helpers[BPF_BUILTIN_PRESERVE_FIELD_INFO] = >> BPF_CORE_HELPER_SET (kind_preserve_field_info, >> NULL, >> - pack_field_expr_for_preserve_field, >> + pack_field_expr, >> process_field_expr, >> true); >> core_builtin_helpers[BPF_BUILTIN_BTF_TYPE_ID] = >> @@ -1167,13 +1182,17 @@ bpf_init_core_builtins (void) >> BPF_CORE_HELPER_SET (NULL, NULL, NULL, NULL, true); >> >> /* Initialize plugin handler to record enums value for use in >> - * __builtin_preserve_enum_value. */ >> + __builtin_preserve_enum_value. */ >> plugin_state = (enum bpf_plugin_states) flag_plugin_added; >> flag_plugin_added = true; >> register_callback ("bpf_collect_enum_info", PLUGIN_FINISH_TYPE, >> bpf_handle_plugin_finish_type, NULL); >> } >> >> +/* This function returns the related __builtin_core_reloc call tree node to >> a >> + particular CO-RE builtin definition in FNDECL when called with >> + arguments ARGS. */ >> + >> static tree >> construct_builtin_core_reloc (location_t loc, tree fndecl, tree *args, >> int nargs) >> @@ -1200,14 +1219,12 @@ construct_builtin_core_reloc (location_t loc, tree >> fndecl, tree *args, >> local_data.reloc_data.orig_arg_expr = args[0]; >> } >> else >> - local_data.reloc_decision = KEEP_ORIGINAL_NO_RELOCATION; >> + gcc_unreachable (); >> >> if (local_data.fail == true) >> return error_mark_node; >> >> - if (local_data.reloc_decision == REPLACE_NO_RELOCATION) >> - return local_data.reloc_data.expr; >> - else if (local_data.reloc_decision == REPLACE_CREATE_RELOCATION) >> + if (local_data.reloc_decision == REPLACE_CREATE_RELOCATION) >> { >> int index = search_builtin_data (helper.compare, >> &local_data.reloc_data); >> @@ -1216,37 +1233,253 @@ construct_builtin_core_reloc (location_t loc, tree >> fndecl, tree *args, >> struct cr_builtins *data = get_builtin_data (index); >> memcpy (data, &local_data.reloc_data, sizeof (struct cr_builtins)); >> >> - tree new_fndecl = bpf_builtins[BPF_BUILTIN_CORE_RELOC]; >> + tree fndecl = bpf_builtins[BPF_BUILTIN_CORE_RELOC]; >> + return build_call_expr_loc (loc, >> + fndecl, 1, >> + build_int_cst (integer_type_node, index)); >> + } >> + } >> + return NULL_TREE; >> +} >> + >> +/* This function constructs the CO-RE safe code around field expressions. >> + If the expression is an array access, create an offset by multiplying the >> + index access by the __builtin_type_info call requesting the size of the >> + array element and multiplying by the index. The offset is added to the >> + base pointer. >> + In a more formal way: >> + - base + (__blt_preserve_type_info (typeof(expr), SIZEOF) * array_i) >> + >> + Please notice that __builtin_preserve_type_info is never really created, >> but >> + rather a call to __builtin_core_reloc that represents it. >> + >> + Any other case, it is assumed to be a field access and instead a CO-RE >> field >> + expressions offset relocation is created and added to the base node. >> + More precisely: >> + - base + __builtin_preserve_field_expr (expr, SIZEOF) */ >> + >> +static tree >> +core_expr_with_field_expr_plus_base (tree base, tree expr, bool leaf_node) >> +{ >> + tree type = TREE_TYPE (expr); >> + tree args[2]; >> + >> + if (base == expr) >> + return expr; >> + else if (TREE_CODE (expr) == ARRAY_REF >> + && leaf_node == false) >> + { >> + if (TREE_CODE (base) == MEM_REF) >> + base = TREE_OPERAND (base, 0); >> + >> + tree array_index = TREE_OPERAND (expr, 1); >> + tree fndecl = bpf_builtins[BPF_BUILTIN_PRESERVE_TYPE_INFO]; >> + >> + tree type = TREE_TYPE (base); >> + gcc_assert (POINTER_TYPE_P (type) >> + && TREE_CODE (type = TREE_TYPE (type)) == ARRAY_TYPE >> + && (type = TREE_TYPE (type)) != NULL_TREE); >> + args[0] = type; >> + args[1] = build_int_cst (integer_type_node, BPF_TYPE_SIZE); >> + tree builtin_call = construct_builtin_core_reloc (UNKNOWN_LOCATION, >> + fndecl, >> + args, 2); >> + >> + tree offset = fold_build2 (MULT_EXPR, size_type_node, >> + fold_build1 (NOP_EXPR, size_type_node, builtin_call), >> + fold_build1 (NOP_EXPR, size_type_node, array_index)); >> + >> + if (!POINTER_TYPE_P (TREE_TYPE (base))) >> + base = fold_build1 (ADDR_EXPR, >> + build_pointer_type (TREE_TYPE (base)), base); >> + >> + tree tmp = fold_build2 (POINTER_PLUS_EXPR, ptr_type_node, >> + fold_build1 (NOP_EXPR, ptr_type_node, base), >> + offset); >> + >> + tmp = fold_build1 (NOP_EXPR, build_pointer_type (type), tmp); >> + return tmp; >> + } >> + else >> + { >> + tree fndecl = bpf_builtins[BPF_BUILTIN_PRESERVE_FIELD_INFO]; >> + args[0] = expr; >> + args[1] = build_int_cst (integer_type_node, BPF_FIELD_BYTE_OFFSET); >> + tree builtin_call = construct_builtin_core_reloc (UNKNOWN_LOCATION, >> + fndecl, >> + args, 2); >> + >> + if (!POINTER_TYPE_P (TREE_TYPE (base))) >> + base = fold_build1 (ADDR_EXPR, >> + build_pointer_type (TREE_TYPE (base)), base); >> + >> + tree tmp = fold_build2 (POINTER_PLUS_EXPR, ptr_type_node, >> + fold_build1 (NOP_EXPR, ptr_type_node, base), >> + fold_build1 (NOP_EXPR, size_type_node, builtin_call)); >> + tmp = fold_build1 (NOP_EXPR, build_pointer_type (type), tmp); >> + return tmp; >> + } >> +} >> + >> +/* This function takes arbitrary field expression and returns a CO-RE >> + compatible version by introducing CO-RE relocations. >> + >> + In cases where the expression is not supported in CO-RE with a single >> + relocation, it creates multiple levels of access, i.e. if the expression >> + contains multiple indirections. >> + For example: >> + - A->B->C >> + >> + It recursively traverses the expression to its leaf nodes and rollbacks >> + constructing the CO-RE relocations. It calls> + >> core_expr_with_field_expr_plus_base that creates the necessary CO-RE >> + relocations. >> + >> + Arguments: >> + - EXPR: is the expression to be converted. It should be validated by >> + compute_field_expr function before this function is called. >> + - CHANGED: is set to true if the returned tree node is different from >> + the input expr argument. >> + - ENTRY: internal only. Should not be set in a call. */ >> + >> +static tree >> +make_core_safe_access_index (tree expr, bool *changed, bool entry = true) >> +{ >> + poly_int64 bitsize, bitpos; >> + tree var_off; >> + machine_mode mode; >> + int sign, reverse, vol; >> + >> + tree base = get_inner_reference (expr, &bitsize, &bitpos, &var_off, &mode, >> + &sign, &reverse, &vol); >> + >> + if (base == NULL_TREE || base == expr) >> + return expr; >> + >> + tree ret = NULL_TREE; >> + int n; >> + bool valid = true; >> + tree access_node = NULL_TREE; >> + >> + /* In case the base is itself a valid field expression, first convert the >> + base in a CO-RE safe expression. >> + This seems to be a requirement since get_inner_reference not always >> + returns the true base of the expression. */ >> + if ((n = compute_field_expr (base, NULL, &valid, &access_node)) > 0 >> + && valid == true) >> + { >> + if (TREE_CODE (access_node) == INDIRECT_REF) >> + base = TREE_OPERAND (access_node, 0); >> + >> + bool local_changed = false; >> + ret = make_core_safe_access_index (base, &local_changed, false); >> + if (local_changed == true) >> + { >> + if (TREE_CODE (access_node) == INDIRECT_REF) >> + base = fold_build1 (INDIRECT_REF, >> + TREE_TYPE (base), >> + ret); >> + else >> + base = ret; >> + } >> + } >> + >> + /* The remaining is to traverse the field part of the field expression. >> */ >> + if (mode != VOIDmode && var_off == NULL_TREE) >> + { >> + *changed = true; >> + return core_expr_with_field_expr_plus_base (base, expr, true); >> + } >> + else >> + { >> + switch (TREE_CODE (expr)) >> + { >> + case COMPONENT_REF: >> + case ARRAY_REF: >> + case ADDR_EXPR: >> + { >> + bool local_changed = false; >> + tree type = TREE_TYPE (TREE_OPERAND (expr, 0)); >> + ret = make_core_safe_access_index (TREE_OPERAND (expr, 0), >> + &local_changed, false); >> + >> + /* This variable is a replaced in the expr for algorithmic purposes. >> + It reduces the expression just to the remaining sub-expression >> + that still was not processed. */ >> + if (local_changed == true) >> + TREE_OPERAND (expr, 0) = create_tmp_var (type, "fake"); >> + } >> + break; >> + default: >> + ret = expr; >> + break; >> + } >> + } >> + >> + base = get_inner_reference (expr, &bitsize, &bitpos, &var_off, &mode, >> + &sign, &reverse, &vol); >> + >> + if ((ret != NULL_TREE >> + && mode != VOIDmode && var_off != NULL_TREE) >> + || entry == true)> + { >> + *changed = true; >> + ret = core_expr_with_field_expr_plus_base (ret, expr, false); >> + } >> + return ret; >> +} >> + >> +/* This function verifies if the NODE expression is a field expression and >> + changes and converts it to CO-RE. This is used by a tree walker for any >> + __builtin_preserve_access_index argument expression from within >> + bpf_resolve_overloaded_core_builtin. */ >> + >> +static tree >> +replace_core_access_index_comp_expr (tree *node, int *walk_subtrees, >> + void *data ATTRIBUTE_UNUSED) >> +{ >> + bool valid = true; >> + gcc_assert (*node != NULL_TREE); >> + >> + tree *expr = node; >> + bool should_indirect = false; >> + if (TREE_CODE (*expr) == ADDR_EXPR) >> + expr = &TREE_OPERAND (*expr, 0); >> + else >> + should_indirect = true; >> + >> + int n = compute_field_expr (*node, NULL, &valid, NULL); >> + if (valid == true && n > 0) >> + { >> + bool changed = false; >> + tree expr_test = make_core_safe_access_index (*expr, &changed); >> + *walk_subtrees = 0; >> + >> + if (changed == true) >> + { >> + if (should_indirect == true) >> + expr_test = fold_build1 (INDIRECT_REF, >> + TREE_TYPE (TREE_TYPE (expr_test)), >> + expr_test); >> >> - tree ret_type = TREE_TYPE (local_data.reloc_data.default_value); >> - if (ret_type != ptr_type_node) >> - new_fndecl = get_core_builtin_fndecl_for_type (ret_type); >> - return build_call_expr_loc (loc, >> - new_fndecl, 1, >> - build_int_cst (integer_type_node, >> - index)); >> + *expr = expr_test; >> } >> } >> return NULL_TREE; >> } >> >> -/* This function is used by bpf_resolve_overloaded_builtin which is the >> - implementation of the TARGET_RESOLVE_OVERLOADED_BUILTIN. It is executed >> in >> - a very early stage and allows to adapt the builtin to different arguments >> - allowing the compiler to make builtins polymorphic. In this particular >> - implementation, it collects information of the specific builtin call, >> - converts it to the internal __builtin_core_reloc, stores any required >> - information from the original builtin call in a vec<cr_builtins> and >> assigns >> - the index within the *vec*, replacing by __builtin_core_reloc. In the >> - process we also adjust return type of the __builtin_core_reloc to permit >> - polymorphic return type, as it is expected in some of the BPF CO-RE >> - builtins. */ >> +/* This function is used by bpf_resolve_overloaded_builtin defined in >> bpf.cc. >> + It is executed in a very early stage and processes any CO-RE builtins, >> + adapting the code and creating the more generic __builtin_core_reloc >> calls. >> + */ >> >> #define MAX_CORE_BUILTIN_ARGS 3 >> tree >> bpf_resolve_overloaded_core_builtin (location_t loc, tree fndecl, >> void *arglist) >> { >> + remove_parser_plugin (); >> + >> if (!bpf_require_core_support ()) >> return error_mark_node; >> >> @@ -1255,29 +1488,43 @@ bpf_resolve_overloaded_core_builtin (location_t loc, >> tree fndecl, >> for (unsigned int i = 0; i < argsvec->length (); i++) >> args[i] = (*argsvec)[i]; >> >> - remove_parser_plugin (); >> + int code = DECL_MD_FUNCTION_CODE (fndecl); >> + if (code == BPF_BUILTIN_PRESERVE_ACCESS_INDEX) >> + { >> + walk_tree (&args[0], replace_core_access_index_comp_expr, NULL, NULL); >> + return args[0]; >> + } >> >> return construct_builtin_core_reloc (loc, fndecl, args, argsvec->length >> ()); >> } >> >> /* Used in bpf_expand_builtin. This function is called in RTL expand stage >> to >> convert the internal __builtin_core_reloc in unspec:UNSPEC_CORE_RELOC >> RTL, >> - which will contain a third argument that is the index in the vec >> collected in >> - bpf_resolve_overloaded_core_builtin. */ >> + which will contain a third argument that is the index in the vec >> collected >> + in bpf_resolve_overloaded_core_builtin. */ >> >> rtx >> bpf_expand_core_builtin (tree exp, enum bpf_builtins code) >> { >> - if (code == BPF_BUILTIN_CORE_RELOC) >> + if (!TARGET_BPF_CORE) >> + return NULL_RTX; >> + >> + switch (code) >> { >> - tree index = CALL_EXPR_ARG (exp, 0); >> - struct cr_builtins *data = get_builtin_data (TREE_INT_CST_LOW >> (index)); >> - >> - rtx v = expand_normal (data->default_value); >> - rtx i = expand_normal (index); >> - return gen_rtx_UNSPEC (DImode, >> - gen_rtvec (2, v, i), >> - UNSPEC_CORE_RELOC); >> + case BPF_BUILTIN_CORE_RELOC: >> + { >> + tree index = CALL_EXPR_ARG (exp, 0); >> + struct cr_builtins *data = get_builtin_data (TREE_INT_CST_LOW (index)); >> + >> + rtx v = expand_normal (data->default_value); >> + rtx i = expand_normal (index); >> + return gen_rtx_UNSPEC (DImode, >> + gen_rtvec (2, v, i), >> + UNSPEC_CORE_RELOC); >> + } >> + break; >> + default: >> + break; >> } >> >> return NULL_RTX; >> @@ -1287,8 +1534,8 @@ bpf_expand_core_builtin (tree exp, enum bpf_builtins >> code) >> unspec:UNSPEC_CORE_RELOC. It recovers the vec index kept as the third >> operand and collects the data from the vec. With that it calls the >> process >> helper in order to construct the data required for the CO-RE relocation. >> - Also it creates a label pointing to the unspec instruction and uses it >> - in the CO-RE relocation creation. */ >> + Also it creates a label pointing to the unspec instruction and uses it in >> + the CO-RE relocation creation. */ >> >> const char * >> bpf_add_core_reloc (rtx *operands, const char *templ) >> @@ -1297,7 +1544,7 @@ bpf_add_core_reloc (rtx *operands, const char *templ) >> builtin_helpers helper; >> helper = core_builtin_helpers[data->orig_builtin_code]; >> >> - rtx_code_label * tmp_label = gen_label_rtx (); >> + rtx_code_label tmp_label = gen_label_rtx (); > > Hm, again why drop the '*'? This doesn't make sense. > Maybe your editor did something weird? > > extern rtx_code_label *gen_label_rtx (void); > >> output_asm_label (tmp_label); >> assemble_name (asm_out_file, ":\n"); >> >> @@ -1306,67 +1553,196 @@ bpf_add_core_reloc (rtx *operands, const char >> *templ) >> make_core_relo (&reloc_data, tmp_label); >> >> /* Replace default value for later processing builtin types. >> - Example if the type id builtins. */ >> + Example if the type id builtins. */ >> if (data->rtx_default_value != NULL_RTX) >> operands[1] = data->rtx_default_value; >> >> return templ; >> } >> >> -/* This function is used within the defined_expand for mov in bpf.md file. >> - It identifies if any of the operands in a move is a expression with a >> - type with __attribute__((preserve_access_index)), which case it >> - will emit an unspec:UNSPEC_CORE_RELOC such that it would later create a >> - CO-RE relocation for this expression access. */ >> +static tree >> +maybe_get_base_for_field_expr (tree expr) >> +{ >> + poly_int64 bitsize, bitpos; >> + tree var_off; >> + machine_mode mode; >> + int sign, reverse, vol; >> >> -void >> -bpf_replace_core_move_operands (rtx *operands) >> + if (expr == NULL_TREE) >> + return NULL_TREE; >> + >> + return get_inner_reference (expr, &bitsize, &bitpos, &var_off, &mode, >> + &sign, &reverse, &vol); >> +} >> + >> +/* Access functions to mark sub expressions as attributed with >> + __preserve_access_index. >> + This is required since in gimple format, in order to convert an >> expression as >> + CO-RE safe, we must create multiple gimple statements. >> + Also, only the type of the base of the expression might be attributed >> with >> + __preserve_access_index. Nevertheless all the consecutive accesses to >> this >> + attributed node should also be converted to CO-RE safe. >> + Any LHS assigned values with CO-RE converted expressions are marked and >> + any uses of these values are later checked for further convertion. >> + The core_access_index_map functions allow to mark this nodes for later >> + convertion to CO-RE. >> + This mechanism are used by make_gimple_core_safe_access_index. */ >> + >> +static GTY(()) hash_map<tree, tree> *core_access_index_map = NULL; >> + >> +static void >> +core_access_clean (void) >> { >> - for (int i = 0; i < 2; i++) >> - if (MEM_P (operands[i])) >> - { >> - tree expr = MEM_EXPR (operands[i]); >> + if (core_access_index_map == NULL) >> + core_access_index_map = hash_map<tree, tree>::create_ggc (10); >> + core_access_index_map->empty (); >> +} >> >> - if (expr == NULL_TREE) >> - continue; >> +static bool >> +core_is_access_index (tree expr) >> +{ >> + if (TREE_CODE (expr) == MEM_REF >> + || TREE_CODE (expr) == INDIRECT_REF) >> + expr = TREE_OPERAND (expr, 0); >> >> - if (TREE_CODE (expr) == MEM_REF >> - && TREE_CODE (TREE_OPERAND (expr, 0)) == SSA_NAME) >> - { >> - gimple *def_stmt = SSA_NAME_DEF_STMT (TREE_OPERAND (expr, 0)); >> - if (def_stmt && is_gimple_assign (def_stmt)) >> - expr = gimple_assign_rhs1 (def_stmt); >> - } >> - if (is_attr_preserve_access (expr) >> - && bpf_require_core_support ()) >> - { >> - struct cr_local local_data = pack_field_expr_for_access_index ( >> - &expr, >> - BPF_RELO_FIELD_BYTE_OFFSET, >> - BPF_BUILTIN_PRESERVE_ACCESS_INDEX); >> + tree *def = core_access_index_map->get (expr); >> + if (def) >> + return true; >> + return false; >> +} >> >> - local_data.reloc_decision = REPLACE_CREATE_RELOCATION; >> - local_data.reloc_data.orig_arg_expr = expr; >> - local_data.reloc_data.orig_builtin_code = >> BPF_BUILTIN_PRESERVE_ACCESS_INDEX; >> +static void >> +core_mark_as_access_index (tree expr) >> +{ >> + if (TREE_CODE (expr) == MEM_REF >> + || TREE_CODE (expr) == INDIRECT_REF) >> + expr = TREE_OPERAND (expr, 0); >> >> - int index = allocate_builtin_data (); >> - struct cr_builtins *data = get_builtin_data (index); >> - memcpy (data, &local_data.reloc_data, sizeof (struct cr_builtins)); >> + if (bpf_enum_mappings->get (expr) == NULL) >> + core_access_index_map->put (expr, NULL_TREE); >> +} >> >> - rtx reg = XEXP (operands[i], 0); >> - if (!REG_P (reg)) >> - { >> - reg = gen_reg_rtx (Pmode); >> - operands[i] = gen_rtx_MEM (GET_MODE (operands[i]), reg); >> - } >> +/* This function is an adaptation of make_core_safe_access_index but to be >> used >> + in gimple format trees. It is used by execute_lower_bpf_core, when >> + traversing the gimple tree looking for nodes that would have its type >> + attributed with __preserve_access_index. In this particular cases any of >> + the expressions using such attributed types must be made CO-RE safe. */ >> >> - emit_insn ( >> - gen_mov_reloc_coredi (reg, >> - gen_rtx_CONST_INT (Pmode, 0), >> - gen_rtx_CONST_INT (Pmode, index))); >> - return; >> - } >> - } >> +static tree >> +make_gimple_core_safe_access_index (tree *tp, >> + int *walk_subtrees ATTRIBUTE_UNUSED, >> + void *data) >> +{ >> + struct walk_stmt_info *wi = (struct walk_stmt_info *) data; >> + bool valid = true; >> + int n = 0; >> + >> + tree *patch = tp; >> + if (TREE_CODE (*patch) == ADDR_EXPR) >> + patch = &(TREE_OPERAND (*tp, 0)); >> + tree orig_type = TREE_TYPE (*patch); >> + >> + if ((is_attr_preserve_access (*patch) >> + || core_is_access_index (maybe_get_base_for_field_expr (*patch))) >> + && (n = compute_field_expr (*patch, NULL, &valid, NULL)) > 0 >> + && valid == true) >> + { >> + bool changed = false; >> + tree expr_test = make_core_safe_access_index (*patch, &changed); >> + >> + >> + gimple_seq before = NULL; >> + push_gimplify_context (); >> + gimplify_expr (&expr_test, &before, NULL, is_gimple_val, fb_rvalue); >> + >> + /* In case the ADDR_EXPR bypassed above is no longer needed. */ >> + if (patch != tp && TREE_TYPE (expr_test) == TREE_TYPE (*tp)) >> + *tp = expr_test; >> + /* For non pointer value accesses. */ >> + else if (TREE_TYPE (expr_test) == build_pointer_type (orig_type)) >> + *patch = fold_build2 (MEM_REF, TREE_TYPE (*patch), >> + expr_test, build_int_cst (ptr_type_node, 0)); >> + else >> + *patch = expr_test; >> + >> + *tp = fold (*tp); >> + >> + gsi_insert_seq_before (&(wi->gsi), before, GSI_LAST_NEW_STMT); >> + pop_gimplify_context (NULL); >> + >> + wi->changed = true; >> + *walk_subtrees = false; >> + >> + if (!wi->is_lhs) >> + core_mark_as_access_index (gimple_get_lhs (wi->stmt)); >> + } >> + return NULL_TREE; >> +} >> + >> +/* This is the entry point for the pass_data_lower_bpg_core. It walks all >> the >> + statements in gimple, looking for expressions that are suppose to be >> CO-RE >> + preserve_access_index attributed. >> + Those expressions are processed and split by >> + make_gimple_core_safe_access_index function, which will both create the >> + calls to __build_core_reloc and split the expression in smaller parts in >> + case it cannot be represented CO-RE safeguarded by a single CO-RE >> + relocation. */ >> + >> +static unsigned int >> +execute_lower_bpf_core (void) >> +{ >> + if (!TARGET_BPF_CORE) >> + return 0; >> + >> + gimple_seq body = gimple_body (current_function_decl); >> + >> + struct walk_stmt_info wi; >> + core_access_clean (); >> + >> + memset (&wi, 0, sizeof (wi)); >> + wi.info = NULL; >> + >> + /* Split preserve_access_index expressions when needed. */ >> + walk_gimple_seq_mod (&body, NULL, make_gimple_core_safe_access_index, >> &wi); >> + return 0; >> +} >> + >> +namespace { >> + >> +const pass_data pass_data_lower_bpf_core = >> +{ >> + GIMPLE_PASS, /* type */ >> + "bpf_core_lower", /* name */ >> + OPTGROUP_NONE, /* optinfo_flags */ >> + TV_NONE, /* tv_id */ >> + PROP_gimple_any, /* properties_required */ >> + 0, /* properties_provided */ >> + 0, /* properties_destroyed */ >> + 0, /* todo_flags_start */ >> + 0, /* todo_flags_finish */ >> +}; >> + >> +class pass_lower_bpf_core: public gimple_opt_pass >> +{ >> +public: >> + pass_lower_bpf_core (gcc::context *ctxt) >> + : gimple_opt_pass (pass_data_lower_bpf_core, ctxt) >> + {} >> + >> + /* opt_pass methods: */ >> + unsigned int execute (function *) final override >> + { >> + return execute_lower_bpf_core (); >> + } >> + >> +}; /* class pass_lower_bpf_core */ >> + >> +} /* anon namespace */ >> + >> +gimple_opt_pass * >> +make_pass_lower_bpf_core (gcc::context *ctxt) >> +{ >> + return new pass_lower_bpf_core (ctxt); >> } >> >> #include "gt-core-builtins.h" >> diff --git a/gcc/config/bpf/core-builtins.h b/gcc/config/bpf/core-builtins.h >> index 15cd3d3a94e5..c54f6ddac812 100644 >> --- a/gcc/config/bpf/core-builtins.h >> +++ b/gcc/config/bpf/core-builtins.h >> @@ -25,6 +25,34 @@ enum bpf_builtins >> BPF_BUILTIN_MAX, >> }; >> >> +enum bpf_field_info_kind { >> + BPF_FIELD_BYTE_OFFSET = 0, /* field byte offset */ >> + BPF_FIELD_BYTE_SIZE = 1, >> + BPF_FIELD_EXISTS = 2, /* field existence in target kernel */ >> + BPF_FIELD_SIGNED = 3, >> + BPF_FIELD_LSHIFT_U64 = 4, >> + BPF_FIELD_RSHIFT_U64 = 5, >> +}; >> + >> +/* second argument to __builtin_btf_type_id () built-in */ >> +enum bpf_type_id_kind { >> + BPF_TYPE_ID_LOCAL = 0, /* BTF type ID in local program */ >> + BPF_TYPE_ID_TARGET = 1, /* BTF type ID in target kernel */ >> +}; >> + >> +/* second argument to __builtin_preserve_type_info () built-in */ >> +enum bpf_type_info_kind { >> + BPF_TYPE_EXISTS = 0, /* type existence in target kernel */ >> + BPF_TYPE_SIZE = 1, /* type size in target kernel */ >> + BPF_TYPE_MATCHES = 2, /* type match in target kernel */ >> +}; >> + >> +/* second argument to __builtin_preserve_enum_value () built-in */ >> +enum bpf_enum_value_kind { >> + BPF_ENUMVAL_EXISTS = 0, /* enum value existence in kernel */ >> + BPF_ENUMVAL_VALUE = 1, /* enum value value relocation */ >> +}; >> + >> extern GTY (()) tree bpf_builtins[(int) BPF_BUILTIN_MAX]; >> >> void bpf_init_core_builtins (void); >> diff --git a/gcc/config/bpf/t-bpf b/gcc/config/bpf/t-bpf >> index c289dde8b173..18f1fa67794d 100644 >> --- a/gcc/config/bpf/t-bpf >> +++ b/gcc/config/bpf/t-bpf >> @@ -8,3 +8,5 @@ coreout.o: $(srcdir)/config/bpf/coreout.cc >> core-builtins.o: $(srcdir)/config/bpf/core-builtins.cc >> $(COMPILE) $< >> $(POSTCOMPILE) >> + >> +PASSES_EXTRA += $(srcdir)/config/bpf/bpf-passes.def >> diff --git a/gcc/testsuite/gcc.target/bpf/core-attr-5.c >> b/gcc/testsuite/gcc.target/bpf/core-attr-5.c >> new file mode 100644 >> index 000000000000..c0dc15fbb271 >> --- /dev/null >> +++ b/gcc/testsuite/gcc.target/bpf/core-attr-5.c >> @@ -0,0 +1,62 @@ >> +/* Test for BPF CO-RE __attribute__((preserve_access_index)) with accesses >> on >> + LHS and both LHS and RHS of assignment. */ >> + >> +/* { dg-do compile } */ >> +/* { dg-options "-O2 -dA -gbtf -mco-re" } */ >> + >> +struct U { >> + int c; >> + struct V { >> + int d; >> + int e[4]; >> + int f; >> + int *g; >> + } v; >> +} __attribute__((preserve_access_index)); >> + >> +struct T { >> + int a; >> + int b; >> + struct U u; >> + struct U *ptr_u; >> + struct U *array_u; >> +} __attribute__((preserve_access_index)); >> + >> + >> +extern void mset(void *); >> +void >> +func (struct T *t, int i) >> +{ >> + /* This next expression is silently generating incomplete CO-RE >> relocations >> + * because of a front-end optimization to the array access with i. It is >> + * converting the access to pointer arithmetics, which completely removes >> the >> + * reference to the type of array_u. With current folding it is not >> possible >> + * to generate correct/complete CO-RE relocations for such cases. The >> XFAIL >> + * is exactly for this reason. Once we do XPASS this test there is a >> slight >> + * chance we are doing the proper code generation. */ >> + >> + /* 0:4 sizeof(struct U) 0:1:1:3 */ >> + t->array_u[i].v.e[3] = 0xc1; >> + >> + /* This was commented since in this case it is detectable as not a field >> + * expression. */ >> + /* 0:4 sizeof(struct U) 0:1:1:2 */ >> + //__builtin_preserve_access_index (t->array_u[i].v.e[2]) = 0xa1; >> + >> + /* 0:3 0:1:1:1 */ >> + t->ptr_u->v.e[1] = 0xb1; >> + >> + /* 0:3 0:1:2 */ >> + mset (&(t->ptr_u->v.f)); >> + >> + mset (&t->a); >> +} >> + >> +/* { dg-final { scan-assembler-times "ascii \"0:4.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:1:1:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:1:1:1.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 2 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:0.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "bpfcr_type \\(struct T \\*\\)" 4 } } >> */ >> +/* { dg-final { scan-assembler-times "bpfcr_type \\(struct U \\*\\)" 4 { >> xfail *-*-* } } } */ >> + >> diff --git a/gcc/testsuite/gcc.target/bpf/core-attr-6.c >> b/gcc/testsuite/gcc.target/bpf/core-attr-6.c >> new file mode 100644 >> index 000000000000..858ae627cb8b >> --- /dev/null >> +++ b/gcc/testsuite/gcc.target/bpf/core-attr-6.c >> @@ -0,0 +1,46 @@ >> +/* Test for BPF CO-RE __attribute__((preserve_access_index)) with accesses >> on >> + LHS and both LHS and RHS of assignment. */ >> + >> +/* { dg-do compile } */ >> +/* { dg-options "-O2 -dA -gbtf -mco-re" } */ >> + >> +struct U { >> + int c; >> + struct V { >> + int d; >> + int e[4]; >> + int f; >> + int *g; >> + } v; >> +} u; >> + >> +struct T { >> + int a; >> + int b; >> + struct U u; >> + struct U *ptr_u; >> + struct U *array_u; >> +} __attribute__((preserve_access_index)); >> + >> +extern void mset(void *); >> + >> +void >> +func (struct T *t, int i) >> +{ >> + /* 0:3 0:1:1:1 */ >> + t->ptr_u->v.e[1] = 0xb1; >> + >> + /* 0:3 0:1:2 */ >> + mset (&(t->ptr_u->v.f)); >> + >> + /* 0:0 */ >> + mset (&t->a); >> +} >> + >> +/* { dg-final { scan-assembler-times "ascii \"0:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 2 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:1:1:1.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:1:2.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:0.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "bpfcr_type \\(struct T \\*\\)" 3 } } >> */ >> +/* { dg-final { scan-assembler-times "bpfcr_type \\(struct U \\*\\)" 2 } } >> */ >> + >> diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-1.c >> b/gcc/testsuite/gcc.target/bpf/core-builtin-1.c >> index e994dc6add48..3f15980a4c78 100644 >> --- a/gcc/testsuite/gcc.target/bpf/core-builtin-1.c >> +++ b/gcc/testsuite/gcc.target/bpf/core-builtin-1.c >> @@ -21,7 +21,7 @@ unsigned long ula[8]; >> >> #define _(x) (__builtin_preserve_access_index (x)) >> >> -void >> +unsigned long >> func (void) >> { >> /* 1 */ >> @@ -35,6 +35,8 @@ func (void) >> >> /* 6 */ >> unsigned long ul = _(ula[6]); >> + >> + return b + c + uc + ul; >> } >> >> char >> diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue-opt.c >> b/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue-opt.c >> index 363ad0c41ecc..c87e1a3ba3b0 100644 >> --- a/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue-opt.c >> +++ b/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue-opt.c >> @@ -26,8 +26,8 @@ unsigned long foo(void *data) >> return 0; >> } >> >> -/* { dg-final { scan-assembler-times "\t.4byte\t0x8\t; bpfcr_type >> \\(named_ue64\\)" 2 } } */ >> -/* { dg-final { scan-assembler-times "\t.4byte\t0x9\t; bpfcr_type >> \\(named_se64\\)" 2} } */ >> +/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type >> \\(named_ue64\\)" 2 } } */ >> +/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type >> \\(named_se64\\)" 2} } */ >> /* { dg-final { scan-assembler-times "\t.4byte\t0xa\t; bpfcr_kind" 2 } } >> BPF_ENUMVAL_EXISTS */ >> /* { dg-final { scan-assembler-times "\t.4byte\t0xb\t; bpfcr_kind" 2 } } >> BPF_ENUMVAL_VALUE */ >> >> diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue.c >> b/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue.c >> index 3e3334dc089a..2f16903b8d64 100644 >> --- a/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue.c >> +++ b/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue.c >> @@ -40,10 +40,10 @@ int foo(void *data) >> return 0; >> } >> >> -/* { dg-final { scan-assembler-times "\t.4byte\t0x8\t; bpfcr_type >> \\(named_ue64\\)" 5 } } */ >> -/* { dg-final { scan-assembler-times "\t.4byte\t0x9\t; bpfcr_type >> \\(named_se64\\)" 5} } */ >> -/* { dg-final { scan-assembler-times "\t.4byte\t0xb\t; bpfcr_type >> \\(named_ue\\)" 5 } } */ >> -/* { dg-final { scan-assembler-times "\t.4byte\t0xc\t; bpfcr_type >> \\(named_se\\)" 5} } */ >> +/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type >> \\(named_ue64\\)" 5 } } */ >> +/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type >> \\(named_se64\\)" 5} } */ >> +/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type >> \\(named_ue\\)" 5 } } */ >> +/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type >> \\(named_se\\)" 5} } */ >> /* { dg-final { scan-assembler-times "\t.4byte\t0xa\t; bpfcr_kind" 12 } } >> BPF_ENUMVAL_EXISTS */ >> /* { dg-final { scan-assembler-times "\t.4byte\t0xb\t; bpfcr_kind" 8 } } >> BPF_ENUMVAL_VALUE */ >> >> diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-1.c >> b/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-1.c >> new file mode 100644 >> index 000000000000..b8bdeeaa125c >> --- /dev/null >> +++ b/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-1.c >> @@ -0,0 +1,76 @@ >> +/* { dg-do compile } */ >> +/* { dg-options "-O0 -dA -gbtf -mco-re" } */ >> + >> +struct S { >> + int a; >> + int b; >> + char c; >> +}; >> + >> +union U { >> + unsigned int u; >> + int i; >> + unsigned char uc[4]; >> + signed char ic[4]; >> +}; >> + >> +struct S my_s; >> +union U my_u; >> + >> +unsigned long ula[8]; >> + >> +#define _(x) (__builtin_preserve_access_index (x)) >> + >> +unsigned long >> +func (void) >> +{ >> + int b; >> + char c; >> + unsigned char uc; >> + unsigned long ul; >> + int ic; >> + >> + __builtin_preserve_access_index (({ >> + /* 1 */ >> + b = my_s.b; >> + >> + /* 2 */ >> + ic = my_s.c; >> + >> + /* 2:3 */ >> + uc = my_u.uc[3]; >> + >> + /* 6 */ >> + ul = ula[6]; >> + })); >> + >> + return b + ic + uc + ul; >> +} >> + >> +char >> +s_ptr (struct S *ps) >> +{ >> + /* 0:2 */ >> + char x; >> + __builtin_preserve_access_index (({ x = ps->c; })); >> + return x; >> +} >> + >> +unsigned char >> +u_ptr (union U *pu) >> +{ >> + /* 0:2:3 */ >> + unsigned char x; >> + __builtin_preserve_access_index (({ x = pu->uc[3]; })); >> + return x; >> +} >> + >> +/* { dg-final { scan-assembler-times "ascii \"1.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"2.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"2:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"6.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:2.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:2:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> + >> +/* { dg-final { scan-assembler-times "bpfcr_type" 6 } } */ >> +/* { dg-final { scan-assembler-times "\[\t \]0x6c\[\t >> \]+\[^\n\]*core_relo_len" 1 } } */ >> diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-2.c >> b/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-2.c >> new file mode 100644 >> index 000000000000..3a22b99f8e6f >> --- /dev/null >> +++ b/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-2.c >> @@ -0,0 +1,35 @@ >> +/* { dg-do compile } */ >> +/* { dg-options "-O0 -dA -gbtf -mco-re" } */ >> + >> +struct T { >> + int a; >> + int b; >> + struct U { >> + int c; >> + struct V { >> + int d; >> + int e[4]; >> + int f; >> + } v; >> + } u; >> +}; >> + >> +void func (struct T * foo) >> +{ >> + /* Access string: "0:2:1:1:3" */ >> + int *x; >> + int *y; >> + __builtin_preserve_access_index (({ >> + x = &(foo->u.v.e[3]); >> + struct U *tmp = &(foo->u); >> + y = &(tmp->v.f); >> + })); >> + >> + *x = 17; >> + *y = 16; >> +} >> + >> +/* { dg-final { scan-assembler-times "ascii \"0:2:1:1:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:2.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:1:2.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "bpfcr_type" 3 } } */ >> diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-3.c >> b/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-3.c >> new file mode 100644 >> index 000000000000..bccf2b3a6968 >> --- /dev/null >> +++ b/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-3.c >> @@ -0,0 +1,37 @@ >> +/* { dg-do compile } */ >> +/* { dg-options "-O0 -dA -gbtf -mco-re" } */ >> + >> +struct T { >> + int a; >> + int b; >> + struct U { >> + int c; >> + struct V { >> + int d; >> + int e[4]; >> + int f; >> + } v; >> + } u; >> +}; >> + >> +void func (struct T * foo) >> +{ >> + /* Access string: "0:2:1:1:3" */ >> + int *x; >> + int *y; >> + struct U *tmp; >> + __builtin_preserve_access_index (({ >> + x = &(foo->u.v.e[3]); >> + tmp = &(foo->u); >> + })); >> + y = __builtin_preserve_access_index(tmp ? 0 : &(tmp->v.f)); >> + >> + *x = 17; >> + if (y != 0) >> + *y = 16; >> +} >> + >> +/* { dg-final { scan-assembler-times "ascii \"0:2:1:1:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:2.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:1:2.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "bpfcr_type" 3 } } */ >> diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-4.c >> b/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-4.c >> new file mode 100644 >> index 000000000000..8ef239c30c15 >> --- /dev/null >> +++ b/gcc/testsuite/gcc.target/bpf/core-builtin-exprlist-4.c >> @@ -0,0 +1,31 @@ >> +/* { dg-do compile } */ >> +/* { dg-options "-O0 -dA -gbtf -mco-re" } */ >> + >> +struct T { >> + int a; >> + int b; >> + struct U { >> + int c; >> + struct V { >> + int d; >> + int e[4]; >> + int f; >> + } *v; >> + } u; >> +}; >> + >> +void func (struct T * foo) >> +{ >> + /* Access string: "0:2:1:1:3" */ >> + int *x; >> + int *y; >> + __builtin_preserve_access_index (({ >> + x = &((foo->u.v)->e[3]); >> + })); >> + >> + *x = 17; >> +} >> + >> +/* { dg-final { scan-assembler-times "ascii \"0:2:1.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:1:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "bpfcr_type" 2 } } */ >> diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-fieldinfo-offset-1.c >> b/gcc/testsuite/gcc.target/bpf/core-builtin-fieldinfo-offset-1.c >> index 590eea007ae1..a4af9a53282a 100644 >> --- a/gcc/testsuite/gcc.target/bpf/core-builtin-fieldinfo-offset-1.c >> +++ b/gcc/testsuite/gcc.target/bpf/core-builtin-fieldinfo-offset-1.c >> @@ -20,6 +20,7 @@ enum { >> FIELD_BYTE_OFFSET = 0, >> }; >> >> +extern struct T *bar(); >> >> unsigned int foo (struct T *t) >> { >> @@ -34,23 +35,26 @@ unsigned int foo (struct T *t) >> unsigned c = __builtin_preserve_field_info (t->c, FIELD_BYTE_OFFSET); >> unsigned d = __builtin_preserve_field_info (t->d, FIELD_BYTE_OFFSET); >> >> - return s0a1 + s0a4 + s0x + s1a1 + s1a4 + s1x + c + d; >> + unsigned e1 = __builtin_preserve_field_info (bar()->d, FIELD_BYTE_OFFSET); >> + unsigned e2 = __builtin_preserve_field_info (bar()->s[1].a4, >> FIELD_BYTE_OFFSET); >> + >> + return s0a1 + s0a4 + s0x + s1a1 + s1a4 + s1x + c + d + e1 + e2; >> } >> >> /* { dg-final { scan-assembler-times "\[\t \]mov\[\t \]%r\[0-9\],4" 2 } } */ >> /* { dg-final { scan-assembler-times "\[\t \]mov\[\t \]%r\[0-9\],8" 1 } } */ >> -/* { dg-final { scan-assembler-times "\[\t \]mov\[\t \]%r\[0-9\],12" 2 } } >> */ >> +/* { dg-final { scan-assembler-times "\[\t \]mov\[\t \]%r\[0-9\],12" 3 } } >> */ >> /* { dg-final { scan-assembler-times "\[\t \]mov\[\t \]%r\[0-9\],16" 1 } } >> */ >> /* { dg-final { scan-assembler-times "\[\t \]mov\[\t \]%r\[0-9\],20" 1 } } >> */ >> -/* { dg-final { scan-assembler-times "\[\t \]mov\[\t \]%r\[0-9\],21" 1 } } >> */ >> +/* { dg-final { scan-assembler-times "\[\t \]mov\[\t \]%r\[0-9\],21" 2 } } >> */ >> >> /* { dg-final { scan-assembler-times "ascii \"0:1:0:0.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> /* { dg-final { scan-assembler-times "ascii \"0:1:0:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> /* { dg-final { scan-assembler-times "ascii \"0:1:0:4.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> /* { dg-final { scan-assembler-times "ascii \"0:1:1:0.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> -/* { dg-final { scan-assembler-times "ascii \"0:1:1:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:1:1:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 2 } } */ >> /* { dg-final { scan-assembler-times "ascii \"0:1:1:4.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> /* { dg-final { scan-assembler-times "ascii \"0:2.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> -/* { dg-final { scan-assembler-times "ascii \"0:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 1 } } */ >> +/* { dg-final { scan-assembler-times "ascii \"0:3.0\"\[\t >> \]+\[^\n\]*btf_aux_string" 2 } } */ >> >> -/* { dg-final { scan-assembler-times "0\[\t \]+\[^\n\]*bpfcr_kind" 8 } } */ >> +/* { dg-final { scan-assembler-times "0\[\t \]+\[^\n\]*bpfcr_kind" 10 } } */ >>
