Hi,
this patch enables IPA-SRA for internal .ASSUME calls. Consider the
following testcase:
struct S
{
bool x;
};
void
foo(struct S s)
{
[[assume (!s.x)]];
if (s.x)
__builtin_abort ();
}
The [[assume (...)]] statement gets lowered to (simplified):
bool foo.assume_fn(struct S s)
{
return !s.x;
}
void
foo(struct S s)
{
.ASSUME (foo.assume_fn, s);
if (s.x)
__builtin_abort ();
}
The internal call to .ASSUME represents an indirect call to
foo.assume_fn. Later on, the assume function is walked back from the
return statement and vrp tries to figure out value ranges to make the
function return true. However, as it encouters the struct, it assigns
it a varying range and gives up, making the assumption pointless.
This patch allows GCC to split the struct, remove pointers and so on
through SRA, making the value range propagation more useful. It does so
through adding callback edges to appropriately describe the indirect
call to the assume body function and allowing GCC to modify the call
statement.
I am not particuarly happy with some parts of this patch:
- The sra part uses newly added fields to summaries to force sra to
take action. Otherwise, none of the arguments would ever get split,
as they don't fit the heuristics. It seems odd to force
transformations, but I think the transformation is always worth it,
considering it provides us extra information and code never makes it
to the binary.
- As the call statement of the callback edge is analyzed by sra, the
arguments need to be remapped according to the callback edge
information, otherwise all argument indices would be off by one. It
does so through wrapper functions gimple_call_arg_summary and
gimple_call_num_args_summary. They take a pointer to a callback_info
summary as a parameter and if it's not null, they remap the argument.
I am not very happy with them, as they obfucaste what is happening
and act like their non-summary counterparts in most cases, but this
is the cleanest solution I could think of. They also introduce
branching for all calls, even though the callback branch rarely ever
applies.
With this patch, GCC is able to optimize the testcase above into a
simple
void
foo(struct S s)
{
return;
}
Collapsing pointers into scalars also works, as you can see in the
testcase provided in the patch.
This patch requires
https://gcc.gnu.org/pipermail/gcc-patches/2025-December/703348.html
as a dependency.
This is likely not the final version of this patch, so I left things in
for transparency, e.g. commented out lines instead of removing them.
Bootstraps and passes the testsuite on x86_64-linux. Any
feedback is welcome.
Best regards,
Josef
PR c++/122779
PR ipa/109112
gcc/ChangeLog:
* attr-callback.cc (callback_redirect_edge): New function,
redirects a callback edge.
* attr-callback.h (callback_redirect_edge): Add decl.
* callback-info.cc (callback_info::initialize_internal_summary):
New method, initializes the summary for internal calls.
(gimple_call_arg_summary): New function, wrapper around
gimple_call_arg. Remaps arg indices for callback functions if
necessary.
(gimple_call_num_args_summary): New function, wrapper around
gimple_call_num_args. Returns the correct number of args for
callback functions.
* callback-info.h (gimple_call_arg_summary): Add decl.
(gimple_call_num_args_summary): Add decl.
* cgraph.cc (cgraph_node::get_edge): Don't return the
dispatching edge if the edge is attached to an internal call.
(cgraph_edge::make_callback_internal): New static method,
creates a callback edge attached to an internal call.
(cgraph_edge::redirect_call_stmt_to_callee): Implement internal
callback edge redirection.
(cgraph_node::assume_fn_p): New predicate, returns TRUE iff node
represents an outlined .ASSUME body.
(cgraph_node::verify_node): Adjust sanity checks.
* cgraph.h (struct cgraph_node): Add decl for assume_fn_p.
(struct cgraph_node): Add decl for make_callback_internal.
* gimple-low.cc (create_assumption_fn): Don't add noipa and
noclone attrs.
(lower_assumption): Unset stdarg flag.
* ipa-fnsummary.cc (analyze_function_body): Implement summary
modifications for internal callback calls.
* ipa-param-manipulation.cc (ipa_param_adjustments::modify_call):
Implement call stmt modifications for IFN_ASSUME.
* ipa-sra.cc (struct isra_param_desc): Add dont_disqualify flag.
(struct gensum_param_desc): Likewise.
(ipa_sra_function_summaries::duplicate): Copy dont_disqualify.
(ptr_parm_has_nonarg_uses): Use gimple_call_arg_summary.
(create_parameter_descriptors): Initialize dont_disqualify.
(disqualify_split_candidate): Early exit iff dont_disqualify.
(scan_function): Add logic for internal callback edge creation.
(isra_analyze_call): Remap args for callback edges.
(process_scan_results): Likewise.
(ipa_sra_generate_summary): Check-create callback_info_sum.
(isra_write_node_summary): Stream out dont_disqualify.
(isra_read_node_info): Read dont_disqualify.
(can_be_local_p): New function, returns TRUE iff
node->can_be_local_p or if node is an .ASSUME body.
(ipa_sra_ipa_function_checks): Use can_be_local_p.
(size_would_violate_limit_p): Use dont_disqualify.
(retval_used_p): Return TRUE for .ASSUME bodies.
* tree-inline.cc (initialize_cfun): Copy assume_function field.
gcc/testsuite/ChangeLog:
* g++.dg/ipa/ipa-sra-assume.C: New test.
Signed-off-by: Josef Melcr <[email protected]>
---
gcc/attr-callback.cc | 13 ++++
gcc/attr-callback.h | 4 ++
gcc/callback-info.cc | 44 ++++++++++++
gcc/callback-info.h | 15 +++-
gcc/cgraph.cc | 84 +++++++++++++++++-----
gcc/cgraph.h | 13 ++++
gcc/gimple-low.cc | 10 ++-
gcc/ipa-fnsummary.cc | 22 +++++-
gcc/ipa-param-manipulation.cc | 23 ++++--
gcc/ipa-sra.cc | 87 +++++++++++++++++++----
gcc/testsuite/g++.dg/ipa/ipa-sra-assume.C | 37 ++++++++++
gcc/tree-inline.cc | 1 +
12 files changed, 312 insertions(+), 41 deletions(-)
create mode 100644 gcc/testsuite/g++.dg/ipa/ipa-sra-assume.C
diff --git a/gcc/attr-callback.cc b/gcc/attr-callback.cc
index 76374f1e6b1..18b112d6478 100644
--- a/gcc/attr-callback.cc
+++ b/gcc/attr-callback.cc
@@ -24,6 +24,7 @@
#include "backend.h"
#include "tree.h"
#include "gimple.h"
+#include "fold-const.h"
#include "alloc-pool.h"
#include "cgraph.h"
#include "diagnostic.h"
@@ -362,3 +363,15 @@ callback_num_args (tree attr)
;
return res;
}
+
+/* Redirect call statement of E to the callee of E. Used in
+ cgraph_edge::redirect_call_stmt_to_callee. */
+void
+callback_redirect_edge (cgraph_edge *e)
+{
+ callback_info *ci = callback_info_sum->get (e);
+ tree previous_arg = gimple_call_arg (e->call_stmt, ci->fn_idx);
+ location_t loc = EXPR_LOCATION (previous_arg);
+ tree new_addr = build_fold_addr_expr_loc (loc, e->callee->decl);
+ gimple_call_set_arg (e->call_stmt, ci->fn_idx, new_addr);
+}
diff --git a/gcc/attr-callback.h b/gcc/attr-callback.h
index 34e4e08d665..4068803041f 100644
--- a/gcc/attr-callback.h
+++ b/gcc/attr-callback.h
@@ -75,4 +75,8 @@ bool callback_edge_useful_p (cgraph_edge *e);
takes. */
size_t callback_num_args (tree attr);
+/* Redirect call statement of E to the callee of E. Used in
+ cgraph_edge::redirect_call_stmt_to_callee. */
+void callback_redirect_edge (cgraph_edge *e);
+
#endif /* ATTR_CALLBACK_H */
diff --git a/gcc/callback-info.cc b/gcc/callback-info.cc
index cec44937c25..8d0d83ce343 100644
--- a/gcc/callback-info.cc
+++ b/gcc/callback-info.cc
@@ -70,6 +70,30 @@ callback_info::stream_in (lto_input_block *ib)
redirected = bp_unpack_value (&bp, 1);
}
+
+/* Initialize the summary for a callback call CBE of internal function IFN. */
+callback_info *
+callback_info::initialize_internal_summary (cgraph_edge *cbe, internal_fn ifn)
+{
+ callback_info_sum_t::check_create_info_sum ();
+ callback_info *ci = callback_info_sum->get_create (cbe);
+ switch (ifn)
+ {
+ case IFN_ASSUME:
+ {
+ ci->id = ci->fn_idx = 0;
+ unsigned n = gimple_call_num_args (cbe->call_stmt);
+ for (unsigned i = 0; i < n - 1; i++)
+ ci->arg_mapping.safe_push (i + 1);
+ break;
+ }
+ default:
+ gcc_unreachable ();
+ }
+ ci->redirected = false;
+ return ci;
+}
+
void
callback_info_sum_t::check_create_info_sum (void)
{
@@ -96,3 +120,23 @@ callback_info_sum_t::duplicate (cgraph_edge *, cgraph_edge
*,
dst_s->arg_mapping = src_s->arg_mapping.copy ();
dst_s->redirected = src_s->redirected;
}
+
+/* Return the argument at IDX if CI is null, otherwise use the mapping in CI
and
+ return the remapped argument. */
+tree
+gimple_call_arg_summary (gimple *stmt, unsigned idx, callback_info *ci)
+{
+ if (ci)
+ idx = ci->arg_mapping[idx];
+ return gimple_call_arg (stmt, idx);
+}
+
+/* Return the number of arguments of STMT if CI is null,
+ ci->arg_mapping.length() otherwise. */
+unsigned
+gimple_call_num_args_summary (const gcall *stmt, callback_info *ci)
+{
+ if (ci)
+ return ci->arg_mapping.length ();
+ return gimple_call_num_args (stmt);
+}
diff --git a/gcc/callback-info.h b/gcc/callback-info.h
index 1745e7f83d6..ddc45b8823a 100644
--- a/gcc/callback-info.h
+++ b/gcc/callback-info.h
@@ -36,15 +36,18 @@ public:
*/
auto_vec<int> arg_mapping;
/* TRUE iff the associated callback edge was redirected. */
- bool redirected;
+ bool redirected : 1;
/* Stream in callback_info. */
void stream_in (lto_input_block *ib);
/* Stream out callback_info. */
void stream_out (lto_simple_output_block *ib) const;
-};
+ /* Initialize the summary for a callback call CBE of internal function IFN.
*/
+ static callback_info *initialize_internal_summary (cgraph_edge *cbe,
+ internal_fn ifn);
+};
class callback_info_sum_t : public call_summary<callback_info *>
{
@@ -66,4 +69,12 @@ public:
extern callback_info_sum_t *callback_info_sum;
+/* Return the argument at IDX if CI is null, otherwise use the mapping in CI
and
+ return the remapped argument. */
+tree gimple_call_arg_summary (gimple *stmt, unsigned idx, callback_info *ci);
+
+/* Return the number of arguments of STMT if CI is null,
+ ci->arg_mapping.length() otherwise. */
+unsigned gimple_call_num_args_summary (const gcall *stmt, callback_info *ci);
+
#endif /* CALLBACK_INFO_H */
diff --git a/gcc/cgraph.cc b/gcc/cgraph.cc
index 315b4de15b7..7f36473bc94 100644
--- a/gcc/cgraph.cc
+++ b/gcc/cgraph.cc
@@ -937,7 +937,7 @@ cgraph_node::get_edge (gimple *call_stmt)
comes to callback edges, a call statement might have multiple callback
edges attached to it. These can be easily obtained from the carrying edge
instead. */
- if (e && e->callback)
+ if (e && e->callback && !e->callee->assume_fn_p ())
e = e->get_callback_carrying_edge ();
if (n > 100)
@@ -1368,6 +1368,37 @@ cgraph_edge::make_callback (cgraph_node *n2)
return e2;
}
+
+/* Create a callback edge, representing an indirect call to n2
+ passed to an internal function by argument. As internal calls are not
+ represented in the callgraph, the created edge won't have any associated
+ dispatching edge. edge. The created callback edge will be attached to the
+ internal call statement STMT. Returns created callback edge. */
+
+cgraph_edge *
+cgraph_edge::make_callback_internal (cgraph_node *caller, cgraph_node *target,
+ gcall *call_stmt)
+{
+ cgraph_edge *ret;
+
+ ret
+ = caller->create_edge (target, call_stmt, profile_count::uninitialized ());
+ if (dump_file)
+ {
+ fprintf (dump_file,
+ "Created callback edge %s -> %s belonging to internal call ",
+ ret->caller->dump_name (), ret->callee->dump_name ());
+ print_gimple_stmt (dump_file, call_stmt, 0);
+ }
+ ret->inline_failed = CIF_CALLBACK_EDGE;
+ ret->callback = true;
+ if (TREE_NOTHROW (target->decl))
+ ret->can_throw_external = false;
+ caller->create_reference (target, IPA_REF_ADDR, call_stmt);
+ target->mark_address_taken ();
+ return ret;
+}
+
/* Returns the callback_carrying edge of a callback edge on which
it is called on or NULL when no such edge can be found.
@@ -1821,21 +1852,18 @@ cgraph_edge::redirect_call_stmt_to_callee (cgraph_edge
*e,
/* When redirecting a callback edge, all we need to do is replace
the original address with the address of the function we are
redirecting to. */
- if (e->callback)
+ if (e->callback && !e->callee->assume_fn_p ())
{
cgraph_edge *carrying = e->get_callback_carrying_edge ();
- if (!callback_is_special_cased (carrying->callee->decl, e->call_stmt)
+ if (carrying
+ && !callback_is_special_cased (carrying->callee->decl, e->call_stmt)
&& !lookup_attribute (CALLBACK_ATTR_IDENT,
DECL_ATTRIBUTES (carrying->callee->decl)))
/* Callback attribute is removed if the dispatching function changes
signature, as the indices wouldn't be correct anymore. These edges
will get cleaned up later, ignore their redirection for now. */
return e->call_stmt;
- callback_info *ci = callback_info_sum->get (e);
- tree previous_arg = gimple_call_arg (e->call_stmt, ci->fn_idx);
- location_t loc = EXPR_LOCATION (previous_arg);
- tree new_addr = build_fold_addr_expr_loc (loc, e->callee->decl);
- gimple_call_set_arg (e->call_stmt, ci->fn_idx, new_addr);
+ callback_redirect_edge (e);
return e->call_stmt;
}
@@ -1888,6 +1916,9 @@ cgraph_edge::redirect_call_stmt_to_callee (cgraph_edge *e,
&& old_fntype == TREE_TYPE (origin->former_clone_of))
|| old_fntype == TREE_TYPE (origin->decl))
gimple_call_set_fntype (new_stmt, TREE_TYPE (e->callee->decl));
+ /* Don't set fntype on internal calls. */
+ else if (gimple_call_internal_p (new_stmt))
+ ;
else
{
tree new_fntype = padjs->build_new_function_type (old_fntype, true);
@@ -1904,10 +1935,20 @@ cgraph_edge::redirect_call_stmt_to_callee (cgraph_edge
*e,
BUILT_IN_UNREACHABLE_TRAP))
ipa_verify_edge_has_no_modifications (e);
new_stmt = e->call_stmt;
- gimple_call_set_fndecl (new_stmt, e->callee->decl);
+ /* Redirects callback edges attached to internal calls. */
+ if (e->callback)
+ callback_redirect_edge (e);
+ else
+ gimple_call_set_fndecl (new_stmt, e->callee->decl);
update_stmt_fn (DECL_STRUCT_FUNCTION (e->caller->decl), new_stmt);
}
+ if (e->callback && e->callee->assume_fn_p ())
+ {
+ e->caller->set_call_stmt_including_clones (e->call_stmt, new_stmt,
false);
+ return new_stmt;
+ }
+
/* If changing the call to __cxa_pure_virtual or similar noreturn function,
adjust gimple_call_fntype too. */
if (gimple_call_noreturn_p (new_stmt)
@@ -2821,6 +2862,16 @@ cgraph_node::can_be_local_p (void)
NULL, true));
}
+
+/* Returns true if the node represents an outlined body created by lowering
+ [[assume(...)]]. */
+bool
+cgraph_node::assume_fn_p (void)
+{
+ function *f = DECL_STRUCT_FUNCTION (decl);
+ return f && f->assume_function;
+}
+
/* Call callback on cgraph_node, thunks and aliases associated to cgraph_node.
When INCLUDE_OVERWRITABLE is false, overwritable symbols are
skipped. When EXCLUDE_VIRTUAL_THUNKS is true, virtual thunks are
@@ -4334,13 +4385,14 @@ cgraph_node::verify_node (void)
if (e->callback)
{
- if (!e->get_callback_carrying_edge ())
- {
- error ("callback edge %s->%s has no callback-carrying",
- identifier_to_locale (e->caller->name ()),
- identifier_to_locale (e->callee->name ()));
- error_found = true;
- }
+ if (!e->get_callback_carrying_edge ()
+ && (!e->call_stmt || !gimple_call_internal_p (e->call_stmt)))
+ {
+ error ("callback edge %s->%s has no callback-carrying edge",
+ identifier_to_locale (e->caller->name ()),
+ identifier_to_locale (e->callee->name ()));
+ error_found = true;
+ }
}
if (e->has_callback
diff --git a/gcc/cgraph.h b/gcc/cgraph.h
index 0276e7652c8..12aeced398c 100644
--- a/gcc/cgraph.h
+++ b/gcc/cgraph.h
@@ -1272,6 +1272,10 @@ struct GTY((tag ("SYMTAB_FUNCTION"))) cgraph_node :
public symtab_node
it is safe to ignore its side effects for IPA analysis. */
bool cannot_return_p (void);
+ /* Returns true if the node represents an outlined body created by lowering
+ [[assume(...)]]. */
+ bool assume_fn_p (void);
+
/* Return true when function cgraph_node and all its aliases are only called
directly.
i.e. it is not externally visible, address was not taken and
@@ -1794,6 +1798,15 @@ public:
callback edge. */
cgraph_edge *make_callback (cgraph_node *n2);
+ /* Create a callback edge, representing an indirect call to n2
+ passed to an internal function by argument. As internal calls are not
+ represented in the callgraph, the created edge won't have any associated
+ dispatching edge. edge. The created callback edge will be attached to the
+ internal call statement STMT. Returns created callback edge. */
+ static cgraph_edge *make_callback_internal (cgraph_node *caller,
+ cgraph_node *target,
+ gcall *call_stmt);
+
/* Returns the callback-carrying edge of a callback edge or NULL, if such
edge
cannot be found. An edge is considered callback-carrying, if it has it's
has_callback flag set and shares it's call statement with the edge
diff --git a/gcc/gimple-low.cc b/gcc/gimple-low.cc
index 79cdd7763da..7f2d8c263cd 100644
--- a/gcc/gimple-low.cc
+++ b/gcc/gimple-low.cc
@@ -305,11 +305,13 @@ create_assumption_fn (location_t loc)
tree attributes = DECL_ATTRIBUTES (current_function_decl);
if (lookup_attribute ("noipa", attributes) == NULL)
{
- attributes = tree_cons (get_identifier ("noipa"), NULL, attributes);
+ /* We want to allow some IPA optimizations, e.g. ipa-sra. */
+
+ // attributes = tree_cons (get_identifier ("noipa"), NULL, attributes);
if (lookup_attribute ("noinline", attributes) == NULL)
attributes = tree_cons (get_identifier ("noinline"), NULL, attributes);
- if (lookup_attribute ("noclone", attributes) == NULL)
- attributes = tree_cons (get_identifier ("noclone"), NULL, attributes);
+ // if (lookup_attribute ("noclone", attributes) == NULL)
+ // attributes = tree_cons (get_identifier ("noclone"), NULL, attributes);
if (lookup_attribute ("no_icf", attributes) == NULL)
attributes = tree_cons (get_identifier ("no_icf"), NULL, attributes);
}
@@ -673,6 +675,8 @@ lower_assumption (gimple_stmt_iterator *gsi, struct
lower_data *data)
}
DECL_ARGUMENTS (lad.id.dst_fn) = parms;
TREE_TYPE (lad.id.dst_fn) = build_function_type (boolean_type_node, parmt);
+ /* The body function no longer has var. args, unset stdarg. */
+ DECL_STRUCT_FUNCTION (lad.id.dst_fn)->stdarg = 0;
cgraph_node::add_new_function (lad.id.dst_fn, false);
diff --git a/gcc/ipa-fnsummary.cc b/gcc/ipa-fnsummary.cc
index bf98a449584..3ee7b8239c3 100644
--- a/gcc/ipa-fnsummary.cc
+++ b/gcc/ipa-fnsummary.cc
@@ -3041,9 +3041,20 @@ analyze_function_body (struct cgraph_node *node, bool
early)
}
if (is_gimple_call (stmt)
- && !gimple_call_internal_p (stmt))
+ && (!gimple_call_internal_p (stmt)
+ || gimple_call_internal_fn (stmt) == IFN_ASSUME))
{
struct cgraph_edge *edge = node->get_edge (stmt);
+ if (!edge)
+ {
+ /* The edge for .ASSUME outlined bodies is created when
+ generating ipa-sra summaries. Ignore the call statement
+ for now. */
+ gcc_checking_assert (gimple_call_internal_p (stmt)
+ && gimple_call_internal_fn (stmt)
+ == IFN_ASSUME);
+ continue;
+ }
ipa_call_summary *es = ipa_call_summaries->get_create (edge);
/* Special case: results of BUILT_IN_CONSTANT_P will be always
@@ -3130,6 +3141,15 @@ analyze_function_body (struct cgraph_node *node, bool
early)
es2->call_stmt_time = 0;
}
}
+ /* Internal calls are not in the callgraph, so edges pointing to
+ .ASSUME bodies are not covered by the above branch, as the
+ dispatching edge doesn't exist. */
+ else if (edge->callback)
+ {
+ gcc_checking_assert (edge->callee->assume_fn_p ());
+ es->call_stmt_size = 0;
+ es->call_stmt_time = 0;
+ }
}
/* TODO: When conditional jump or switch is known to be constant, but
diff --git a/gcc/ipa-param-manipulation.cc b/gcc/ipa-param-manipulation.cc
index 96ab125dee1..9770bc4ecf2 100644
--- a/gcc/ipa-param-manipulation.cc
+++ b/gcc/ipa-param-manipulation.cc
@@ -50,7 +50,9 @@ along with GCC; see the file COPYING3. If not see
#include "sreal.h"
#include "ipa-cp.h"
#include "ipa-prop.h"
+#include "data-streamer.h"
#include "attr-callback.h"
+#include "callback-info.h"
/* Actual prefixes of different newly synthetized parameters. Keep in sync
with IPA_PARAM_PREFIX_* defines. */
@@ -719,7 +721,8 @@ ipa_param_adjustments::modify_call (cgraph_edge *cs,
unsigned len = vec_safe_length (m_adj_params);
auto_vec<tree, 16> vargs (len);
- unsigned old_nargs = gimple_call_num_args (stmt);
+ callback_info *ci = callback_info_sum->get (cs);
+ unsigned old_nargs = gimple_call_num_args_summary (stmt, ci);
unsigned orig_nargs = mod_info ? mod_info->index_map.length () : old_nargs;
auto_vec<bool, 16> kept (old_nargs);
kept.quick_grow_cleared (old_nargs);
@@ -747,7 +750,7 @@ ipa_param_adjustments::modify_call (cgraph_edge *cs,
gcc_assert (index >= 0);
}
- tree arg = gimple_call_arg (stmt, index);
+ tree arg = gimple_call_arg_summary (stmt, index, ci);
vargs.quick_push (arg);
kept[index] = true;
@@ -773,7 +776,7 @@ ipa_param_adjustments::modify_call (cgraph_edge *cs,
{
int repl_idx = mod_info->pass_through_map[j].new_index;
gcc_assert (repl_idx >= 0);
- repl = gimple_call_arg (stmt, repl_idx);
+ repl = gimple_call_arg_summary (stmt, repl_idx, ci);
break;
}
if (repl)
@@ -797,7 +800,7 @@ ipa_param_adjustments::modify_call (cgraph_edge *cs,
index = mod_info->index_map[apm->base_index];
gcc_assert (index >= 0);
}
- tree base = gimple_call_arg (stmt, index);
+ tree base = gimple_call_arg_summary (stmt, index, ci);
/* We create a new parameter out of the value of the old one, we can
do the following kind of transformations:
@@ -920,7 +923,7 @@ ipa_param_adjustments::modify_call (cgraph_edge *cs,
gcc_assert (always_copy_start >= 0);
}
for (unsigned i = always_copy_start; i < old_nargs; i++)
- vargs.safe_push (gimple_call_arg (stmt, i));
+ vargs.safe_push (gimple_call_arg_summary (stmt, i, ci));
}
/* For optimized away parameters, add on the caller side
@@ -1005,7 +1008,15 @@ ipa_param_adjustments::modify_call (cgraph_edge *cs,
print_gimple_stmt (dump_file, gsi_stmt (gsi), 0);
}
- gcall *new_stmt = gimple_build_call_vec (callee_decl, vargs);
+ gcall *new_stmt;
+ if (cs->callback && cs->callee->assume_fn_p ())
+ {
+ /* Has to be special-cased, as we have to create an internal call. */
+ vargs.safe_insert (0, build_fold_addr_expr (callee_decl));
+ new_stmt = gimple_build_call_internal_vec (IFN_ASSUME, vargs);
+ }
+ else
+ new_stmt = gimple_build_call_vec (callee_decl, vargs);
hash_set <tree> *ssas_to_remove = NULL;
if (tree lhs = gimple_call_lhs (stmt))
diff --git a/gcc/ipa-sra.cc b/gcc/ipa-sra.cc
index 6e6cf895988..f4aa5b01462 100644
--- a/gcc/ipa-sra.cc
+++ b/gcc/ipa-sra.cc
@@ -87,6 +87,8 @@ along with GCC; see the file COPYING3. If not see
#include "sreal.h"
#include "ipa-cp.h"
#include "ipa-prop.h"
+#include "attr-callback.h"
+#include "callback-info.h"
static void ipa_sra_summarize_function (cgraph_node *);
@@ -185,6 +187,10 @@ struct GTY(()) isra_param_desc
unsigned locally_unused : 1;
/* An aggregate that is a candidate for breaking up or complete removal. */
unsigned split_candidate : 1;
+ /* True iff the parameter should not be disqualified as a split candidate.
+ Only useful for forcing transformations, such as when optimizing .ASSUME
+ bodies. */
+ unsigned dont_disqualify : 1;
/* Is this a parameter passing stuff by reference? */
unsigned by_ref : 1;
/* If set, this parameter can only be a candidate for removal if the function
@@ -233,6 +239,10 @@ struct gensum_param_desc
by reference that is a candidate for being converted to a set of
parameters passing those data by value. */
bool split_candidate;
+ /* True iff the parameter should not be disqualified as a split candidate.
+ Only useful for forcing transformations, such as when optimizing .ASSUME
+ bodies. */
+ bool dont_disqualify;
/* Is this a parameter passing stuff by reference (either a pointer or a
source language reference type)? */
bool by_ref;
@@ -461,6 +471,7 @@ ipa_sra_function_summaries::duplicate (cgraph_node *,
cgraph_node *,
d->safe_size = s->safe_size;
d->locally_unused = s->locally_unused;
d->split_candidate = s->split_candidate;
+ d->dont_disqualify = s->dont_disqualify;
d->by_ref = s->by_ref;
d->remove_only_when_retval_removed = s->remove_only_when_retval_removed;
d->split_only_when_retval_removed = s->split_only_when_retval_removed;
@@ -1131,12 +1142,13 @@ ptr_parm_has_nonarg_uses (cgraph_node *node, function
*fun, tree parm,
cgraph_edge *cs = node->get_edge (stmt);
gcc_checking_assert (cs);
+ callback_info *ci = callback_info_sum->get (cs);
isra_call_summary *csum = call_sums->get_create (cs);
csum->init_inputs (arg_count);
for (unsigned i = 0; i < arg_count; ++i)
{
- tree arg = gimple_call_arg (stmt, i);
+ tree arg = gimple_call_arg_summary (stmt, i, ci);
if (arg == name)
{
@@ -1216,6 +1228,7 @@ create_parameter_descriptors (cgraph_node *node,
gensum_param_desc *desc = &(*param_descriptions)[num];
/* param_descriptions vector is grown cleared in the caller. */
desc->param_number = num;
+ desc->dont_disqualify = node->assume_fn_p ();
decl2desc->put (parm, desc);
if (dump_file && (dump_flags & TDF_DETAILS))
@@ -1387,7 +1400,7 @@ get_gensum_param_desc (tree decl)
static void
disqualify_split_candidate (gensum_param_desc *desc, const char *reason)
{
- if (!desc->split_candidate)
+ if (!desc->split_candidate || desc->dont_disqualify)
return;
if (dump_file && (dump_flags & TDF_DETAILS))
@@ -2056,13 +2069,36 @@ scan_function (cgraph_node *node, struct function *fun)
unsigned argument_count = gimple_call_num_args (stmt);
isra_scan_context ctx = ISRA_CTX_ARG;
scan_call_info call_info, *call_info_p = &call_info;
+
if (gimple_call_internal_p (stmt))
{
- call_info_p = NULL;
- ctx = ISRA_CTX_LOAD;
- internal_fn ifn = gimple_call_internal_fn (stmt);
- if (internal_store_fn_p (ifn))
- ctx = ISRA_CTX_STORE;
+ if (gimple_call_internal_fn (stmt) == IFN_ASSUME)
+ {
+ tree callee_decl
+ = TREE_OPERAND (gimple_call_arg (stmt, 0), 0);
+ if (!lookup_attribute ("noipa",
+ DECL_ATTRIBUTES (callee_decl)))
+ {
+ cgraph_node *target
+ = cgraph_node::get_create (callee_decl);
+ cgraph_edge *cbei
+ = cgraph_edge::make_callback_internal (
+ node, target, as_a<gcall *> (stmt));
+ callback_info *ci
+ = callback_info::initialize_internal_summary (
+ cbei, IFN_ASSUME);
+ argument_count = ci->arg_mapping.length ();
+ }
+ }
+ else
+ {
+ call_info_p = NULL;
+ call_info.cs = NULL;
+ ctx = ISRA_CTX_LOAD;
+ internal_fn ifn = gimple_call_internal_fn (stmt);
+ if (internal_store_fn_p (ifn))
+ ctx = ISRA_CTX_STORE;
+ }
}
else
{
@@ -2073,8 +2109,11 @@ scan_function (cgraph_node *node, struct function *fun)
for (unsigned i = 0; i < argument_count; i++)
{
call_info.arg_idx = i;
- scan_expr_access (gimple_call_arg (stmt, i), stmt,
- ctx, bb, call_info_p);
+ callback_info *ci = NULL;
+ if (call_info.cs)
+ ci = callback_info_sum->get (call_info.cs);
+ scan_expr_access (gimple_call_arg_summary (stmt, i, ci),
+ stmt, ctx, bb, call_info_p);
}
tree lhs = gimple_call_lhs (stmt);
@@ -2183,13 +2222,14 @@ ssa_name_only_returned_p (function *fun, tree name,
bitmap analyzed)
static void
isra_analyze_call (cgraph_edge *cs)
{
+ callback_info *ci = callback_info_sum->get (cs);
gcall *call_stmt = cs->call_stmt;
- unsigned count = gimple_call_num_args (call_stmt);
+ unsigned count = gimple_call_num_args_summary (call_stmt, ci);
isra_call_summary *csum = call_sums->get_create (cs);
for (unsigned i = 0; i < count; i++)
{
- tree arg = gimple_call_arg (call_stmt, i);
+ tree arg = gimple_call_arg_summary (call_stmt, i, ci);
if (TREE_CODE (arg) == ADDR_EXPR)
{
poly_int64 poffset, psize, pmax_size;
@@ -2607,7 +2647,8 @@ process_scan_results (cgraph_node *node, struct function
*fun,
const attribute is wrong) and then we just don't care. */
bool uses_memory_as_obtained = vuse && SSA_NAME_IS_DEFAULT_DEF (vuse);
- unsigned count = gimple_call_num_args (call_stmt);
+ callback_info *ci = callback_info_sum->get (cs);
+ unsigned count = gimple_call_num_args_summary (call_stmt, ci);
isra_call_summary *csum = call_sums->get_create (cs);
csum->init_inputs (count);
csum->m_before_any_store = uses_memory_as_obtained;
@@ -2690,6 +2731,7 @@ process_scan_results (cgraph_node *node, struct function
*fun,
d->size_reached = s->nonarg_acc_size;
d->locally_unused = s->locally_unused;
d->split_candidate = s->split_candidate;
+ d->dont_disqualify = s->dont_disqualify;
d->by_ref = s->by_ref;
d->remove_only_when_retval_removed = s->remove_only_when_retval_removed;
d->split_only_when_retval_removed = s->split_only_when_retval_removed;
@@ -2778,6 +2820,7 @@ ipa_sra_generate_summary (void)
= (new (ggc_alloc_no_dtor <ipa_sra_function_summaries> ())
ipa_sra_function_summaries (symtab, true));
call_sums = new ipa_sra_call_summaries (symtab);
+ callback_info_sum_t::check_create_info_sum ();
FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node)
ipa_sra_summarize_function (node);
@@ -2852,6 +2895,7 @@ isra_write_node_summary (output_block *ob, cgraph_node
*node)
bitpack_d bp = bitpack_create (ob->main_stream);
bp_pack_value (&bp, desc->locally_unused, 1);
bp_pack_value (&bp, desc->split_candidate, 1);
+ bp_pack_value (&bp, desc->dont_disqualify, 1);
bp_pack_value (&bp, desc->by_ref, 1);
gcc_assert (!desc->not_specially_constructed);
bp_pack_value (&bp, desc->remove_only_when_retval_removed, 1);
@@ -2978,6 +3022,7 @@ isra_read_node_info (struct lto_input_block *ib,
cgraph_node *node,
bitpack_d bp = streamer_read_bitpack (ib);
desc->locally_unused = bp_unpack_value (&bp, 1);
desc->split_candidate = bp_unpack_value (&bp, 1);
+ desc->dont_disqualify = bp_unpack_value (&bp, 1);
desc->by_ref = bp_unpack_value (&bp, 1);
desc->not_specially_constructed = 0;
desc->remove_only_when_retval_removed = bp_unpack_value (&bp, 1);
@@ -3114,13 +3159,21 @@ ipa_sra_dump_all_summaries (FILE *f, bool hints)
fprintf (f, "\n\n");
}
+/* Return TRUE iff the node can be made local or if it's an artificial .ASSUME
+ body (addressable, but only visible in the current TU). */
+static bool
+can_be_local_p (cgraph_node *n)
+{
+ return n->can_be_local_p () || n->assume_fn_p ();
+}
+
/* Perform function-scope viability tests that can be only made at IPA level
and return false if the function is deemed unsuitable for IPA-SRA. */
static bool
ipa_sra_ipa_function_checks (cgraph_node *node)
{
- if (!node->can_be_local_p ())
+ if (!can_be_local_p (node))
{
if (dump_file)
fprintf (dump_file, "Function %s disqualified because it cannot be "
@@ -3280,6 +3333,9 @@ find_param_access (isra_param_desc *param_desc, unsigned
offset, unsigned size)
static bool
size_would_violate_limit_p (isra_param_desc *desc, unsigned size)
{
+ if (desc->dont_disqualify)
+ return false;
+
unsigned limit = desc->param_size_limit;
if (size > limit
|| (!desc->by_ref && size == limit))
@@ -4028,6 +4084,11 @@ retval_used_p (cgraph_node *node, void *)
return true;
}
+ /* Keep the return value, even though it's not used in the code directly,
+ because it's used by vrp later. */
+ if (node->assume_fn_p ())
+ return true;
+
return false;
}
diff --git a/gcc/testsuite/g++.dg/ipa/ipa-sra-assume.C
b/gcc/testsuite/g++.dg/ipa/ipa-sra-assume.C
new file mode 100644
index 00000000000..21e93ebf610
--- /dev/null
+++ b/gcc/testsuite/g++.dg/ipa/ipa-sra-assume.C
@@ -0,0 +1,37 @@
+/* Test that ipa-sra can work with .ASSUME outlined bodies. */
+
+/* { dg-do compile } */
+/* { dg-options "-O2 -fipa-sra -fdump-ipa-sra-details -fdump-tree-optimized" }
*/
+
+struct S
+{
+ bool x;
+ int y;
+};
+
+void
+split_struct (struct S s)
+{
+ [[assume (!s.x)]];
+ if (s.x)
+ __builtin_abort ();
+}
+
+void
+remove_ptr (bool *x)
+{
+ [[assume (!*x)]];
+ if (*x)
+ __builtin_abort ();
+}
+
+void
+two_values (struct S s)
+{
+ [[assume (s.y == 3 || s.y == 4)]];
+ if (s.y == 5)
+ __builtin_abort ();
+}
+
+/* { dg-final { scan-ipa-dump-times "Will split parameter 0" 3 "sra" } } */
+/* { dg-final { scan-tree-dump-times "__builtin_abort" 0 "optimized" } } */
diff --git a/gcc/tree-inline.cc b/gcc/tree-inline.cc
index 7fecf487af7..9323c2d4046 100644
--- a/gcc/tree-inline.cc
+++ b/gcc/tree-inline.cc
@@ -2893,6 +2893,7 @@ initialize_cfun (tree new_fndecl, tree callee_fndecl,
profile_count count)
cfun->can_delete_dead_exceptions = src_cfun->can_delete_dead_exceptions;
cfun->returns_struct = src_cfun->returns_struct;
cfun->returns_pcc_struct = src_cfun->returns_pcc_struct;
+ cfun->assume_function = src_cfun->assume_function;
init_empty_tree_cfg ();
--
2.52.0
