> > gcc/ChangeLog
> >
> > 2019-12-02 Xiong Hu Luo <luo...@linux.ibm.com>
> >
> > PR ipa/69678
> > * Makefile.in (GTFILES): Add ipa-profile.c.
> > * cgraph.c (symbol_table::create_edge): Init speculative_id.
> > (cgraph_edge::make_speculative): Add param for setting speculative_id.
> > (cgraph_edge::speculative_call_info): Update comments and find reference
> > by speculative_id for multiple indirect targets.
> > (cgraph_edge::resolve_speculation): Decrease the speculations
> > for indirect edge, drop it's speculative if not direct target
> > left. Update comments.
> > (cgraph_edge::redirect_call_stmt_to_callee): Likewise.
> > (cgraph_node::dump): Print num_speculative_call_targets.
> > (cgraph_node::verify_node): Don't report error if speculative
> > edge not include statement.
> > (cgraph_edge::num_speculative_call_targets_p): New function.
> > * cgraph.h (int common_target_id): Remove.
> > (int common_target_probability): Remove.
> > (num_speculative_call_targets): New variable.
> > (make_speculative): Add param for setting speculative_id.
> > (cgraph_edge::num_speculative_call_targets_p): New declare.
> > (speculative_id): New variable.
> > * cgraphclones.c (cgraph_node::create_clone): Clone speculative_id.
> > * ipa-profile.c (struct speculative_call_target): New struct.
> > (class speculative_call_summary): New class.
> > (class speculative_call_summaries): New class.
> > (call_sums): New variable.
> > (ipa_profile_generate_summary): Generate indirect multiple targets
> > summaries.
> > (ipa_profile_write_edge_summary): New function.
> > (ipa_profile_write_summary): Stream out indirect multiple targets
> > summaries.
> > (ipa_profile_dump_all_summaries): New function.
> > (ipa_profile_read_edge_summary): New function.
> > (ipa_profile_read_summary_section): New function.
> > (ipa_profile_read_summary): Stream in indirect multiple targets
> > summaries.
> > (ipa_profile): Generate num_speculative_call_targets from
> > summaries.
> > * ipa-ref.h (speculative_id): New variable.
> > * lto-cgraph.c (lto_output_edge): Remove indirect common_target_id and
> > common_target_probability. Stream out speculative_id and
> > num_speculative_call_targets.
> > (input_edge): Likewise.
> > * predict.c (dump_prediction): Remove edges count assert to be
> > precise.
> > * symtab.c (symtab_node::create_reference): Init speculative_id.
> > (symtab_node::clone_references): Clone speculative_id.
> > (symtab_node::clone_referring): Clone speculative_id.
> > (symtab_node::clone_reference): Clone speculative_id.
> > (symtab_node::clear_stmts_in_references): Clear speculative_id.
> > * tree-inline.c (copy_bb): Duplicate all the speculative edges
> > if indirect call contains multiple speculative targets.
> > * tree-profile.c (gimple_gen_ic_profiler): Use the new variable
> > __gcov_indirect_call.counters and __gcov_indirect_call.callee.
> > (gimple_gen_ic_func_profiler): Likewise.
> > (pass_ipa_tree_profile::gate): Fix comment typos.
> > * value-prof.h (check_ic_target): Remove.
> > * value-prof.c (gimple_value_profile_transformations):
> > Use void function gimple_ic_transform.
> > * value-prof.c (gimple_ic_transform): Handle topn case.
> > Fix comment typos. Change it to a void function.
> >
> > gcc/testsuite/ChangeLog
> >
> > 2019-12-02 Xiong Hu Luo <luo...@linux.ibm.com>
> >
> > PR ipa/69678
> > * gcc.dg/tree-prof/indir-call-prof-topn.c: New testcase.
> > * gcc.dg/tree-prof/crossmodule-indir-call-topn-1.c: New testcase.
> > * gcc.dg/tree-prof/crossmodule-indir-call-topn-1a.c: New testcase.
> > * gcc.dg/tree-prof/crossmodule-indir-call-topn-2.c: New testcase.
> > * lib/scandump.exp: Dump executable file name.
> > * lib/scanwpaipa.exp: New scan-pgo-wap-ipa-dump.
> > -cgraph_edge::make_speculative (cgraph_node *n2, profile_count direct_count)
> > +cgraph_edge::make_speculative (cgraph_node *n2, profile_count direct_count,
> > + unsigned int speculative_id)
Please document speculative_id parameter.
> > +/* Speculative calls represent a transformation of indirect calls
> > + which may be later inserted into gimple in the following form:
> >
> > - Given speculative call edge, return all three components.
> > + if (call_dest == target1)
> > + target1 ();
> > + else if (call_dest == target2)
> > + target2 ();
> > + else
> > + call_dest ();
> > +
> > + This is a win in case when target1 and target2 are common values for
... in the case ...
I guess :)
> > -/* Collect histogram from CFG profiles. */
> > +/* Structure containing speculative target information from profile. */
> > +
> > +struct GTY (()) speculative_call_target
There is no need to place this in the garbage collector, so remove GTY.
> > +{
> > + speculative_call_target (unsigned int id, int prob)
> > + : target_id (id), target_probability (prob)
> > + {
> > + }
> > +
> > + /* Profile_id of target obtained from profile. */
> > + unsigned int target_id;
> > + /* Probability that call will land in function with target_id. */
> > + int target_probability;
> > +};
> > +
> > +class GTY ((for_user)) speculative_call_summary
Similarly here, drop GTY marker.
> > +{
> > +public:
> > + speculative_call_summary () : speculative_call_targets ()
> > + {}
> > +
> > + vec<speculative_call_target, va_gc> *speculative_call_targets;
Making this auto_vec will save the need to deallocate explicitly.
> > + ~speculative_call_summary ();
> > +
> > + void dump (FILE *f);
> > +
> > + /* Check whether this is a empty summary. */
> > + bool is_empty ();
> > +};
> > +
> > + /* Class to manage call summaries. */
> > +
> > +class GTY ((user)) ipa_profile_call_summaries
> > + : public call_summary<speculative_call_summary *>
Remove GTY and make this non-ggc
> > +{
> > +public:
> > + ipa_profile_call_summaries (symbol_table *table, bool ggc)
> > + : call_summary<speculative_call_summary *> (table, ggc)
> > + {}
Do we need the empty constructor at all?
> > +
> > + /* Duplicate info when an edge is cloned. */
> > + virtual void duplicate (cgraph_edge *, cgraph_edge *,
> > + speculative_call_summary *old_sum,
> > + speculative_call_summary *new_sum);
> > +};
> > +
> > +static GTY (()) ipa_profile_call_summaries *call_sums = NULL;
> > +
> > +speculative_call_summary::~speculative_call_summary ()
> > +{
> > + if (speculative_call_targets)
> > + {
> > + vec_free (speculative_call_targets);
> > + speculative_call_targets = NULL;
> > + }
> > +}
This desctuctor is unnecesary if you turn call tarets to auto_vec.
> > +
> > +/* Dump all information in speculative call summary to F. */
> > +
> > +void
> > +speculative_call_summary::dump (FILE *f)
> > +{
> > + speculative_call_target *item;
> > + cgraph_node *n2;
> > + unsigned int i;
> > +
> > + FOR_EACH_VEC_SAFE_ELT (speculative_call_targets, i, item)
> > + {
> > + n2 = find_func_by_profile_id (item->target_id);
> > + if (n2)
> > + fprintf (f, " The %i speculative target is %s with prob %3.2f\n", i,
> > + n2->dump_name (),
> > + item->target_probability / (float) REG_BR_PROB_BASE);
> > + else
> > + fprintf (f, " The %i speculative target is %u with prob %3.2f\n", i,
> > + item->target_id,
> > + item->target_probability / (float) REG_BR_PROB_BASE);
> > + }
> > +}
> > +
> > +/* Check whether this is a empty summary. */
> > +bool
> > +speculative_call_summary::is_empty ()
> > +{
> > + return speculative_call_targets == NULL
> > + || speculative_call_targets->is_empty ();
> > +}
We probably do not want to have empty summaries around, so the second
test chould be assert?
> > +
> > +/* Duplicate info when an edge is cloned. */
> > +
> > +void
> > +ipa_profile_call_summaries::duplicate (cgraph_edge *, cgraph_edge *,
> > + speculative_call_summary *old_sum,
> > + speculative_call_summary *new_sum)
> > +{
> > + if (!old_sum || !old_sum->speculative_call_targets)
> > + return;
> > +
> > + speculative_call_target *item;
> > + unsigned int i;
> > +
> > + FOR_EACH_VEC_SAFE_ELT (old_sum->speculative_call_targets, i, item)
> > + {
> > + vec_safe_push (new_sum->speculative_call_targets, *item);
> > + }
vec_copy will do.
> > diff --git a/gcc/ipa-ref.h b/gcc/ipa-ref.h
> > index 00af24c77db..b68661d45c8 100644
> > --- a/gcc/ipa-ref.h
> > +++ b/gcc/ipa-ref.h
> > @@ -47,30 +47,31 @@ public:
> > bool cannot_lead_to_return ();
> >
> > /* Return true if reference may be used in address compare. */
> > bool address_matters_p ();
> >
> > /* Return reference list this reference is in. */
> > struct ipa_ref_list * referring_ref_list (void);
> >
> > /* Return reference list this reference is in. */
> > struct ipa_ref_list * referred_ref_list (void);
> >
> > symtab_node *referring;
> > symtab_node *referred;
> > gimple *stmt;
> > unsigned int lto_stmt_uid;
> > + unsigned int speculative_id;
Please add documentation.
I would also make them 16bit and packed with bitfeilds to be consistent
with cgraph edges.
ipa_refs are quite critical for memory use of WPA stage. I see it will
not save memory now but it will help us to not forget about the fact
that they have limited value range in future.
> >
> > /* Speculative calls consist of two edges - direct and
> > indirect. Duplicate the whole thing and distribute
> > frequencies accordingly. */
This comment is out of date now.
> > if (edge->speculative)
> > {
> > struct cgraph_edge *direct, *indirect;
> > struct ipa_ref *ref;
> >
> > gcc_assert (!edge->indirect_unknown_callee);
> > old_edge->speculative_call_info (direct, indirect,
> > ref);
> > + while (old_edge->next_callee
> > + && old_edge->next_callee->speculative
> > + && indirect->num_speculative_call_targets_p ()
> > + > 1)
> > + {
> > + id->dst_node->clone_reference (ref, stmt);
> > +
> > + edge = old_edge->next_callee;
> > + edge = edge->clone (id->dst_node, call_stmt,
> > + gimple_uid (stmt), num, den,
> > + true);
> > + old_edge = old_edge->next_callee;
> > + gcc_assert (!edge->indirect_unknown_callee);
> > +
> > + /* If the indirect edge has multiple speculative
> > + calls, iterate through all direct calls
> > + associated to the speculative call and clone
> > + all related direct edges before cloning the
> > + related indirect edge. */
> > + old_edge->speculative_call_info (direct, indirect,
> > + ref);
> > + }
> >
> > profile_count indir_cnt = indirect->count;
> > indirect = indirect->clone (id->dst_node, call_stmt,
> > gimple_uid (stmt),
> > num, den,
> > true);
> >
> > profile_probability prob
> > = indir_cnt.probability_in (old_cnt + indir_cnt);
> > indirect->count
> > = copy_basic_block->count.apply_probability (prob);
The udpating of indirect edges needs to be done in loop as well so we
update all of them.
I think you are also not removing the common_target and
common_target_probability from cgraph_edge.
There is now code in ipa-utils merging the histograms. You will need to
update that to your representation. It should not be hard - it either
copies all the values from target function or merges them with given
probability scales.
Honza
