Re: [RFC] Return Value Propagation in IPA-CP

[Jan Hubicka via Gcc]

> Ping (https://gcc.gnu.org/pipermail/gcc/2024-August/244625.html).
Hi,
> > * Proposed Solution *
> > 
> > * Extending IPA-CP:
> > 
> > 1. Add return jump function data structures
> >    - This involves updating ipa_node_params to contain information 
> > regarding the
> >      return statements of the function, namely the lattice and the jump 
> > function
> >      describing the return value, where both use existing data structures.
> >    - The ipa_node_params class is reused to avoid introducing a new class 
> > and a
> >      corresponding function_summary type, though it is trivial to add if 
> > deemed
> >      a better solution. The ipa_return_value_summary class may also be a 
> > good
> >      place for this information, however using it would involve moving it 
> > from
> >      ipa-prop.cc to ipa-prop.h.

Yes, I think the jump functions for return values are not realy
different from jump functions for call parameters, so indeed we want to
reuse same datatstructure.
We want to track all the usual stuff we do (value ranges, constant
values, known fields in structures...)

In longer run we will want to extend jump functions to represent
properties like "value returned by call X is passed as argument n to
call Y". But for the first version we probably do not need to worry
about this.
> >    - Additionally, ipa_edge_args is updated to track whether or not it is a
> >      callgraph edge originating from a return statement. This enables the
> >      propagation of information in the WPA phase.

What you exactly mean by callgraph edge originating from return
statement?
> > 
> > 2. LGEN
> >    - Set up return jump functions for each function similar to the parameter
> >      jump function. When it cannot be conclusively determined that one 
> > formal
> >      parameter is always returned (such as conditionally returning either of
> >      two), the jump function is marked as invalid.
> >    - This involves looking through phi nodes when return statements of
> >      conditional branches are merged into a single exit block. Note that
> >      returning a call expression does not count towards invalidating the
> >      information for that function.

The analysis is essentially the same as for function parameters, just
looking at the return statements, so I think most of code can be shared
with what we have.
> > 
> > 3. WPA
> >    - Implement return value information propagation. It is not possible to 
> > do
> >      this in the existing propagate_constants_topo function, because it 
> > iterates
> >      in reverse postorder i.e. from caller to callee. However return value
> >      propagation needs to be done from callee to caller, thus there is a 
> > need to
> >      iterate in a postorder fashion.
> >    - The lattice values are initialized using the jump functions computed 
> > in the
> >      LGEN phase. These values are then propagated over the callgraph.
> >    - The existing lattices are reused, with three possible values like 
> > usual.
> >      The possible values are:
> > 
> >    return_lattice -> { TOP, param_decl, BOTTOM }
> > 
> >      where TOP and BOTTOM are defined as usual. param_decl refers to the 
> > tree of
> >      either the parameter declaration or its type, whichever is available. 
> > The
> >      meet operator is defined as usual for TOP and BOTTOM. When both are
> >      param_decl, the following meet operation is defined:
> > 
> >    meet(x, y) = x if x == y, BOTTOM otherwise
> > 
> >    - Finally, nodes to which no information could be propagated are marked 
> > as
> >      BOTTOM and the BOTTOM value is propagated to a fixed-point. This is 
> > required
> >      when information about a caller has been inferred but not about one of 
> > its
> >      callees. For example:
> > 
> >    extern int foo(int);
> > 
> >    __attribute__((noinline))
> >    int bar (int z)
> >    {
> >      return foo (z);
> >    }
> > 
> >    __attribute__((noinline))
> >    int baz (int z)
> >    {
> >      return z;
> >    }
> > 
> >    int quux (int z)
> >    {
> >      if (z & 1)
> >    return bar (z);
> >      else
> >    return baz (z);
> >    }
> > 
> >    In this case, after propagation, the lattice values look like the 
> > following:
> >    - bar/ret  ->  TOP
> >    - baz/ret  ->  baz/0  (first parameter being returned)
> >    - quux/ret ->  quux/0 (lattice partially inferred from baz)
> > 
> >    Here, we are not able to infer any information about bar because it 
> > calls a
> >    function foo that is not visible to the compiler. If we only set bar to
> >    BOTTOM and do not propagate this BOTTOM value to quux, we will 
> > incorrectly
> >    infer that quux always returns its 1st parameter.

Yep, it is essentially a simple dataflow.  propagate_constants_topo
can probably be still saved by simply considering calls with interesting
return functions as bi-directional for discovery of strongly connected
component.
Somewhat tricky will be implementing cloning heuristi

Re: LTO progress indicator

[Jan Hubicka via Gcc]

> On Sat, Sep 14, 2024 at 1:17 PM Ghorban M. Tavakoly via Gcc
>  wrote:
> >
> > >> Is there any change to have some LTO progress indicator information in
> > upstream GCC output? Do I need to report a bug?
> > Is there any chance ... (sorry for typo)
> 
> You can add -Q to the command line which makes GCC output some sort
> of progress indication.  Otherwise no - we do not really know how long a
> compile will take so a true progress indicator isn't possible.

For ltrans we do make estimate used by partitioning, so for large
projects that consits of many functions we may be able to do some sort
of progress info on ltrans stage.

But indeed, when waiting for large copmiles, I use -Q to make myself
entertained :)

Honza

GNU Tools Cauldron 2024 (2nd announcement)

[Jan Hubicka via Gcc]

Hello,
we are pleased to invite you all to the next GNU Tools Cauldron,
taking place in Prague, Czech Republic, on September 14-16, 2024.

As for the previous instances, we have setup a wiki page for
details:

  https://gcc.gnu.org/wiki/cauldron2024


Like last year, we are having to charge for attendance. It is 2300 CZK
before August 20 and 2900 CZK afterwards. The payment gate will open
within a week and we will send email to everyone pre-registered.

The GNU Toolchain Fund (https://gcc.gnu.org/wiki/GNUToolchainFund)
sponsors developer travel to attend the GNU Tools Cauldron.
Having developers meet face to face is important and builds connections
in our communities.

For all details of how to register, and how to submit a proposal for a  
track session, please see the wiki page.

The Cauldron is organized by a group of volunteers. We are keen to add
some more people so others can stand down. If you'd like to be part of
that organizing committee, please email the same address.

This announcement is being sent to the main mailing list of the
following groups: GCC, GDB, binutils, CGEN, DejaGnu, newlib and glibc.

Please feel free to share with other groups as appropriate.

Honza (on behalf of the GNU Tools Cauldron organizing committee).

GNU Tools Cauldron 2024

[Jan Hubicka via Gcc]

Hello,
we are pleased to invite you all to the next GNU Tools Cauldron,
taking place in Prague, Czech Republic, on September 14-16, 2024.

As for the previous instances, we have setup a wiki page for
details:

  https://gcc.gnu.org/wiki/cauldron2024


Like last year, we are having to charge for attendance.  We are still
working out what we will need to charge.

Attendance will remain free for community volunteers and others who do
not have a commercial backer and we will be providing a small number of
travel bursaries for students to attend.

For all details of how to register, and how to submit a proposal for a  
track session, please see the wiki page.

The Cauldron is organized by a group of volunteers. We are keen to add
some more people so others can stand down. If you'd like to be part of
that organizing committee, please email the same address.

This announcement is being sent to the main mailing list of the
following groups: GCC, GDB, binutils, CGEN, DejaGnu, newlib and glibc.

Please feel free to share with other groups as appropriate.

Honza (on behalf of the GNU Tools Cauldron organizing committee).

Re: How stable is the CFG and basic block IDs?

[Jan Hubicka via Gcc]

> > The problem is testing. If gcc would re-number the basic blocks then
> > tests comparing hard-coded test paths would break, even though the path
> > coverage itself would be just fine (and presumably the change to the
> > basic block indices), which would add an unreasonable maintenance
> > burden. If anything, it feels very fragile to write tests against the
> > basic block indices.
> 
> Problematic is usually when early canonicalization changes the
> direction of a branch which affects the block IDs of the true/false
> destinations (and their downstream blocks).

I think we can renumber Bbs sequentially before profile generation (or
maybe we do already).  But indeed the CfG may change with time.
> 
> > On the other hand, if it can be expected that the same code should
> > always yield the same CFG, the same BBs, and the same BB indices then I
> > would happily test against them. I suppose this makes the basic blocks
> > basically a public interface, which granted feels odd.
> >
> > If you have any good idea on how to test paths in a robust way please
> > let me know.
> 
> Is there enough info to print a path like the following?
> 
> path not covered: t.c:2(true) t.c:4(false) t.c:11(true) ...
> 
> instead of printing (condition destination?) block IDs?

This was my first idea too.  Thre can be multiple BBs per line but you
can print both BB ID and source file nameand ignore the BB ID for
testsuite pruposes.

Honza
> 
> Richard.
> 
> >
> > Thanks,
> > Jørgen

Re: libgcov, fork, and mingw (and other targets without the full POSIX set)

[Jan Hubicka via Gcc]

> On Dez 01 2023, Richard Biener via Gcc wrote:
> 
> > Hmm, so why's it then referenced and not "GCed"?
> 
> This has nothing to do with garbage collection.  It's just the way
> libgcc avoids having too many source files.  It would be exactly the
> same if every function were in its own file.

THe ifdef machinery makes every function to go insto its own .o file
which are then archived.  So if user code never calls to fork, the .o
file with fork wrapper should not be picked by linker and we should not
have link error.

If user code calls fork, then the .o file with wrapper should be picked
and we will get linker error on missing fork.  So I think it ought to
work as it is now.  Does mingw linker behave somehow differently with
archives?  Or is there problem with a libgcov being DLL or something?

Honza
> 
> -- 
> Andreas Schwab, sch...@linux-m68k.org
> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
> "And now for something completely different."

Re: Help needed in output relocations

[Jan Hubicka via Gcc]

> Hello,
Hi,
> I have almost completed the output of relocation entries. The only thing
> that remains is to output the corresponding symbols in .symtab. In my
> current design, I store the info about relocation entry and the symbol
> name. However, the problem I am facing with this approach is that many
> relocation entries will have the same name, so we will need a hash table
> with the key as symbol name and value as symbol index in symtab. It would
> be really helpful if you could point out the relevant docs or help me
> figure out how to use the hash table in GCC.
> Another approach is, as we have only 4-5 unique relocation symbol names. We
> can output just those and store their .symtab index somewhere.

I am not 100% sure what precisely you need, but if you need just
hashtable translating strings to integers, you can probably just
follow section_name_hash in symtab.cc which is doing pretty much
the same.

In case you can already rely on the fact that the strings are unified
(which is the case for GCC's identifiers) you can also simply use
hash_map which has bit easier to set up.

Honza
> 
> --
> Rishi

Re: Clarification regarding various classes DIE's attribute value class

[Jan Hubicka via Gcc]

> Hello,
> I am working on a project to produce the LTO object file from the compiler
> directly. So far, we have
> correctly outputted .symtab along with various .debug sections. The only
> thing remaining is to
> correctly output attribute values and their corresponding values in the
> .debug_info section. This is done by the output_die function in
> dwarf2out.cc based on the value's class. However, the same
> function is used in dwarf2out_finish  as well as dwarf2out_early_finish, so
> I suspect that not every value class is being used in dwarfout_early_finish
> (mainly I am interested in -flto mode). As there is little documentation on
> the same, I experimented by commenting out the various cases of value class
> in output die. I found that the classes such as dw_val_class_addr,
> dw_val_class_high_pc, and dw_val_class_vms_delta aren't being used during
> the early_finish of LTO mode. I might be wrong, as my observation is based
> on commenting out and testing a few pieces of code that might need to be
> completed. So, can anyone please tell out of these 30 classes that are
> relevant to dwarf2out_early_finish in LTO mode or at least point out some
> documentation if it exists?

You can probably do gcc_assert (!in_lto_p && flag_lto && !flag_fat_lto_objects)
on parts of output_die which you think are unused and then do make check
and if it passes also make bootstrap.  That should probably catch all
relevant cases.

Honza
> enum dw_val_class
> {
>   dw_val_class_none,
>   dw_val_class_addr,
>   dw_val_class_offset,
>   dw_val_class_loc,
>   dw_val_class_loc_list,
>   dw_val_class_range_list,
>   dw_val_class_const,
>   dw_val_class_unsigned_const,
>   dw_val_class_const_double,
>   dw_val_class_wide_int,
>   dw_val_class_vec,
>   dw_val_class_flag,
>   dw_val_class_die_ref,
>   dw_val_class_fde_ref,
>   dw_val_class_lbl_id,
>   dw_val_class_lineptr,
>   dw_val_class_str,
>   dw_val_class_macptr,
>   dw_val_class_loclistsptr,
>   dw_val_class_file,
>   dw_val_class_data8,
>   dw_val_class_decl_ref,
>   dw_val_class_vms_delta,
>   dw_val_class_high_pc,
>   dw_val_class_discr_value,
>   dw_val_class_discr_list,
>   dw_val_class_const_implicit,
>   dw_val_class_unsigned_const_implicit,
>   dw_val_class_file_implicit,
>   dw_val_class_view_list,
>   dw_val_class_symview
> };
> 
> --
> Rishi

Re: Incremental LTO Project

[Jan Hubicka via Gcc]

> Hi!
> 
> On 2023-09-07T19:00:49-0400, James Hu via Gcc  wrote:
> > I noticed that adding incremental LTO was a GSoC project that was not
> > claimed this cycle (
> > https://summerofcode.withgoogle.com/programs/2023/organizations/gnu-compiler-collection-gcc).
> > I was curious about working on this project, but wanted to check on the
> > state of the project.
> 
> Thanks for your interest!  (... as a potential contributor, I presume?)
> 
> > Has it already been completed? Is someone actively
> > working on it?
> 
> Yesterday, when browsing the schedule of the GNU Tools Cauldron 2023,
> , I noticed there's going to be a
> presentation on "Incremental LTO in GCC" (Michal Jireš),
> .

Indeed Michal Jires (who is my student at Charles University) did a lot
of work on incremental LTO.  He is finishing his second bachelor in
physics and then we plan to start working towards contributing it to the
mainline.
His imlementation is described in thesis
https://dspace.cuni.cz/bitstream/handle/20.500.11956/183051/130360194.pdf?sequence=1

This is just a start of the project, further improvemnts will be welcome

Honza
> 
> > If not, what would be the appropriate method to contact the
> > mentor (Jan Hubička)?
> 
> He's reading this list, but I've now also put Honza in CC.
> 
> 
> Grüße
>  Thomas
> -
> Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 
> München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas 
> Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht 
> München, HRB 106955

Re: [Predicated Ins vs Branches] O3 and PGO result in 2x performance drop relative to O2

[Jan Hubicka via Gcc]

> > If I comment it out as above patch, then O3/PGO can get 16% and 12% 
> > performance
> > improvement compared to O2 on x86.
> >
> > O2  O3  PGO
> > cycles  2,497,674,824   2,104,993,224   2,199,753,593
> > instructions10,457,508,646  9,723,056,131   10,457,216,225
> > branches2,303,029,380   2,250,522,323   2,302,994,942
> > branch-misses   0.00%   0.01%   0.01%
> >
> > The main difference in the compilation output about code around the 
> > miss-prediction
> > branch is:
> >   o In O2: predicated instruction (cmov here) is selected to eliminate above
> > branch. cmov is true better than branch here.
> >   o In O3/PGO: bitout() is inlined into encode_file(), and branch 
> > instruction
> > is selected. But this branch is obviously *unpredictable* and the 
> > compiler
> > doesn't know it. This why O3/PGO are are so bad for this program.
> >
> > Gcc doesn't support __builtin_unpredictable() which has been introduced by 
> > llvm.
> > Then I tried to see if __builtin_expect_with_probability(e,x, 0.5) can 
> > serve the
> > same purpose. The result is negative.
> 
> But does it appear to be predictable with your profiling data?

Also one thing is that __builtin_expect and
__builtin_expect_with_probability only affects the static branch
prediciton algorithm, so with profile feedback they are ignored on every
branch executed at least once during the train run.

setting probability 0.5 is really not exactly the same as hint that the
branch will be mispredicted, since modern CPUs handle well regularly
behaving branchs (such as a branch firing every even iteration of loop).

So I think having the builting is not a bad idea.  I was thinking if it
makes sense to represent it withing profile_probability type and I am
not convinced, since "unpredictable probability" sounds counceptually
odd and we would need to keep the flag intact over all probability
updates we do.  For things like loop exits we recompute probabilities
from frequencies after unrolling/vectorizaiton and other things and we
would need to invent new API to propagate the flag from previous
probability (which is not even part of the computation right now)

So I guess the challenge is how to pass this info down through the
optimization pipeline, since we would need to annotate gimple
conds/switches and manage it to RTL level.  On gimple we have flags and
on rtl level notes so there is space for it, but we would need to
maintain the info through CFG changes.

Auto-FDO may be interesting way to detect such branches.

Honza
> 
> > I think we could come to a conclusion that there must be something can 
> > improve in
> > Gcc's heuristic strategy about Predicated Instructions and branches, at 
> > least
> > for O3 and PGO.
> >
> > And can we add __builtin_unpredictable() support for Gcc? As usually it's 
> > hard
> > for the compiler to detect unpredictable branches.
> >
> > --
> > Cheers,
> > Changbin Du

Re: Regarding bypass-asm patch and help in an error during bootstrapped build

[Jan Hubicka via Gcc]

> Hi,
> 
> I have added the patch (
> https://gcc.gnu.org/pipermail/gcc-patches/2023-July/623379.html ) on the
> devel/bypass-asm branch.
> Although I am able to build using the --disable-bootstrap option but while
> doing a bootstrapped build, I am getting these errors ( as warnings while
> doing the non-bootstrapped build.)
> 
> In file included from ../../gcc/gcc/lto-object.cc:23:0:
> ../../gcc/gcc/is-a.h:196:22: error: inline function ‘static bool
> is_a_helper::test(U*) [with U = symtab_node; T = cgraph_node*]’ used but
> never defined [enabled by default]
>static inline bool test (U *p);
> 
>   ^
> ../../gcc/gcc/is-a.h:196:22: error: inline function ‘static bool
> is_a_helper::test(U*) [with U = symtab_node; T = varpool_node*]’ used
> but never defined [enabled by default]
> 
> 
> In file included from ../../gcc/gcc/coretypes.h:489:0,
>  from ../../gcc/gcc/lto/lto-lang.cc:23:
> ../../gcc/gcc/is-a.h:196:22: error inline function ‘static bool
> is_a_helper::test(U*) [with U = symtab_node; T = cgraph_node*]’ used but
> never defined [enabled by default]
>static inline bool test (U *p);
>   ^
> ../../gcc/gcc/is-a.h:196:22: error inline function ‘static bool
> is_a_helper::test(U*) [with U = symtab_node; T = varpool_node*]’ used
> but never defined [enabled by default]
> 
> I have tested the .symtab and dummy symbols addition on the
> non-bootstrapped build, and they are working fine. I also ran the lto test
> suite, and it passed as expected. I am looking into the error produced
> during bootstrapped build currently. I would appreciate any help/guidance
> regarding this.
This is because you miss some #inline or you do these in wrong order
(at some point it was decided to drop most #inlines inside header files,
so one needs to always do the transitive closure by  hand which is quite
anoying).

This is a conversion helper from cgraph_node to symtab_node, so pehraps
you need to include cgraph.h?

Honza
> 
> --
> Regards
> Rishi

Re: Patch regarding addition of .symtab while generating object file from libiberty [WIP]

[Jan Hubicka via Gcc]

> > > +simple_object_write_add_symbol(simple_object_write *sobj, const char
> > *name,
> > > +size_t size, unsigned int align);
> >
> > Symbols has much more properties in addition to sizes and alignments.
> > We will eventually need to get dwarf writting, so we will need to
> > support them. However right now we only do these fake lto object
> > symbols, so perhaps for start we could kep things simple and assume that
> > size is always 0 and align always 1 or so.
> >
> > Overall this looks like really good start to me (both API and
> > imllementation looks reasonable to me and it is good that you follow the
> > coding convention).  I guess you can create a branch (see git info on
> > the gcc homepage) and put the patch there?
> >
> I can, but I don't have write access to git.  Also, for the next part, I am
This is easy to fix. Follow instructions here
https://gcc.gnu.org/gitwrite.html
and add me as sponsor.  There are also instructions how to produce a
branch on git correctly.
> thinking of properly configuring the driver
> to directly produce an executable or adding debug info. What do you think?

I think we should try to aim to something that can be tested.  If we add
symbol table and get the ELF header right, linker should accept the
object files and properly recognize them as LTO.

What exactly you mean by configuring the driver?

Honza
> --
> Regards
> Rishi
> 
> >
> > I am also adding Ian to CC as he is maintainer of the simple-object and
> > he may have some ideas.
> >
> > Honza
> > >
> > >  /* Release all resources associated with SIMPLE_OBJECT, including any
> > > simple_object_write_section's that may have been created.  */
> > > diff --git a/libiberty/simple-object-common.h
> > > b/libiberty/simple-object-common.h
> > > index b9d10550d88..df99c9d85ac 100644
> > > --- a/libiberty/simple-object-common.h
> > > +++ b/libiberty/simple-object-common.h
> > > @@ -58,6 +58,24 @@ struct simple_object_write_struct
> > >simple_object_write_section *last_section;
> > >/* Private data for the object file format.  */
> > >void *data;
> > > +  /*The start of the list of symbols.*/
> > > +  simple_object_symbol *symbols;
> > > +  /*The last entry in the list of symbols*/
> > > +  simple_object_symbol *last_symbol;
> > > +};
> > > +
> > > +/*A symbol in object file being created*/
> > > +struct simple_object_symbol_struct
> > > +{
> > > +  /*Next in the list of symbols attached to an
> > > +  simple_object_write*/
> > > +  simple_object_symbol *next;
> > > +  /*The name of this symbol. */
> > > +  char *name;
> > > +  /* Symbol value */
> > > +  unsigned int align;
> > > +  /* Symbol size */
> > > +  size_t size;
> > >  };
> > >
> > >  /* A section in an object file being created.  */
> > > diff --git a/libiberty/simple-object-elf.c
> > b/libiberty/simple-object-elf.c
> > > index eee07039984..cbba88186bd 100644
> > > --- a/libiberty/simple-object-elf.c
> > > +++ b/libiberty/simple-object-elf.c
> > > @@ -787,9 +787,9 @@ simple_object_elf_write_ehdr (simple_object_write
> > > *sobj, int descriptor,
> > >  ++shnum;
> > >if (shnum > 0)
> > >  {
> > > -  /* Add a section header for the dummy section and one for
> > > - .shstrtab.  */
> > > -  shnum += 2;
> > > +  /* Add a section header for the dummy section,
> > > + .shstrtab, .symtab and .strtab.  */
> > > +  shnum += 4;
> > >  }
> > >
> > >ehdr_size = (cl == ELFCLASS32
> > > @@ -882,6 +882,51 @@ simple_object_elf_write_shdr (simple_object_write
> > > *sobj, int descriptor,
> > > errmsg, err);
> > >  }
> > >
> > > +/* Write out an ELF Symbol*/
> > > +
> > > +static int
> > > +simple_object_elf_write_symbol(simple_object_write *sobj, int
> > descriptor,
> > > +off_t offset, unsigned int st_name, unsigned int st_value,
> > > size_t st_size,
> > > +unsigned char st_info, unsigned char st_other, unsigned int
> > > st_shndx,
> > > +const char **errmsg, int *err)
> > > +{
> > > +  struct simple_object_elf_attributes *attrs =
> > > +(struct simple_object_elf_attributes *) sobj->data;
> > > +  const struct elf_type_functions* fns;
> > > +  unsigned char cl;
> > > +  size_t sym_size;
> > > +  unsigned char buf[sizeof (Elf64_External_Shdr)];
> > > +
> > > +  fns = attrs->type_functions;
> > > +  cl = attrs->ei_class;
> > > +
> > > +  sym_size = (cl == ELFCLASS32
> > > +   ? sizeof (Elf32_External_Shdr)
> > > +   : sizeof (Elf64_External_Shdr));
> > > +  memset (buf, 0, sizeof (Elf64_External_Shdr));
> > > +
> > > +  if(cl==ELFCLASS32)
> > > +  {
> > > +ELF_SET_FIELD(fns, cl, Sym, buf, st_name, Elf_Word, st_name);
> > > +ELF_SET_FIELD(fns, cl, Sym, buf, st_value, Elf_Addr, st_value);
> > > +ELF_SET_FIELD(fns, cl, Sym, buf, st_size, Elf_Addr, st_size);
> > > +buf[4]=st_info;
> > > +buf[5]=st_other;
> > > +ELF_SET_FIELD(fns, cl, Sym, buf, st_shndx, Elf_Half, st_shndx);
> > > +  }
> > > +  else
> > > +  {
> > > +E

Re: Patch regarding addition of .symtab while generating object file from libiberty [WIP]

[Jan Hubicka via Gcc]

> Hi,
Hi,
I am sorry for late reaction.
> I am working on the GSOC project "Bypass Assembler when generating LTO
> object files." So as a first step, I am adding .symtab along with
> __gnu_lto_slim symbol into it so that at a later stage, it can be
> recognized that this object file has been produced using -flto enabled.
> This patch is regarding the same. Although I am still testing this patch, I
> want general feedback on my code and design choice.
> I have extended simple_object_wrtie_struct to hold a list of symbols (
> similar to sections ). A function in simple-object.c to add symbols. I am
> calling this function in lto-object.cc to add __gnu_lto_v1.
> Right now, as we are only working on ELF support first, I am adding .symtab
> in elf object files only.
> 
> ---
>  gcc/lto-object.cc|   4 +-
>  include/simple-object.h  |  10 +++
>  libiberty/simple-object-common.h |  18 +
>  libiberty/simple-object-elf.c| 130 +--
>  libiberty/simple-object.c|  32 
>  5 files changed, 187 insertions(+), 7 deletions(-)
> 
> diff --git a/gcc/lto-object.cc b/gcc/lto-object.cc
> index cb1c3a6cfb3..680977cb327 100644
> --- a/gcc/lto-object.cc
> +++ b/gcc/lto-object.cc
> @@ -187,7 +187,9 @@ lto_obj_file_close (lto_file *file)
>int err;
> 
>gcc_assert (lo->base.offset == 0);
> -
> +  /*Add __gnu_lto_slim symbol*/
> +  if(flag_bypass_asm)
> +simple_object_write_add_symbol (lo->sobj_w, "__gnu_lto_slim",1,1);

You can probably do this unconditionally.  The ltrans files we produce
are kind of wrong by missing the symbol table currently.
> +simple_object_write_add_symbol(simple_object_write *sobj, const char *name,
> +size_t size, unsigned int align);

Symbols has much more properties in addition to sizes and alignments.
We will eventually need to get dwarf writting, so we will need to
support them. However right now we only do these fake lto object
symbols, so perhaps for start we could kep things simple and assume that
size is always 0 and align always 1 or so.

Overall this looks like really good start to me (both API and
imllementation looks reasonable to me and it is good that you follow the
coding convention).  I guess you can create a branch (see git info on
the gcc homepage) and put the patch there?

I am also adding Ian to CC as he is maintainer of the simple-object and
he may have some ideas.

Honza
> 
>  /* Release all resources associated with SIMPLE_OBJECT, including any
> simple_object_write_section's that may have been created.  */
> diff --git a/libiberty/simple-object-common.h
> b/libiberty/simple-object-common.h
> index b9d10550d88..df99c9d85ac 100644
> --- a/libiberty/simple-object-common.h
> +++ b/libiberty/simple-object-common.h
> @@ -58,6 +58,24 @@ struct simple_object_write_struct
>simple_object_write_section *last_section;
>/* Private data for the object file format.  */
>void *data;
> +  /*The start of the list of symbols.*/
> +  simple_object_symbol *symbols;
> +  /*The last entry in the list of symbols*/
> +  simple_object_symbol *last_symbol;
> +};
> +
> +/*A symbol in object file being created*/
> +struct simple_object_symbol_struct
> +{
> +  /*Next in the list of symbols attached to an
> +  simple_object_write*/
> +  simple_object_symbol *next;
> +  /*The name of this symbol. */
> +  char *name;
> +  /* Symbol value */
> +  unsigned int align;
> +  /* Symbol size */
> +  size_t size;
>  };
> 
>  /* A section in an object file being created.  */
> diff --git a/libiberty/simple-object-elf.c b/libiberty/simple-object-elf.c
> index eee07039984..cbba88186bd 100644
> --- a/libiberty/simple-object-elf.c
> +++ b/libiberty/simple-object-elf.c
> @@ -787,9 +787,9 @@ simple_object_elf_write_ehdr (simple_object_write
> *sobj, int descriptor,
>  ++shnum;
>if (shnum > 0)
>  {
> -  /* Add a section header for the dummy section and one for
> - .shstrtab.  */
> -  shnum += 2;
> +  /* Add a section header for the dummy section,
> + .shstrtab, .symtab and .strtab.  */
> +  shnum += 4;
>  }
> 
>ehdr_size = (cl == ELFCLASS32
> @@ -882,6 +882,51 @@ simple_object_elf_write_shdr (simple_object_write
> *sobj, int descriptor,
> errmsg, err);
>  }
> 
> +/* Write out an ELF Symbol*/
> +
> +static int
> +simple_object_elf_write_symbol(simple_object_write *sobj, int descriptor,
> +off_t offset, unsigned int st_name, unsigned int st_value,
> size_t st_size,
> +unsigned char st_info, unsigned char st_other, unsigned int
> st_shndx,
> +const char **errmsg, int *err)
> +{
> +  struct simple_object_elf_attributes *attrs =
> +(struct simple_object_elf_attributes *) sobj->data;
> +  const struct elf_type_functions* fns;
> +  unsigned char cl;
> +  size_t sym_size;
> +  unsigned char buf[sizeof (Elf64_External_Shdr)];
> +
> +  fns = attrs->type_functions;
> +  cl = attrs->ei_class;
> +
> +  sym_size = (cl == E

Re: About addition of .symtab and .strtab sections in simple-object-elf.c

[Jan Hubicka via Gcc]

> Hi Everyone,
Hello,
> I am working on the GSOC project "Bypass Assembler when generating LTO
> object files." My mentors and I have decided to work on the ELF files
> first, so I will add .symtab along with the symbol __gnu_lto_slim to
> the ELF file as a first step.
> When I was going through the simple-object-elf.c:
> simple_object_elf_write_to_file() I found out that it writes the
> following:
> /* Write out a complete ELF file.
>Ehdr
>initial dummy Shdr
>user-created Shdrs
>.shstrtab Shdr
>user-created section data
>.shstrtab data  */
> and .symtab is missing here. To add the missing symtab I have thought
> of these two possible implementations.
> 1) Add it in simple-object-elf.c (based on -fbypass-asm flag).
> 2) We can add .symtab section in lto-object.cc along with other LTO sections.
> I am a bit skeptical about the second one as .symtab with other lto
> sections might be confusing. Any comments regarding which one should I
> proceed with will be helpful.

I think we need to take into consideration that eventually we want to
support also coff/xcoff and macho so writting symtab section directly
out of lto-object.cc (which is supposed to be file format agnostic) is
hard.  On the other hand libiberty knows nothing about internals of LTO
implementatoin.
I think we want to extend simple-object.c by (simple at first at
least) API to add symbols to the symbol table and interface it to
simple-object-elf.c via simple_object_functions. That way
lto-object.cc (which knows the symbol names but no particular file
format) can call generic simple-object.cc API which will in turn call
simple-object-elf.c to create the symtab section.

Honza
> --
> Thanks & Regards
> Rishi Raj

Re: [GSOC] few question about Bypass assembler when generating LTO object files

[Jan Hubicka via Gcc]

> Thanks to Martin, Honza, and Théo for your feedback. I have incorporated
> almost all of it, updated my proposal accordingly, and submitted it.
> Regarding grammar errors, I have fixed many, but there may still be some
> left (I could be better at grammar, to be honest :( ).

I could be better too, I think grammar is not critical here.
Thanks a lot for making and submitting the proposal.

Honza
> 
> On Tue, 4 Apr 2023 at 15:55, Jan Hubicka  wrote:
> 
> > Hello,
> > > Thanks, Jan for the Reply! I have completed a draft proposal for this
> > > project. I will appreciate your's, Martin's, or anybody else feedback on
> > > the same.
> > > Here is the link to my proposal
> > >
> > https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing
> >
> > Here are few comments on the proposal:
> >
> > > The current Implementation of GCC first write the IL representation
> > along with other section in an assembly file and then the assembler is used
> > to convert it into LTO object files. Sections containing different IL
> > representation is created and data is appended in lto-streamer-out.cc.I
> >
> > The .o generated withh -flto file contains the IL (in different
> > sections), debug info, symbol table, etc.
> > "along with other section" sounds odd to me. Perhaps sections.
> >
> > Second sentence seems bit odd too. Perhaps "Streaming of individual
> > sections is implemented in lto-streamer-out.cc which can either be used
> > to produce assembly code containing the section data (dumped
> > hexadecimally) or simple-object API provided by libiberty to produce
> > object files directly"
> >
> > > In the slim object file (Default when using -flto, fat lto can be
> > obtained using -ffat-lto-object) some section contains the IL and other
> > contains the info related to architecture, command line options, symbol
> > table, etc.
> >
> > Technically the architecture is part of ELF header and not section
> > itself (I think).
> >
> > There are some other grammar errors, but I am not too good on fixing
> > these, so perhaps Martin can help.
> >
> > The timeline looks reasonable.  It certianly makes sense to look into
> > non-ELF object files to understand what API we need, but implementation
> > wise I would suggest implementing ELF path first to get a working
> > implementation. Adding support for other object formats can be done
> > incrementally.
> >
> > Honza
> > >
> > > On Tue, 4 Apr 2023 at 04:35, Jan Hubicka  wrote:
> > >
> > > > Hello,
> > > > > While going through the patch and simple-object.c I understood that
> > the
> > > > > file simple-object.c is used to handle the object file format.
> > However,
> > > > > this file does not contain all the architecture information required
> > for
> > > > > LTO object files, so the workaround used in the patch is to read the
> > > > > crtbegin.o file and merge the missing attributes. While this
> > workaround
> > > > is
> > > > > functional, it is not optimal, and the ideal solution would be to
> > extend
> > > > > simple-object.c to include the missing information.
> > > >
> > > > Yes, simple-object.c simply uses architecture settings it read earlier
> > > > which is problem since at compile time we do not read any object files,
> > > > just parse sources). In my original patch the architecture flags were
> > > > simply left blank.  I am not sure if there is a version reading
> > > > crtbeing.o which would probably not a be that bad workaround, at least
> > > > for the start.  Having a way to specify this from the machine
> > descriptions
> > > > would be better.
> > > >
> > >
> > >
> > > >
> > > > Besides the architecture bits, for simple-object files to work we need
> > > > to add the symbol table. For practically useful information we also
> > need
> > > > to stream the debug info.
> > > >
> > > >
> > > > > Regarding the phrase "Support in the driver to properly execute *1
> > > > binary",
> > > > > it is not entirely clear what it refers to. My interpretation is
> > that the
> > > > > compiler driver (the program that coordinates the compilation
> > process)
> > > > > needs to be modified to correctly output LTO object files instead of
> > > > > assembler files (the current approach involves passing the -S and -o
> > > > > .o options) and also skip the assembler option while
> > using
> > > > > -fbypass-asm option but I am not sure. Can Jan or Martin please shed
> > some
> > > > > light on this?
> > > > Yes, compiler drivers decides what to do and it needs to know that with
> > > > -flto it does not need to produce assembly file and then invoke gas.
> > If
> > > > we go the way of reading in crtbegin.o it will also need to pass
> > correct
> > > > crtbegin to *1 binary.  This is generally not that hard to do, just
> > > > needs to be done :)
> > > >
> > > Honza
> > > > >
> > > > > Thanks & Regards
> > > > >
> > > > > Rishi Raj
> > > > >
> > > > > On Sun, 2 Apr 2023 at 03:05, Rishi Raj 
> > wrote:
> > > > >
> > > > > > Hii Everyone

Re: [GSOC] few question about Bypass assembler when generating LTO object files

[Jan Hubicka via Gcc]

Hello,
> Thanks, Jan for the Reply! I have completed a draft proposal for this
> project. I will appreciate your's, Martin's, or anybody else feedback on
> the same.
> Here is the link to my proposal
> https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing

Here are few comments on the proposal:

> The current Implementation of GCC first write the IL representation along 
> with other section in an assembly file and then the assembler is used to 
> convert it into LTO object files. Sections containing different IL 
> representation is created and data is appended in lto-streamer-out.cc.I

The .o generated withh -flto file contains the IL (in different
sections), debug info, symbol table, etc.
"along with other section" sounds odd to me. Perhaps sections.

Second sentence seems bit odd too. Perhaps "Streaming of individual
sections is implemented in lto-streamer-out.cc which can either be used
to produce assembly code containing the section data (dumped
hexadecimally) or simple-object API provided by libiberty to produce
object files directly"

> In the slim object file (Default when using -flto, fat lto can be obtained 
> using -ffat-lto-object) some section contains the IL and other contains the 
> info related to architecture, command line options, symbol table, etc. 

Technically the architecture is part of ELF header and not section
itself (I think).

There are some other grammar errors, but I am not too good on fixing
these, so perhaps Martin can help.

The timeline looks reasonable.  It certianly makes sense to look into
non-ELF object files to understand what API we need, but implementation
wise I would suggest implementing ELF path first to get a working
implementation. Adding support for other object formats can be done
incrementally.

Honza
> 
> On Tue, 4 Apr 2023 at 04:35, Jan Hubicka  wrote:
> 
> > Hello,
> > > While going through the patch and simple-object.c I understood that the
> > > file simple-object.c is used to handle the object file format. However,
> > > this file does not contain all the architecture information required for
> > > LTO object files, so the workaround used in the patch is to read the
> > > crtbegin.o file and merge the missing attributes. While this workaround
> > is
> > > functional, it is not optimal, and the ideal solution would be to extend
> > > simple-object.c to include the missing information.
> >
> > Yes, simple-object.c simply uses architecture settings it read earlier
> > which is problem since at compile time we do not read any object files,
> > just parse sources). In my original patch the architecture flags were
> > simply left blank.  I am not sure if there is a version reading
> > crtbeing.o which would probably not a be that bad workaround, at least
> > for the start.  Having a way to specify this from the machine descriptions
> > would be better.
> >
> 
> 
> >
> > Besides the architecture bits, for simple-object files to work we need
> > to add the symbol table. For practically useful information we also need
> > to stream the debug info.
> >
> >
> > > Regarding the phrase "Support in the driver to properly execute *1
> > binary",
> > > it is not entirely clear what it refers to. My interpretation is that the
> > > compiler driver (the program that coordinates the compilation process)
> > > needs to be modified to correctly output LTO object files instead of
> > > assembler files (the current approach involves passing the -S and -o
> > > .o options) and also skip the assembler option while using
> > > -fbypass-asm option but I am not sure. Can Jan or Martin please shed some
> > > light on this?
> > Yes, compiler drivers decides what to do and it needs to know that with
> > -flto it does not need to produce assembly file and then invoke gas.  If
> > we go the way of reading in crtbegin.o it will also need to pass correct
> > crtbegin to *1 binary.  This is generally not that hard to do, just
> > needs to be done :)
> >
> Honza
> > >
> > > Thanks & Regards
> > >
> > > Rishi Raj
> > >
> > > On Sun, 2 Apr 2023 at 03:05, Rishi Raj  wrote:
> > >
> > > > Hii Everyone,
> > > > I had already expressed my interest in the " Bypass assembler when
> > > > generating LTO object files" project and making a proposal for the
> > same. I
> > > > know I should have done it earlier but I was admitted to the hospital
> > for
> > > > past few days :(.
> > > > I have a few doubts.
> > > > 1)
> > > >
> > > > "One problem is that the object files produced by
> > libiberty/simple-object.c
> > > > (which is the low-level API used by the LTO code)
> > > > are missing some information (such as the architecture info and symbol
> > > > table) and API of the simple object will need to be extended to handle
> > > > that" I found this in the previous mailing list discussion. So who
> > output this information currently in the object file, is it assembler?
> > > >
> > > > Also in the current patch for this project by Jan Hubica, from where
> > 

Re: [GSOC] few question about Bypass assembler when generating LTO object files

[Jan Hubicka via Gcc]

Hello,
> While going through the patch and simple-object.c I understood that the
> file simple-object.c is used to handle the object file format. However,
> this file does not contain all the architecture information required for
> LTO object files, so the workaround used in the patch is to read the
> crtbegin.o file and merge the missing attributes. While this workaround is
> functional, it is not optimal, and the ideal solution would be to extend
> simple-object.c to include the missing information.

Yes, simple-object.c simply uses architecture settings it read earlier
which is problem since at compile time we do not read any object files,
just parse sources). In my original patch the architecture flags were
simply left blank.  I am not sure if there is a version reading
crtbeing.o which would probably not a be that bad workaround, at least
for the start.  Having a way to specify this from the machine descriptions
would be better.

Besides the architecture bits, for simple-object files to work we need
to add the symbol table. For practically useful information we also need
to stream the debug info.
> 
> Regarding the phrase "Support in the driver to properly execute *1 binary",
> it is not entirely clear what it refers to. My interpretation is that the
> compiler driver (the program that coordinates the compilation process)
> needs to be modified to correctly output LTO object files instead of
> assembler files (the current approach involves passing the -S and -o
> .o options) and also skip the assembler option while using
> -fbypass-asm option but I am not sure. Can Jan or Martin please shed some
> light on this?
Yes, compiler drivers decides what to do and it needs to know that with
-flto it does not need to produce assembly file and then invoke gas.  If
we go the way of reading in crtbegin.o it will also need to pass correct
crtbegin to *1 binary.  This is generally not that hard to do, just
needs to be done :)

Honza
> 
> Thanks & Regards
> 
> Rishi Raj
> 
> On Sun, 2 Apr 2023 at 03:05, Rishi Raj  wrote:
> 
> > Hii Everyone,
> > I had already expressed my interest in the " Bypass assembler when
> > generating LTO object files" project and making a proposal for the same. I
> > know I should have done it earlier but I was admitted to the hospital for
> > past few days :(.
> > I have a few doubts.
> > 1)
> >
> > "One problem is that the object files produced by libiberty/simple-object.c
> > (which is the low-level API used by the LTO code)
> > are missing some information (such as the architecture info and symbol
> > table) and API of the simple object will need to be extended to handle
> > that" I found this in the previous mailing list discussion. So who output 
> > this information currently in the object file, is it assembler?
> >
> > Also in the current patch for this project by Jan Hubica, from where are we 
> > getting these information from? Is it from crtbegin.o?
> >
> > 2)
> > "Support in driver to properly execute *1 binary." I found this on Jan 
> > original patch's email. what does it mean
> >
> > exactly?
> >
> > Regards
> >
> > Rishi Raj
> >
> >
> >
> >

Re: cgraph: does node->inlined_to imply node->clones is non-empty?

[Jan Hubicka via Gcc]

Hi,
> 
> It seems to me that the most correct fix is to add to
> walk_polymorphic_call_targets a check that the obtained possible target
> is still referenced_from_vtable_p() - because the alias that was
> originally a virtual function is referenced from a vtable that at this
> point is also known to be gone.  But the check looks like it is possibly
> expensive, so I wanted to discuss this with Honza first (hopefully next
> week).

External vtables are bit special since they can refer to things that are
not accessible in current unit (static functions from other translation
units etc).
We already have and test can_refer_decl_in_current_unit_p but we test it
before alias resolution (which is there just to avoid artifically
bumping up the number of possible targets)

So perhaps following untested patch would work?

diff --git a/gcc/ipa-devirt.cc b/gcc/ipa-devirt.cc
index 14cf132c767..b33ec708d47 100644
--- a/gcc/ipa-devirt.cc
+++ b/gcc/ipa-devirt.cc
@@ -2420,7 +2420,8 @@ maybe_record_node (vec  &nodes,
   alias_target = target_node->ultimate_alias_target (&avail);
   if (target_node != alias_target
  && avail >= AVAIL_AVAILABLE
- && target_node->get_availability ())
+ && target_node->get_availability ()
+ && can_refer_decl_in_current_unit_p (target_node->decl, NULL))
target_node = alias_target;
 }
 
I am at conference and will be able to test it only during weekend.
Honza
> 
> >
> > I had already figured that an error could've likely been in
> > reach-ability analysis, but my time ran low, and I had not confirmed
> > anything, or as little as formalized a theory, so I just wrote the
> > original email instead of following this trail of thought fully.
> >
> > Thank you for your guidance!  Have a lovely night :)
> 
> It is good thing that you asked, I also learned something new (that
> virtual and non-virtual functions can be ICFed together).
> 
> Martin

Re: [GSoC] Introduction and query on LTO object emmission project

[Jan Hubicka via Gcc]

Hello,
> Hi! I've been interested in compiler development for a while, and would love 
> to
> work with any of you as part of GSoC, or even just as a side-project on my 
> own.
> 
> I'm an 18 year-old student going into university next year with a passion for 
> all
> things open source and low level. I consider myself fluent in c, and 
> proficient
> with c++, rust, and x86 assembly, but unfamiliar with practical compiler 
> design.
> I have done some reading on the theoretical aspects of compilers, however.
> 
> While I haven't worked with the GCC community before, I have worked with the 
> linux
> community and have made several small patches there, so I am familiar with 
> both
> email-based workflows and the principles of open-source development. 
> 
> This summer, I'm looking for more experience working on larger projects, as 
> well
> as getting into real compilers.
> 
> Of particular interest to me is the project idea labelled "Bypass assembler 
> when
> generating LTO object files." I see that the project was taken last year, but
> I can find no sign of any changes committed to trunk 
> (`git shortlog --after=2022-01-01 | grep -i -E "lto|assembl(er|y)"` shows 
> nothing
> related to this project) and no sign of any needed change made in the code.
> Is this project still available?

yes, the project is available and Maritn Jambor  and me
would be happy to mentor it. Please add me and Martin to CC of any
emails on the proejt.

I think it would be good to start by looking at the original work in
progress patch (linked from the task description, an updated version is
here https://gcc.gnu.org/pipermail/gcc/2022-May/238670.html).   Overall
struture of the LTO optimization is described in paper
https://arxiv.org/abs/1010.2196

One problem is that the object files produced by libiberty/simple-object.c
(which is the low-level API used by the LTO code)
are missing some information (such as the architecture info and symbol
table) and API of the simple object will need to be extended to handle
that.  So I think getting familiar with simple-object.c and the
elf/mach/coff handlers of it would be a good initital step.  It will be
necessary to add API to specify the architecture and symbols in the
symbol table so the resulting object files are recognized by linker.

Second part would be handling of output of dwarf debug information, so
it may make sense to also take a quick look on its documentation.  The
on-disk representation is actually not too hard and it should be
possible to make a direct writer.

Jan
> 
> I'm also willing to work on other projects, ideally in the middle/backend, but
> currently I have only been experimenting with the gcc/[lto,data]-streamer* 
> files.
> If anyone has a small or medium sized project idea, please feel free to let 
> me know.
> 
> 
> I look forward to working with all of you in the future,
> 
> Peter Lafreniere
> 
> 

Re: Stepping down as gcov maintainer and callgraph reviewer

[Jan Hubicka via Gcc]

Martin,
> Hello GCC community.
> 
> After spending last decade (including my diploma thesis even more)
> of my life working on GCC, I decided to leave the project and try
> a different job. I would like to thank all the community members I had
> change to interact with, I learned so much and enjoyed the journey!
> I'll be leaving somewhen at the beginning of May.
> 
> That said, I'm stepping down from my 2 positions as I won't have a time
> for proper patch review and bugs in the area I'm responsible for.

I am sad to hear this news and will definitely miss you as coleague
and co-maintaner.  Thank you for all the work on GCC!

Honza
> 
> I wish the project all the best!
> 
> Cheers,
> Martin

> From bb3aee20cdeeb6399ca77ac05cd8093d66256df3 Mon Sep 17 00:00:00 2001
> From: Martin Liska 
> Date: Thu, 16 Feb 2023 16:50:38 +0100
> Subject: [PATCH] MAINTAINERS: stepping down from my positions
> 
> ChangeLog:
> 
>   * MAINTAINERS: I'm stepping down from my positions.
> ---
>  MAINTAINERS | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 18edc86df67..a61d3ae06df 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -230,7 +230,6 @@ docstring relicensing Gerald Pfeifer  
> 
>  docstring relicensingJoseph Myers
> 
>  predict.def  Jan Hubicka 
>  gcov Jan Hubicka 
> -gcov Martin Liska
>  gcov Nathan Sidwell  
>  option handling  Joseph Myers
> 
>  middle-end   Jeff Law
> @@ -268,7 +267,6 @@ check in changes outside of the parts of the compiler 
> they maintain.
>   Reviewers
>  
>  arc port Claudiu Zissulescu  
> -callgraphMartin Liska
>  callgraphMartin Jambor   
>  C front end  Marek Polacek   
>  CTF, BTF David Faust 
> @@ -519,6 +517,7 @@ Kriang Lerdsuwanakij  
> 
>  Renlin Li
>  Xinliang David Li
>  Chen Liqin   
> +Martin Liska 
>  Jiangning Liu
>  Sa Liu   
>  Ralph Loader 
> -- 
> 2.39.1
> 

Re: GNU Tools Cauldron 2022

[Jan Hubicka via Gcc]

Hello,
> Hello,
> 
> We are pleased to invite you all to the next GNU Tools Cauldron,
> taking place in Paris on September 16-18, 2022.  We are looking forward
> to meet you again after three years!
> 
> As for the previous instances, we have setup a wiki page for
> details:
> 
>  https://gcc.gnu.org/wiki/cauldron2022  
> 

The Cauldron is going to start in few minutes.  We also set up youtube
channel https://www.youtube.com/channel/UCCJUY8xtvLa05oODFj6FHcQ where
you can watch us live.

Honza
> 
> In previous years we have been able to make the Cauldron free to all to attend
> thanks to the generosity of our sponsors. However this year we are having to
> charge for attendance. We are still working out what we will need to charge,
> but it will be no more than £200.
> 
> Attendance will remain free for community volunteers and others who do not 
> have
> a commercial backer and we will be providing a small number of travel 
> bursaries
> for students to attend. 
> 
> We will also make it possible to attend remotely in the hybrid form and
> remote attendance will be for free as well.
> 
> Please email to tools-cauldron-admin AT googlegroups.com if you would like to
> attend, indicating your T-shirt size and any dietary requirements. We'll
> contact you later in the year with details of how to pay.
> 
> To send abstracts or ask administrative questions, please email to
> tools-cauldron-admin AT googlegroups.com  .
> 
> The Cauldron is organized by a group of volunteers. We are keen to add some
> more people so others can stand down. If you'd like to be part of that
> organizing committee, please email the same address.
> 
> This announcement is being sent to the main mailing list of the following
> groups: GCC, GDB, binutils, CGEN, DejaGnu, newlib and glibc.
> 
> Please feel free to share with other groups as appropriate.
> 
> Honza

Re: State of AutoFDO in GCC

[Jan Hubicka via Gcc]

> On Tue, Jul 26, 2022 at 4:13 PM Eugene Rozenfeld via Gcc
>  wrote:
> >
> > Hello GCC community.
> >
> > I started this thread on the state of AutoFDO in GCC more than a year ago. 
> > Here is the first message in the thread: 
> > https://gcc.gnu.org/pipermail/gcc/2021-April/235860.html
> >
> > Since then I committed a number of patches to revive AutoFDO in GCC:
> >
> > Fix a typo in an AutoFDO error 
> > string
> > Update gen_autofdo_event.py and 
> > gcc-auto-profile.
> > Fixes for AutoFDO 
> > tests
> > Fix indir-call-prof-2.c with 
> > AutoFDO
> > Fixes for AutoFDO 
> > testing
> > Fix indirect call inlining with 
> > AutoFDO
> > Improve AutoFDO count propagation 
> > algorithm
> > AutoFDO: don't set param_early_inliner_max_iterations to 
> > 10.
> > AutoFDO: Don't try to promote indirect calls that result in recursive 
> > direct 
> > calls
> > Fix profile count maintenance in vectorizer 
> > peeling.
> >
> > I also made a number of fixes and improvements to create_gcov tool in 
> > https://github.com/google/autofdo .
> >
> > AutoFDO in GCC is in a much better shape now.
> >
> > I have a further set of patches that improve DWARF discriminator support in 
> > GCC and enable AutoFDO to use discriminators. It's based on commits in an 
> > old Google vendor branch as described in Andi's mail below
> > but uses a different approach for keeping track of per-instruction 
> > discriminators.
> >
> > I submitted the first (and the biggest) of these patches almost 2 months 
> > ago on June 2: 
> > https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=5af22024f62f1f596a35d3c138d41d47d5697ca0
> > but only got a review from Andi 
> > (https://gcc.gnu.org/pipermail/gcc-patches/2022-June/596549.html) who is 
> > not allowed to approve patches for commit. I pinged gcc-patches twice with 
> > no success.
> >
> > I would appreciate help in getting a review on this patch so that I can get 
> > it committed and submit patches that depend on it.
> 
> Hi, Eugene
> 
> Thanks for your efforts to fix and improve AutoFDO in GCC.  I believe
> that part of the difficulty with obtaining a review of the patches is
> that the original authors have dispersed and no one in the GCC
> community officially is the maintainer for the feature.  Because you
> seem to be one of the primary users and developers, would you be
> interested to take on the responsibility of maintaining the
> AutoFDO-specific portions of the code, with guidance and mentorship
> from other GCC maintainers, especially the ones responsible for gcov
> and PDO?

I missed the patches (it would help to add me to CC :) and will review
the FDO/profile facing parts.  Since it also extends debug info
generation and front-ends I think we also need reviewer for that part.

Having auto-FDO co-maintainer would be welcome.

Honza
> 
> Thanks, David
> 
> >
> > Thank you,
> >
> > Eugene
> >
> > -Original Message-
> > From: Andi Kleen 
> > Sent: Monday, May 10, 2021 10:21 AM
> > To: Joseph Myers 
> > Cc: Jan Hubicka ; gcc ; Eugene Rozenfeld 
> > 
> > Subject: [EXTERNAL] Re: State of AutoFDO in GCC
> >
> > On Mon, May 10, 2021 at 04:55:50PM +, Joseph Myers wrote:
> > > On Mon, 10 May 2021, Andi Kleen via Gcc wrote:
> > >
> > > > It's difficult to find now because it was a branch in the old SVN
> > > > that wasn't converted. Sadly the great git conversion was quite lossy.
> > >
> > > All branches and tags, including deleted ones, were converted (under
> > > not-fetched-by-default refs in some cases); the git repository has
> > > everything that might plausibly be useful, omitting only a few things
> > > that would have been meaningless to convert, such as mistaken branch
> > > creations in the root of the repository and the SVN hooks directory.
> > > Use "git ls-remote git://gcc.gnu.org/git/gcc.git" to see the full list
> > > of over 5000 refs available in the repository (or do a clone with
> > > --mirror to fetch them all).
> >
> > Okay thanks. I don't see them in any of the web interfaces, neither on
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgcc.gnu.org%2Fgit%2Fgitweb.cgi%3Fp%3Dgcc.git&d

Re: GNU Tools Cauldron 2022

[Jan Hubicka via Gcc]

> On Sat, May 14, 2022 at 7:03 PM Jan Hubicka via Gcc  wrote:
> >
> > Hello,
> >
> > We are pleased to invite you all to the next GNU Tools Cauldron,
> > taking place in Paris on September 16-18, 2022.  We are looking forward
> > to meet you again after three years!
> 
> Did you intend to announce the location as Prague, Czechia, not Paris, France?
Thanks! I was checking it several times for similar mistakes :(
So here should be corrected announcement before I send it to ohter
mailing lists.

Hello,

We are pleased to invite you all to the next GNU Tools Cauldron,
taking place in Prague on September 16-18, 2022.  We are looking forward
to meet you again after three years!

As for the previous instances, we have setup a wiki page for
details:

 https://gcc.gnu.org/wiki/cauldron2022  
<https://gcc.gnu.org/wiki/cauldron2022>

In previous years we have been able to make the Cauldron free to all to attend
thanks to the generosity of our sponsors. However this year we are having to
charge for attendance. We are still working out what we will need to charge,
but it will be no more than £200.

Attendance will remain free for community volunteers and others who do not have
a commercial backer and we will be providing a small number of travel bursaries
for students to attend. 

We will also make it possible to attend remotely in the hybrid form and
remote attendance will be for free as well.

Please email to tools-cauldron-admin AT googlegroups.com if you would like to
attend, indicating your T-shirt size and any dietary requirements. We'll
contact you later in the year with details of how to pay.

To send abstracts or ask administrative questions, please email to
tools-cauldron-admin AT googlegroups.com  <http://googlegroups.com>.

The Cauldron is organized by a group of volunteers. We are keen to add some
more people so others can stand down. If you'd like to be part of that
organizing committee, please email the same address.

This announcement is being sent to the main mailing list of the following
groups: GCC, GDB, binutils, CGEN, DejaGnu, newlib and glibc.

Please feel free to share with other groups as appropriate.

Honza

GNU Tools Cauldron 2022

[Jan Hubicka via Gcc]

Hello,

We are pleased to invite you all to the next GNU Tools Cauldron,
taking place in Paris on September 16-18, 2022.  We are looking forward
to meet you again after three years!

As for the previous instances, we have setup a wiki page for
details:

 https://gcc.gnu.org/wiki/cauldron2022  


In previous years we have been able to make the Cauldron free to all to attend
thanks to the generosity of our sponsors. However this year we are having to
charge for attendance. We are still working out what we will need to charge,
but it will be no more than £200.

Attendance will remain free for community volunteers and others who do not have
a commercial backer and we will be providing a small number of travel bursaries
for students to attend. 

We will also make it possible to attend remotely in the hybrid form and
remote attendance will be for free as well.

Please email to tools-cauldron-admin AT googlegroups.com if you would like to
attend, indicating your T-shirt size and any dietary requirements. We'll
contact you later in the year with details of how to pay.

To send abstracts or ask administrative questions, please email to
tools-cauldron-admin AT googlegroups.com  .

The Cauldron is organized by a group of volunteers. We are keen to add some
more people so others can stand down. If you'd like to be part of that
organizing committee, please email the same address.

This announcement is being sent to the main mailing list of the following
groups: GCC, GDB, binutils, CGEN, DejaGnu, newlib and glibc.

Please feel free to share with other groups as appropriate.

Honza

Re: [GSoC]Bypass assembler when generating LTO object files

[Jan Hubicka via Gcc]

Hi,
> 
> 
> > On 08-Apr-2022, at 6:32 PM, Jan Hubicka  wrote:
> > 
> > Ankur,
> >> I was browsing the list of submitted GSoC projects this year and the
> >> project regarding bypassing assembler when generating LTO object files
> >> caught my eye.
> > I apologize for late reply.  I would be very happy to mentor this
> > project.
> 
> Thanks for the reply, but unfortunately, due to some reasons, I would not
> be able to take part in GSoC this year.  
> But the project seems interesting and would be amazing opportunity to
> learn a lot more things for me, so would it be okay if I try to give it a
> go outside GSoC if no-one else picks it as their GSoC project this year ?

I would be still very happy to help with that! However it would be also
pity to not take part of GSoC, so if there is something I can help with
on that let me know.
> 
> >> 
> >> I already have a gcc built from source (sync-ed with trunk/master) and
> >> launched the test-suite on it.
> >> 
> >> I am currently in process of understanding the primilary patch
> >> (https://gcc.gnu.org/legacy-ml/gcc/2014-09/msg00340.html), and
> >> experimenting with it.
> >> 
> >> are there any other things I should be aware of (useful Doc/blog or a
> >> bug tracking the project) before proceeding further ?
> > 
> > I think it is pretty much all that exists.  Basically we will need to
> > implement everything that is necessary to stream out valid object file
> > directly from GCC rather than going through gas.  The experimental
> > prototype sort of worked but it was lacking few things.
> 
> When I try to apply that patch on my local branch ( branched from trunk ),
> it seem to be incompatible with the current working tree. Is there a
> specific branch that I have to apply it to ? or is it due to the recent
> file rename patch ( changing extensions from .c to .cc ) ? 
> 
> ```
> $ git apply --check bypass_asm_patch
> 
> error: patch failed: Makefile.in:1300
> error: Makefile.in: patch does not apply
> error: common.opt: No such file or directory
> error: langhooks.c: No such file or directory
> error: lto/Make-lang.in: No such file or directory
> error: lto/lto-object.c: No such file or directory
> error: lto/lto.c: No such file or directory
> error: lto/lto.h: No such file or directory
> error: lto-streamer.h: No such file or directory
> error: toplev.c: No such file or directory
> ```

I can try to update the patch, or it probably should apply to trunk
checked out around the date I sent the patch.  Indeed we need to change
c to cc but there are likely more changes since then - most importnatly
the early debug info.
At I will see how easy/hard is to make the patch build with current
trunk.
Honza
> 
> Thanks 
> - Ankur 
> 

Re: [GSoC]Bypass assembler when generating LTO object files

[Jan Hubicka via Gcc]

Ankur,
> I was browsing the list of submitted GSoC projects this year and the
> project regarding bypassing assembler when generating LTO object files
> caught my eye.
I apologize for late reply.  I would be very happy to mentor this
project.
> 
> I already have a gcc built from source (sync-ed with trunk/master) and
> launched the test-suite on it.
> 
> I am currently in process of understanding the primilary patch
> (https://gcc.gnu.org/legacy-ml/gcc/2014-09/msg00340.html), and
> experimenting with it.
> 
> are there any other things I should be aware of (useful Doc/blog or a
> bug tracking the project) before proceeding further ?

I think it is pretty much all that exists.  Basically we will need to
implement everything that is necessary to stream out valid object file
directly from GCC rather than going through gas.  The experimental
prototype sort of worked but it was lacking few things.  First is the
production of proper object file hearder (it encodes things like
architeture ELF is produced for), production of symbol table that is
necessary to mark the LTO object file and also we now need a way to
stream debug info (DWARF) directly to the object from dwarf2out.

So I think first step would be to produce object files w/o debug info
which can be consumed by unmodified linkers and then look into DWARF
bytecode streaming.

Honza
> 
> I am Ankur Saini, a B.Tech CSE 3rd year student at USICT, GGSIPU india
> and a former GSoC contributor at gcc ( worked on expanding gcc static
> analyzer's C++ support in GSoC 2021
> [https://gist.github.com/Arsenic-ATG/8f4ac194f460dd9b2c78cf51af39afef])
> 
> Thanks
> - Ankur

Re: Question on ipa_ref->referring and ipa_ref->stmt on all_late_ipa_passes

[Jan Hubicka via Gcc]

> Hi,
> 
> I would like to use ipa_ref in the PASS_LIST all_late_ipa_passes to query
> the statement (ref->stmt) of where a global variable is used. However, I am
> having some problems achieving this.
> 
> What I do is:
> 
> 1. Check that ipa_ref->referring has a body and is not inlined.
> 2. get_body
> 3. try to print out the gimple statement using print_gimple_stmt
> (dump_file, ref->stmt, 0, TDF_NONE).
> 
> This all seems correct to me, but I have been receiving errors that print
> is trying to print a tree with an incorrect TREE_CODE. I am assuming here
> that ref->stmt is not updated after all_regular_ipa_passes, much like how
> when looking at cgraph_edge the call statement is also not updated. Can
> someone please tell me if this is indeed the case or what is happening here?

Yes, while body materialization we keep cgraph edges up to date but we
do not keep references.  We probably should remove them earlier.
Keeping them up to date would need relatively some work. We do so for
calls since they hold important information (i.e. where to redirect the
call). For references we don't have such machinery in place (even though
I was thinking implementing it). Main difficulty is that inlining also
performs statement folding that moves referneces around statements quite
freely
> 
> Also, while I think that the gimple statements might not be maintained, I
> see that ipa_ref is still used in the ipa_pta pass during
> all_late_ipa_passes. I see that ipa_ref->referring and ipa_ref->stmt are
> not used. Instead the tree of the referred is obtained in the following
> way: ref->referred->decl. I am assuming that it would be possible to use
> ref->referred->decl and search for this tree everywhere in referring to
> find the uses. Can someone confirm this?

I will check what ipa-pta uses here.  I suppose it works since you still
have all references from the pre-IPA-transform stage, so it is
consistent...

Honza
> 
> Thanks!
> -Erick

Re: Question about builtin_free doesn't read memory

[Jan Hubicka via Gcc]

> Hi,
> In function ref_maybe_used_by_call_p_1, there is below code snippet
>  /* The following builtins do not read from memory.  */
>  case BUILT_IN_FREE:
>  ...
>return false;
> 
> I am confused because free function does read from (and even write to)
> memory pointed to by passed argument?

Free is a black box and makes the memory pointed to disappear without
actually worrying what values it holds. We rely on fact that we do not
see free imlementation and does not worry about the details of its
implementation (whcih probably has sort of linked list before address
ptr points to)
> I am thinking DSE optimizations like:
>   *ptr = value;
>   free(ptr);
>   *ptr = undef;
> Does GCC take advantage of UB to eliminate the first store to ptr if
> free is considered not reading memory?

The aim here is to optimize out *ptr = value.

Honza
> 
> Thanks,
> bin

Re: distinguishing gcc compilation valgrind false positives

[Jan Hubicka via Gcc]

> 
> I can confirm that zero-initializing node_is_self_scc prevents the 
> uninitialised use warnings in incorporate_penalties (ipa-cp.c:3282)
Great, I will commit the patch.  But I also wonder if there are any
remaining unitialized warnings in ipa code?

Honza
> 
> Thanks,
> Zdenek
> 
> 

Re: distinguishing gcc compilation valgrind false positives

[Jan Hubicka via Gcc]

> >
> > diff --git a/gcc/ipa-prop.h b/gcc/ipa-prop.h
> > index 42842d9466a..1d0c115465c 100644
> > --- a/gcc/ipa-prop.h
> > +++ b/gcc/ipa-prop.h
> > @@ -623,8 +623,8 @@ ipa_node_params::ipa_node_params ()
> >  : descriptors (NULL), lattices (NULL), ipcp_orig_node (NULL),
> >known_csts (vNULL), known_contexts (vNULL), analysis_done (0),
> >node_enqueued (0), do_clone_for_all_contexts (0), is_all_contexts_clone 
> > (0),
> > -  node_dead (0), node_within_scc (0), node_calling_single_call (0),
> > -  versionable (0)
> > +  node_dead (0), node_within_scc (0), node_is_self_scc (0),
> > +  node_calling_single_call (0), versionable (0)
> >  {
> >  }
> 
> Oops, can you please commit the change to master and all active
> branches?
OK.  To be honest I am not 100% sure if the flag is not always
initialized later.  

Zdenek, can you, please, check if the undefined warning in ipcp goes
away with this patch and if so, post what are the remaining ones?
> 
> Thanks,
> 
> Martin
> 

Re: distinguishing gcc compilation valgrind false positives

[Jan Hubicka via Gcc]

> ==5404== Conditional jump or move depends on uninitialised value(s)
> ==5404==    at 0x25DAAD7: incorporate_penalties (ipa-cp.c:3282)
> ==5404==    by 0x25DAAD7: good_cloning_opportunity_p(cgraph_node*, sreal,
> sreal, profile_count, int) (ipa-cp.c:3340)

I looked at this one (since it is in code I am familiar with) and I
think it may be real bug.  There are flags node_is_self_scc which does
not seem to be initialized in the constructor.

diff --git a/gcc/ipa-prop.h b/gcc/ipa-prop.h
index 42842d9466a..1d0c115465c 100644
--- a/gcc/ipa-prop.h
+++ b/gcc/ipa-prop.h
@@ -623,8 +623,8 @@ ipa_node_params::ipa_node_params ()
 : descriptors (NULL), lattices (NULL), ipcp_orig_node (NULL),
   known_csts (vNULL), known_contexts (vNULL), analysis_done (0),
   node_enqueued (0), do_clone_for_all_contexts (0), is_all_contexts_clone (0),
-  node_dead (0), node_within_scc (0), node_calling_single_call (0),
-  versionable (0)
+  node_dead (0), node_within_scc (0), node_is_self_scc (0),
+  node_calling_single_call (0), versionable (0)
 {
 }
 
Honza

Re: Question on calling possible_polymorphic_call_targets, side effects, and virtual tables.

[Jan Hubicka via Gcc]

> Hello,
> 
> I have a SIMPLE_IPA_PASS that parses the program multiple times. As it
> parses gimple, it builds a data structure with the information
> collected that will provide some invariants to future iterations over
> the program.
> 
> I was looking into adding a new feature that would take advantage of
> devirtualization by calling possible_polymorphic_call_targets. All
> looks good, a large program that I use for verifying changes seems to
> compile nicely. However, as I generate a test file in which I discard
> most optimizations (except the one I am currently working on) I
> realize that an assertion on my pass is triggered.
> 
> I dig in and it looks like I am ignoring error_mark_nodes in varpools'
> DECL_INITIAL. The first passes essentially encode the information that
> these are error_mark_nodes. On the pass in which I call
> possible_polymorphic_call_targets I find that suddenly, these
> error_mark_nodes are gone and replaced with a virtual table, thus
> triggering the assertion.
> 
> Is there a way I can get rid of the error_mark_nodes from the earlier
> passes to match the changes brought by
> possible_polymorphic_call_targets? I suppose that finding
> polymorphic_call_targets at an earlier stage is a possibility, but I
> was wondering exactly which function/statement is able to fix these
> error_mark_nodes so that I can also learn about this.

Like function bodies the initializers are read on demand.
If you call get_constructor (varpool_node) you will trigger the lazy
stream in.

Honza
> 
> Thanks!