On Fri, 30 Apr 2021, David Faust wrote: > > On 4/30/21 9:11 AM, Jose E. Marchesi via Gcc-patches wrote: > > > >>> For a moment, for the sake of this question, if we establish that CTF/BTF > >>> generation always feeds off DWARF DIEs (so there is no need to access > >>> type/decl tree nodes), what will it take to keep LTO support while keeping > >>> ctf_debug_finalize in dwarf2out_finish ? > >> > >> I don't think it's possible without major surgery. Now, it looks like > >> you want to emit .ctf at the LTO compile stage and BTF at the LTRANS > >> stage. Note you do not have access to the CTF data produced at the > >> LTRANS stage. What DWARF has access to is pointers (symbol name + offset) > >> to the early DWARF generated DIEs for GCCs tree decls from what it > >> adds DW_AT_abstract_origin references and then amending those entities > >> with furthr DWARF attributes. > >> > >> Now, the "major surgery" could for example entail streaming the CTF > >> data you produce from compile through WPA to the LTRANS stage. Or > >> alternatively simply reading it from the compile time produced objects > >> by means of the DWARF DIE references. I don't know what kind of (if any) > >> references you need to produce from BTF to the CTF data (I suppose they > >> might share indexes into a same symbol table...). > > > > I am thinking, given that we have both "early DWARF" and "late DWARF", > > we may want to have three init/fini hooks for debug formats instead of > > two: > > > > DEBUG_FORMAT_initialize (called in dwarf2out_early_finish) > > > > [... traverse early DWARF (DEBUG_FORMAT_do_die) ...] > > > > DEBUG_FORMAT_early_finish (called in dwarf2out_early_finish) > > DEBUG_FORMAT_finish (called in dwarf2out_finish) > > > > Note that CTF and BTF share the ctfc internal data structure (which is > > built from traversing the early DWARF) but these are otherwise distinct > > debug formats: CTF can be completely emitted in ctf_early_finish, > > whereas BTF needs additional information that is only available in > > ctf_finish.) > > > > I like this idea. > > As Indu mentioned, the change to writing CTF/BTF in dwarf2out_finish > rather than dwar2out_early_finish is motivated purely by the needs of BPF. > > Specifically, it is to support BPF Compile Once - Run Everywhere (BPF > CO-RE). The BPF backend needs BTF information to generate CO-RE > 'relocation' records, and it needs to add to the BTF string table. The > relocation records themselves are placed in a .BTF.ext section, but > refer to BTF records by ID. > > Note that CO-RE only comes into play when generating BTF information for > a BPF target. Otherwise, the BTF information can also be emitted in > early_finish. That is, BTF is effectively "late DWARF" ONLY for the BPF > target. Of course, this is also the usual use case for BTF. > > I do not have much experience with LTO. I am still trying to digest the > LTO section of the manual. But from what what I do understand, and what > is discussed here, I think the following may work: > > 1. Always emit CTF in early_finish. There is no reason to delay it. This > simplifies both LTO and non-LTO cases. > > 2. For non-BPF targets, emit BTF in early_finish as well. > > 3. For non-LTO + BTF + BPF, emit BTF in in (late) finish, to support > generating CO-RE information in the BPF backend. > > 4. For LTO + BTF + BPF, either emit BTF in early_finish then read it > back later, or stream it through WPA to LTRANS stage. I am not sure > exactly what this streaming means. In either case, end up with enough > information at LTRANS stage to re-emit the BTF, now with CO-RE information. > > Does the above make any sense? Is this overly complicated?
So DWARF would likely emit BTF early and amend it late with the CO-RE information. Whatever exactly CO-RE is - I'm assuming it's backend stuff like DWARF location expressions. Note when emitting CTF/BTF early there's still the issue of fat objects, but as said, eventually the "early" CTF/BTF can be literally shared. Of course the "fat" part would have the CO-RE stuff emitted at compile-time and the "lto" part at link time. I suppose we might want to introduce a target hook that says whether a target supports fat LTO objects or not, side-stepping this issue. Richard. > Thank you very much Richard for the review, and for raising this LTO > discussion. > > David > -- Richard Biener <rguent...@suse.de> SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany; GF: Felix Imendörffer; HRB 36809 (AG Nuernberg)
