Review Request: icemon - A GUI monitor for Icecream

[Jan Kratochvil]

Hi,

Review Request: icemon - A GUI monitor for Icecream
https://bugzilla.redhat.com/show_bug.cgi?id=1965529
Spec URL: http://people.redhat.com/jkratoch/icemon.spec
SRPM URL: http://people.redhat.com/jkratoch/icemon-3.3-6.fc34.src.rpm

It should be very simple IMO, it is an unretirement.


Thanks,
Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Mon, 28 Sep 2020 17:58:58 +0200, Jakub Jelinek wrote:
> On Mon, Sep 28, 2020 at 05:46:08PM +0200, Jan Kratochvil wrote:
> > On Mon, 28 Sep 2020 17:35:26 +0200, Jakub Jelinek wrote:
> > > A way out of this could be either to use comdat .debug_info etc. sections
> > > (but that would result in quite large increase of *.o file sizes), or let
> > > the linker or a tool like DWZ discard or simplify such DIEs.
> > > I don't see how could you see at compile time that the linker will not
> > > choose the particular copy.
> > 
> > Another option is to use clang which should have such optimization 
> > implemented
> > soon:
> > https://whova.com/embedded/session/llvm_202010/1193947/
> 
> If you do it on the compiler side, you'll get a lot of those pesky partial
> units you so hate on the lldb side.

No. The LLVM patches from Sony are using COMDAT groups you mentioned above:
https://youtu.be/oSCbzLC46Vg?t=312


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Wed, 07 Oct 2020 09:58:37 +0200, Jan Kratochvil wrote:
> On Wed, 07 Oct 2020 00:46:24 +0200, Jeff Law wrote:
> > On 9/30/20 3:50 AM, Jan Kratochvil wrote:
> > > When we start talking about RHEL (and CentOS) DWZ is completely pointless 
> > > then
> > > as DWZ there saves only 0.28% of *-debuginfo.rpm (20MB of 7.2GB).
> > > Therefore approx. 0.14% of the distribution size.
> > 
> > Umm, we're fighting with PM these days over things in the 10M range. 
> 
> So it is better to slow down getting a finally usable debugger by years to
> save 10MB of distro size? I really do not believe that. :-)

I think you mean 10MB of normal rpm size. That is important for containers and
other small devices. But the 20MB are only *-debuginfo.rpm size, those are
only for developer machines. Developer machines are not concerned by 20MB.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Wed, 07 Oct 2020 00:46:24 +0200, Jeff Law wrote:
> On 9/30/20 3:50 AM, Jan Kratochvil wrote:
> > On Wed, 30 Sep 2020 01:31:29 +0200, Jeff Law wrote:
> >> But the GCC community
> >> doesn't really test that option and it's known to be broken with LTO.
> > I haven't seen any GCC PR for -fdebug-types-section being broken with LTO.
> 
> I'm not aware of one either.  But as Jakub has previously pointed out
> debug-types-section is disabled when LTO is enabled.  I don't know the
> details of why that is done.

Because Jakub made a mistake and he still has not corrected himself.
I have explained it in:

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/2HIK25T3MHP7D6LC5CDJNINP3ZEO7RPW/

If -fdebug-types-section was disabled in LTO mode then there would not be
6.78% vs. 31.23% difference of Fedora mass rebuild of -fdebug-types-section
vs. -fno-debug-types-section.

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/J34DKZK7MWH725CMCS5YLVZ77YURSMRZ/

Also this simple example shows it is not true:
c="static struct C{int a;C(){}} v;";o="-Wall -gdwarf-5 
-fdebug-types-section -flto -O2";echo "$c"|gcc -c -o 1.o $o -x c++ -;echo 
"${c}int main(){}"|gcc -c -o 2.o $o -x c++ -;gcc -o 1 1.o 2.o $o;llvm-dwarfdump 
1|grep DW_UT_type
0x0042: Type Unit: length = 0x005b version = 0x0005 unit_type = 
DW_UT_type abbr_offset = 0x0023 addr_size = 0x08 name = 'C' type_signature = 
0x4e76c0dda193eb61 type_offset = 0x0026 (next unit at 0x00a1)


> >> So the only paths forward I see are to either fix -fdebug-types-section or
> >> improve dwz.
> > And obviously much easier is to fix -fdebug-types-section than DWZ (if there
> > are really any bugs in -fdebug-types-section, there are known bugs nobody
> > wants to fix in DWZ).
> 
> I think you're making an unsubstantiated leap here.  Neither of us know
> what's wrong with GCC LTO and debug-types-section

I know what - nothing! Although sure the problem is GCC as its LTO still
produces -fdebug-types-section at all. clang LTO ignores -fdebug-types-section
as clang already does the DWZ-style class unification itself during lld phase.

This discussion and the state of GCC vs. clang shows me that getting rid of
DWZ is less important and as it is more productive for Fedora to rather get
rid of GCC with DWZ altogether.


> and others are working on dwz.

That is their problem. I am trying to work on things that make sense (but
I cannot).


> > When we start talking about RHEL (and CentOS) DWZ is completely pointless 
> > then
> > as DWZ there saves only 0.28% of *-debuginfo.rpm (20MB of 7.2GB).
> > Therefore approx. 0.14% of the distribution size.
> 
> Umm, we're fighting with PM these days over things in the 10M range. 

So it is better to slow down getting a finally usable debugger by years to
save 10MB of distro size? I really do not believe that. :-)


> Most customers don't use dwz.  But they consume its output for the RPMs
> that we provide.

They cannot because the LLVM tools Red Hat ships still do not support DWZ.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Mon, 05 Oct 2020 09:20:53 +0200, Florian Weimer wrote:
> Fedora Server still defaults to LVM and XFS, as far as I know.  I expect
> that downstream will continue to use XFS as well.
> 
> I don't think you can assume that debugging information will be stored
> on btrfs file systems.

OK. Then DWARF consumers could just decompress zstd themselves. Then we can
argue about tracing tools overhead. Either cache of uncompressed files or even
some little performance hit. Still that would be worth 50% of debuginfo size.
And that would be needed only on non-btrfs filesystems which (maybe) will
disappear in the future.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Mon, 05 Oct 2020 08:28:03 +0200, Florian Weimer wrote:
> Why do you think that?  Using debuginfo for perf and the like seems to
> be much more common than actual debugging, based on what I see
> downstream.

OK, interesting, thanks for the info. Still that does not change anything with
btrfs zstd. btrfs zstd would save on-disk size ~50% (I did not measure zstd
for Fedora distro, zlib saves on Fedora distro 52%) compared to DWZ's on-disk
size saving of 20% (or just 10% if one makes some non-DWZ optimizations in
addition to -fdebug-types-section).


> > The problem is that you have to wait for minutes for GDB to print anything.
> 
> Is this about slow tab completion?

No. GDB always converts DWARF->IR (=expands) for whole CUs (Compilation Units).
Moreover for some constructs GDB requires complete type definition which makes
CUs dependent on other CUs. So in practice accessing one variable will expand
around 50 CUs. And each CU is nowadays huge for real C++ templatized programs.

In other cases (such as in lambda functions) GDB even expands completely all
CUs. That is probably some GDB bug. But it makes GDB to eat 20+ GB of memory
and several minutes of runtime on fast machines to print a variable.

Compared to that LLDB expands only one DIE and it is done. But LLDB needs to
know where the DIE is. This is why LLDB needs .debug_names index and not
.gdb_index which does not contain DIE offsets. And this is why .gdb_index is
easy to update for DWZed DWARF but for .debug_names there is currently even no
DWARF specification how to make it compatible with DWZ. DWZ .debug_names index
has to express two CUs for each DIE:
(1) DW_TAG_partial_unit where the DIE is located in the DWARF file
(and whether it is in DWZ common file or not)
(2) DW_TAG_compile_unit which did DW_TAG_imported_unit that
DW_TAG_partial_unit)
LLDB DWZ support had to extend its in-memory index this way but it does waste
LLDB memory+runtime on all OSes not using DWZ (=most of the LLDB use - for
Android/iOS/OSX).


> > It is faster to add cout<<, recompile and rerun the program (with clang+lld 
> > as
> > with g++ it takes more than 3x as much time) than to wait for GDB. LLDB 
> > would
> > sure print it immediately but it is incompatible with Fedora DWARF. Enjoy.
> 
> I can't use LLDB because it does not support thread-local variables.

I have this item in my LLDB TODO list (and I even fixed the TLS support in GDB
in the past). But without DWARF (DWZ) support it makes no sense to support TLS.
So I rather returned to working on the DWZ support saving 1.8% of distribution
size wasting 2 years on it and probably going to continue wasting more
forthcoming years on those 1.8% of distribution size.


> Not even initial-exec variables, which could be implemented without
> peeking at glibc internals.

Yes, a lot of useful things could be working if we did not have to waste time
on useless stuff.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Tue, 29 Sep 2020 22:29:44 +0200, Mark Wielaard wrote:
> I was just discussing that recently with the Hotspot Perf GUI
> maintainer. And we concluded that if .debug files would be compressed
> then we would need an uncompressed cache somewhere. The issue with
> having the on-disk debuginfo files compressed is that for
> debugger/tracing/profiling tools it incurs an significant
> decompression time delay and extra memory usage. Especially for a
> profiling tool that only needs to quickly get a little information it
> is much more convenient to be able to simply mmap the .debug file,
> check the aranges and directly jump to the correct DIE offset. See
> e.g. https://github.com/KDAB/hotspot/issues/115

First is is a marginal use case. For the GDB popular here I tested zlib on
some IIRC 500MB+ .debug file and the startup time was 11.00s->12.45s
= +13.20%. Given GDB takes minutes to print something on such .debug files the
<2s larger startup does not matter.

And then all this is about zlib compression. Facebook has developed zstd which
is much faster. Google says faster than reading the .debug files, on my
machine both zstd and NVMe disk read are both 2GB/s. I expect btrfs has even
in-memory cache for decompressed files but I have not checked it now as all
the numbers I have collected have no effect here anyway.


> Because I believe it is the most important benchmark. The on-disk size
> is not just the installed file size, but it is also the in memory size
> of the data structures that need to be stored and parsed. So a 20%
> smaller on-disk size also (roughly) means 20% less DIEs and abbrevs
> that need to be parsed or held into memory.

The problem is that you have to wait for minutes for GDB to print anything.
It is faster to add cout<<, recompile and rerun the program (with clang+lld as
with g++ it takes more than 3x as much time) than to wait for GDB. LLDB would
sure print it immediately but it is incompatible with Fedora DWARF. Enjoy.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Mon, 28 Sep 2020 16:51:02 +0200, Jan Kratochvil wrote:
> To make DWZ better consumable it needs to have the partial units separately
> parseable. That way they can be shared at IR level and not just at DWARF level
> That means:
>  * DW_TAG_partial_unit should have DW_AT_language.
>  * DW_TAG_partial_unit must contain only types (struct/class).
>Currently they contain for example also static constant variables but when
>you parse such independent DW_TAG_partial_unit into which dictionary you
>will register such variable? That makes no sense.
> Currently DWZ has benefits only with DWZ common file. Without DWZ common files
> DWZ produces 1.6% files bigger than -fdebug-types-section. Therefore if the
> DWZ common files saving of 3.3% per Fedora distribution size is worth it then
> the existing DWZ tool should be dropped anyway as one can use normal
> -fdebug-types-section and one can just write a simple tool moving DW_UT_type
> units to the DWZ common file and converting DW_AT_signature declarations to
> DW_FORM_ref_sup4 and we are done.

I have measured this "a different common file" for DW_UT_type and it brings
18.14% size reduction of what DWZ does. With additional dropping of dead DIEs
+ dropping -fdebug-types-section type declaratins which achieves about 28%
this makes it together 46% of what DWZ reduces possible without those
overcomplicated constructs of DWZ.

Original comparison of plain -fdebug-types-section DWZ makes the Fedora
distribution 3.3% smaller.

-fdebug-types-section with some simple optimizations (just reusing existng
-fdebug-types-section code in consumers + DWZ common file opening in consumers
together with removal of dead DIEs) DWZ is still slightly smaller but already
only by 1.8% of the Fedora distribution size.

Sure my proposal does not expect that few percents matter. I have checked
further possibilities based on a the mail replies here which seem to insist on
any debuginfo size savings.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Switching to DWARF5 default for GCC11 (and the default Fedora 34 toolchain)

[Jan Kratochvil]

On Fri, 02 Oct 2020 18:11:45 +0200, Jan Kratochvil wrote:
> On Wed, 30 Sep 2020 15:58:51 +0200, Stephen John Smoogen wrote:
> > On Wed, 30 Sep 2020 at 09:41, Jan Kratochvil  
> > wrote:
> > > On Wed, 30 Sep 2020 14:50:39 +0200, Mark Wielaard wrote:
> > > > = What I am NOT working on
> > > [...]
> > > > - Any other tool, project not mentioned above or other
> > > >   native toolchains like golang, rust, clang/llvm or ocaml.
> > > >   I expect those to simply keep producing DWARF4.
> > >
> > > So because of a DWZ deficiency you want to keep DWARF-5 in clang disabled.
> > > Despite clang supports DWARF-5 better and for a longer time than GCC.
> >
> > I did not take it to mean that. I took it to mean that he isn't going to
> > tell other groups what to work on which a change request seems to have
> > become. He instead expects them to keep doing what they are doing if they
> > want versus getting forced to do what he is working on.
> 
> Currently on files built by clang -gdwarf-5 DWZ will fail:
>   dwz: ./usr/lib64/libmatrix_client.so.0.3.1-0.3.1-2.fc34.x86_64.debug: 
> Unknown debugging section .debug_addr
> 
> Which is fine as the file just does not get optimized. But that results in rpm
> size bigger for clang-built binaries by 31.23% as -fdebug-types-section is not
> used. If -fdebug-types-section was used for clang-built binaries DWZ would
> fail a similar way but the size increase would be "just" by 6.78%.

I had a wrong idea. For clang -flto the option -fdebug-types-section no longer
makes sense as clang produces direct DW_TAG_class references (DW_FORM_ref4).

So DWZ cannot optimize clang output better, DWZ makes sense only for GCC, not
clang.

(A bit, DWZ can still find slightly more optimal DW_FORM_* variants than
clang, I will check if it cannot be fixed in clang.)


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Wed, 30 Sep 2020 23:42:56 +0200, Michel Alexandre Salim wrote:
> On Mon, 2020-09-28 at 16:50 +0200, Jan Kratochvil wrote:
> > For example during Fedora Package Review Process do some packages get
> > rejected because they would make the distribution too large? Not worth of
> > including such new package? I am not aware of such decision and it even
> > sounds funny to me. But that is what you choose here by enforcing DWZ no
> > matter how little savings it has.
> 
> I'm not aware of this ever happening. What has happened is packages
> being dropped from the installation images because it makes them too
> large -- is that perhaps what you're referring to?

No because installation images never contain debuginfos.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Switching to DWARF5 default for GCC11 (and the default Fedora 34 toolchain)

[Jan Kratochvil]

On Wed, 30 Sep 2020 15:58:51 +0200, Stephen John Smoogen wrote:
> On Wed, 30 Sep 2020 at 09:41, Jan Kratochvil  
> wrote:
> > On Wed, 30 Sep 2020 14:50:39 +0200, Mark Wielaard wrote:
> > > = What I am NOT working on
> > [...]
> > > - Any other tool, project not mentioned above or other
> > >   native toolchains like golang, rust, clang/llvm or ocaml.
> > >   I expect those to simply keep producing DWARF4.
> >
> > So because of a DWZ deficiency you want to keep DWARF-5 in clang disabled.
> > Despite clang supports DWARF-5 better and for a longer time than GCC.
>
> I did not take it to mean that. I took it to mean that he isn't going to
> tell other groups what to work on which a change request seems to have
> become. He instead expects them to keep doing what they are doing if they
> want versus getting forced to do what he is working on.

Currently on files built by clang -gdwarf-5 DWZ will fail:
dwz: ./usr/lib64/libmatrix_client.so.0.3.1-0.3.1-2.fc34.x86_64.debug: 
Unknown debugging section .debug_addr

Which is fine as the file just does not get optimized. But that results in rpm
size bigger for clang-built binaries by 31.23% as -fdebug-types-section is not
used. If -fdebug-types-section was used for clang-built binaries DWZ would
fail a similar way but the size increase would be "just" by 6.78%.

I do not find there much a difference, just stating.

(These percents are relative to total *-debuginfo.rpm size, not to total
distribution size.)


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Switching to DWARF5 default for GCC11 (and the default Fedora 34 toolchain)

[Jan Kratochvil]

On Wed, 30 Sep 2020 14:50:39 +0200, Mark Wielaard wrote:
> = What I am NOT working on
[...]
> - Any other tool, project not mentioned above or other
>   native toolchains like golang, rust, clang/llvm or ocaml.
>   I expect those to simply keep producing DWARF4.

So because of a DWZ deficiency you want to keep DWARF-5 in clang disabled.
Despite clang supports DWARF-5 better and for a longer time than GCC.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Tue, 29 Sep 2020 22:31:28 +0200, Mark Wielaard wrote:
> Note that you are using -ffunction-sections together with -flto.
> With -flto you don't need -ffunction-sections.
> 
> -ffunction sections might cause functions to be dropped by the linker
> without updating the DWARF DIEs, causing things like a zero
> DW_AT_low_pc.
> 
> Just using -flto should not cause such issues.

Thanks for this investigation. You are right in gcc -flto binaries I cannot
find these dead DIEs.

Found C++ with LTO in libabigail, libreoffice, powertop. Surprisingly gcc has
LTO turned off.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Tue, 29 Sep 2020 22:35:14 +0200, Mark Wielaard wrote:
> On Mon, Sep 28, 2020 at 04:50:59PM +0200, Jan Kratochvil wrote:
> >  * DW_TAG_partial_unit should have DW_AT_language.
> >  * DW_TAG_partial_unit must contain only types (struct/class).
> >Currently they contain for example also static constant variables but 
> > when
> >you parse such independent DW_TAG_partial_unit into which dictionary you
> >will register such variable? That makes no sense.
> 
> You might want to look at the experiments to do something like that
> from Tom de Vries:
> https://sourceware.org/pipermail/dwz/2020q1/000579.html

This is another extension of the DWZ tool and tags. The goal is to make the
file format (and tooling) more fast and simple, not more slow and complex.


> https://sourceware.org/pipermail/dwz/2020q1/000568.html

This looks as the DWZ DW_AT_language problem #1 fix listed above.


> Hacking on dwz and supporting partial units and DWARF supplemential
> files in debugger like tools isn't trivial. But it is IMHO also not
> such a big effort that we have to drop everything else.

So why is Fedora stuck for 3.5 years on DWARF-4? I would switch Fedora to
DWARF-5 long time ago but I could not as there isn't anyone willing to work on
DWZ. Even now you want to port it to DWARF-5 but nothing more. It needs also
to
 * fix bugs
 * support LLVM DWARF-5
 * support .debug_names
 * support -fdebug-types-section (for reduction of size already during linking)
 * with all the effort it is a pity it gives up on large debuginfos as it
   would run out of memory

DWZ could be a nice tool (not really critical but an interesting challenge)
but it is not important enough to make people find time working on it.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Wed, 30 Sep 2020 01:31:29 +0200, Jeff Law wrote:
> -fdebug-types-section a supported option in the sense that it's in the
> compiler and we'll fix bugs in it when we can.  But the GCC community
> doesn't really test that option and it's known to be broken with LTO.

I believe you base this information on Jakub Jelinek's internal company mail:
Message-ID: <20200710092926.GJ2363@tucnak>

IIUC that mail contains incorrect information.
My apologies if my deduction is incorrect, I am also writing "IIUC" here.
I am basing my information on explanation by GCC developer Richard Biener:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88878#c8

It is explained there "in_lto_p" means GCC is in second/later phase of LTO.
Not that LTO is enabled at all (as Jakub Jelinek said in the internal mail).
Also GCC does not produce ICEs (=compiler crash, (*)) Jakub Jelinek was
claiming will happen during the 2+ packages rebuild with LTO and
-fdebug-types-section I have done.


So I really see no indication why GCC would not normally support
-fdebug-types-section even with LTO. Also it is so simple optimization of
DWARF there is no reason why there should be any longterm issues with it.


Jan

(*) There were 7 packages reproducing GCC crashes due to the following two GCC
Bugs specific to -fdebug-types-section.
That is unrelated to the topic of the "in_lto_p" condition discussed above.
ICE: fortran+gnat: build_abbrev_table, at dwarf2out.c: -g 
-fdebug-types-section
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96471
ICE: c++: dwarf2out_abstract_function, at dwarf2out.c: -g 
-fdebug-types-section
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96472
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Fri, 25 Sep 2020 16:29:26 +0200, Jan Kratochvil wrote:
> On Fri, 25 Sep 2020 12:10:22 +0200, Zbigniew Jędrzejewski-Szmek wrote:
> > I would love to see a comparison of numbers for three things:
> > - raw debuginfo without dwz or -fdebug-types-section
> 
> Oops, I do not have this number, I can run new massrebuild, it takes about
> 4 days (depending on availability of beefy machines).

After almost 5 days it finished:
 rpm size:  45643460133
disk size: 233466873738

= neither compression rpm size is bigger by 31.23%.
= neither compression on-disk size is bigger by 29.72%.

It is based on 22126 Fedora Rawhide packages (20810 rebuilt) on 2020-09-30.

That has been arithmetically approximated from:
New mass rebuild's no DWZ && no -fdebug-types-section:
 rpm size:  59324710852
disk size: 299606472067
New mass rebuild's default DWZ:
 rpm size:  45732816106
disk size: 228314986029


> > - debuginfo with dwz (current approach)
> 
>  rpm size:  35186079102
> disk size: 177913332940
> 
> > - debuginfo with -fdebug-types-section
> 
>  rpm size:  37570327765
> disk size: 214927514757
> 
>  = DWZ rpm size is smaller by 6.78%
>  = DWZ on-disk size is smaller by 20.8%
> 
> It is based on 22080 Fedora Rawhide packages rebuilt on 2020-08-24.
  It is based on 22080 Fedora Rawhide packages (20637 rebuilt) on 2020-08-24.



Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Wed, 30 Sep 2020 01:31:29 +0200, Jeff Law wrote:
> But the GCC community
> doesn't really test that option and it's known to be broken with LTO.

I haven't seen any GCC PR for -fdebug-types-section being broken with LTO.

During one abigail diff I did not see any difference. I plan to run a full
distribution abigail massrebuild+check as stated in the Change to exclude any
possible incompatibilities. That would discover unfiled GCC problems with
-fdebug-types-section.

Also I do not see why fixing -fdebug-types-section should be anyhow difficult
if the compiler can produce -fno-debug-types-section. I can also write
postprocessor to get -fdebug-types-section if GCC is unable to do that.
That would sure lose the -fdebug-types-section compilation time performance
benefits.


> And, not surprisingly, our team has had significant input on the options
> we're using *and* the GCC implementation of those options.   We make
> recommendations based on our experiences. That same experience would lead us
> to recommending against -fdebug-types-section at this time.

I think I have also some DWARF experience. Could you suggest what is wrong on
-fdebug-types-section?


> It would certainly be good to improve the on-disk distro size.

OK, going to file a Change to enable gcc -gz option (zlib section compression)
as that will reduce on-disk *.debug size by 52.84%! Then we can disable both
DWZ and -fdebug-types-section as those become pointless then.


> So the only paths forward I see are to either fix -fdebug-types-section or
> improve dwz.

And obviously much easier is to fix -fdebug-types-section than DWZ (if there
are really any bugs in -fdebug-types-section, there are known bugs nobody
wants to fix in DWZ).


> Putting my Red Hat hat on, I get pushback from PM on *any* size increases in
> the RHEL space.

When we start talking about RHEL (and CentOS) DWZ is completely pointless then
as DWZ there saves only 0.28% of *-debuginfo.rpm (20MB of 7.2GB).
Therefore approx. 0.14% of the distribution size.


> As much as we'd like to be in a world were a 1% increase in distro
> size doesn't matter, that's not the actual world we live in.

Unfortunately DWZ cannot decrease RHEL size by that 1%.


> And our RHEL customers absolutey do care about the size of debuginfo
> becuause it affects link times.

System debuginfo format does not affect link times. Using DWZ during linking
customer's applications definitely only increases linking time as it is an
extra step. Not sure what do you talk about.

Production of debuginfo does affect compilation time but that is unrelated to
DWZ. Production of debug info affects linking time only if -gsplit-dwarf is
not used. (-gsplit-dwarf still affects linking time but only very little.)
But that is all unrelated to DWZ.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info LLDB Index (System-Wide change)

[Jan Kratochvil]

On Wed, 30 Sep 2020 10:44:06 +0200, Jakub Jelinek wrote:
> On Wed, Sep 30, 2020 at 10:33:59AM +0200, Jan Kratochvil wrote:
> > Because doing it separately like GDB does is a wrong thing for
> > edit-compile-debug cycle. When clang (lld for LTO) has all the data incl. IR
> > already in memory it can much more faster produce the index for it.
> 
> clang definitely doesn't have all the data already in memory, it has just a
> single translation unit in memory at a time.
> So, such a .debug_names is completely useless, because then you don't have
> one index, but hundreds if not thousands of them that all needs to be
> queried for the same information.

I said "lld for LTO". In Fedora clang has already enabled LTO and then the
.debug_names is produces during lld linking as one unified index:

  String: 0x0030 "C"
  Entry @ 0xad {
Tag: DW_TAG_class_type
DW_IDX_compile_unit: 0x00
DW_IDX_die_offset: 0x0026
  }
  String: 0x04ec "main"
  Entry @ 0xeb {
Tag: DW_TAG_subprogram
DW_IDX_compile_unit: 0x01
DW_IDX_die_offset: 0x002c
  }

It is very mean to spread negative lies about toolchains you have no idea
about and you do not even spend a few seconds to check it yourself.


> That is why the index should be added by linkers or post-link tools.

This is how slow GNU Toolchain does that. LLVM has learned from those
mistakes.


> And, if LLDB can cope only with clang generated .debug_names and can't deal
> with .debug_names generated by other tools, something is seriously wrong.

LLDB can deal with .debug_names generated by other tools but then LLDB will be
affected by bugs in the other tools. In comparison do you want GDB debugging
GCC-built programs to be affected by bugs in LLVM?


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info LLDB Index (System-Wide change)

[Jan Kratochvil]

On Wed, 30 Sep 2020 02:00:52 +0200, Michel Alexandre Salim wrote:
> On Tue, 2020-09-29 at 16:29 -0400, Ben Cotton wrote:
> > https://fedoraproject.org/wiki/Changes/DebugInfoLldbIndex
> > Currently the change will affect only packages using:
> >  %global toolchain clang
> > Those are currently only these packages being built by clang and
> > using
> > this %toolchain framework: dotnet3.1 libcxxabi mtxclient nheko simde
> > wine
> > 
> > FIXME: Which other Fedora packages are being built by clang?
> > 
> Should this be mentioned as part of some packaging guidelines?
> Especially as we expect more packages to use clang in the future.

As long as the .spec file uses
  %global toolchain clang
and the normal configuration variables %__cc %__cxx it all should work out of
the box. Unfortunately for example for libcxxabi it does not work out of the
box as overrides $CXXFLAGS somehow, I have to debug it.

This change itself does not have big impact as currently it affects only the
package dotnet3.1. I have filed it as it is related/dependent on:
https://fedoraproject.org/wiki/Changes/DebugInfoStandardization
And as more packages are expected to use clang + these flags.


> > This Change is dependent on RHEL-5 F-34 feature expected to be filed
> > by Mark Wielaard.
> RHEL-9, not 5?

DWARF-5. :-)  Thanks for catching the typo.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info LLDB Index (System-Wide change)

[Jan Kratochvil]

On Wed, 30 Sep 2020 10:14:55 +0200, Jakub Jelinek wrote:
> it would be great if the
> .debug_names some tool generates (whether it is GDB, some standalone
> post-linking (and post dwz) tool, dwz itself, ...) is usable by both GDB and
> LLDB, because the point in DWARF5 standardizing .debug_names was to make
> those usable by everything rather than each tool rolling its own index.
> We can't have two .debug_names sections, one for GDB and one for LLDB...

This is discussed in the Change.

https://fedoraproject.org/wiki/Changes/DebugInfoLldbIndex#Detailed_Description

You miss in your description that clang-built binaries must use clang-built
.debug_names.

Also AFAIK nobody plans to implement .debug_names support into DWZ (which also
shows how DWZ is unmaintained) so DWZ must be excluded from builds using
clang.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info LLDB Index (System-Wide change)

[Jan Kratochvil]

On Wed, 30 Sep 2020 02:11:34 +0200, Neal Gompa wrote:
> Why don't you add an lldb-add-index tool to generate LLVM indexes for
> LLDB?

Because doing it separately like GDB does is a wrong thing for
edit-compile-debug cycle. When clang (lld for LTO) has all the data incl. IR
already in memory it can much more faster produce the index for it.

With gdb-add-index it is questionable whether to use it or not for
edit-compile-debug cycles. If you run debugger just once for the compiled
program it is slower to generate + use the index just once than to run the
debugger without any index.

The index production performance does not matter much for rpmbuild as that
takes a long time anyway. But keeping the end-user edit-compile-debug cycle
unified with rpm build process makes the code path more tested and more
simple.


> Then we just invoke it as part of the buildroot policy setup and
> get both GDB and LLDB indexes? This proposal seems to be particularly
> destructive to GDB users to favor LLDB.

I do not see how this affects GDB. This Change expects clang will produce
.debug_names and rpmbuild will call gdb-add-index which will add .gdb_index.
Then there will be both indexes for both debuggers.

That is not great longterm but in some time GDB should fix its .debug_names
consumer to be compatible with the DWARF-5 standard and then Fedora can drop
.gdb_index completely.

(TBH I am not completely sure that LLDB conforms to the DWARF-5 standard as
the standard is not too descriptive of the format. But LLDB is at least more
.debug_names standard compliant despite "more compliant" may sound funny.)


> I personally don't use LLDB for *anything*, since I think it's not a
> particularly good debugger right now,

Please do not be offensive to the other toolchain. I also must not be so it is
unfair. I could argue with numbers but I won't.


> performance of debugging clang-built things with GDB degraded.

This is not intended/expected.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Mon, 28 Sep 2020 17:58:58 +0200, Jakub Jelinek wrote:
> On Mon, Sep 28, 2020 at 05:46:08PM +0200, Jan Kratochvil wrote:
> > https://whova.com/embedded/session/llvm_202010/1193947/
> 
> If you do it on the compiler side, you'll get a lot of those pesky partial
> units you so hate on the lldb side.

There are many ways how clang could implement it. I have no idea how is the
draft implemented but unless you have more information I do not think they
would use DW_TAG_partial_unit.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Mon, 28 Sep 2020 17:35:26 +0200, Jakub Jelinek wrote:
> So, was this compiled by GCC or clang?

Fedora Koji package:
lldb-debuginfo-11.0.0-0.2.rc3.fc34.x86_64

GNU GIMPLE 10.2.1 20200916 (Red Hat 10.2.1-4) -m64 -mtune=generic -march=x86-64 
-g -g -g -O2 -O2 -O2 -O2 -fno-openmp -fno-openacc -fcf-protection=none 
-ffat-lto-objects -fexceptions -fstack-protector-strong 
-fasynchronous-unwind-tables -fstack-clash-protection -fPIC -ffunction-sections 
-fdata-sections -fltrans -fplugin=annobin


> A way out of this could be either to use comdat .debug_info etc. sections
> (but that would result in quite large increase of *.o file sizes), or let
> the linker or a tool like DWZ discard or simplify such DIEs.
> I don't see how could you see at compile time that the linker will not
> choose the particular copy.

Another option is to use clang which should have such optimization implemented
soon:
https://whova.com/embedded/session/llvm_202010/1193947/


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Mon, 28 Sep 2020 12:31:59 +0200, Mark Wielaard wrote:
> I do find your statistics per package useful because they show dwz is
> in general effective by producing at least 20% (more) on-disk size
> reduction,

I am ignoring the on-disk size, I always measure just *-debuginfo.rpm size.

If anyone is concerned about on-disk size then Fedora should have already
enabled zlib section compression which would reduce the on-disk *.debug size
by 52.84% for the whole distro. Or even implement zstd section compression
probably with even bigger size decrease (and even lower performance hit
already low enough).

But Fedora has not enabled transparent zstd filesystem compression for F-33
btrfs by default and nobody enabled zlib for *.debug files (you even
implemented decompressing zlib from *.debug files) so apparently nobody / you
do not care about the on-disk size.

So why do you mention the on-disk size now?


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Mon, 28 Sep 2020 12:42:33 +0200, Jakub Jelinek wrote:
> On Mon, Sep 28, 2020 at 12:31:59PM +0200, Mark Wielaard wrote:
> > Finally I am interested in your proposal to implement a different way
> > to reduce the size of DIE trees by eliminating "unused" DIEs. It is
> > hard to predict what effect that would have without seeing an
> > implementation (in theory GCC with LTO would not actually generate
> > debuginfo for unused functions). But I think that can be done separate
> > from your proposal and combined with other size reduction techniques.
> 
> And note that GCC already does implement
> -feliminate-unused-debug-{symbols,types} which are enabled by default and
> are (at least in my eyes) sometimes too aggressive, so by eliminating even 
> further
> DIEs the debug experience might be even worse.

git clone git://git.jankratochvil.net/massrebuild
./dwarfredundant 
lldb-debuginfo-11.0.0-0.2.rc3.fc34.x86_64/usr/lib/debug/usr/lib64/liblldb.so.11.0.0-11.0.0-0.2.rc3.fc34.x86_64.debug

For example:

saved=27:
0x0193b058: DW_TAG_subprogram [47] *
  DW_AT_abstract_origin [DW_FORM_ref_addr]  
(0x0515d936 "_ZN12lldb_private17StoppointLocationD2Ev")
  DW_AT_low_pc [DW_FORM_addr]   (0x)
  DW_AT_high_pc [DW_FORM_udata] (1)
  DW_AT_frame_base [DW_FORM_exprloc](DW_OP_call_frame_cfa)
  DW_AT_GNU_all_call_sites [DW_FORM_flag_present]   (true)
  DW_AT_sibling [DW_FORM_ref_udata] (cu + 0x1da4f => 
{0x0193b073})
0x0193b06b:   DW_TAG_formal_parameter [55]
DW_AT_abstract_origin [DW_FORM_ref_addr]
(0x0515d941)
DW_AT_location [DW_FORM_exprloc](DW_OP_reg5 RDI)
0x0193b072:   NULL

This DIE describes only a concrete function instance at address 0x0.
No function can exist on address 0x0 on x86_64, that is
a discarded/deduplicated function:
[Dwarf-Discuss] DWARF for linker GC'd code

http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/2020-July/004683.html

I do not see what any DWARF consumer could find out from such a DIE.

And there are many such DIEs, something a bit less than 28% of what DWZ saves
(28% is incl. removal of DW_UT_type declarations).


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Mon, 28 Sep 2020 12:31:59 +0200, Mark Wielaard wrote:
> If you want to make -fdebug-types-sections the default you really
> should work with the upstream GCC developers to figure out why they
> don't want that.

I haven't seen that, according to Richard Biener from GCC
-fdebug-types-section is a normally supported GCC feature:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88878#c6
I am aware only of Jakub Jelinek who is against -fdebug-types-section.
I should also state he is the author of DWZ.

If GCC is unable to support such a trivial feature as -fdebug-types-section
then Fedora should really already switch to the standard compiler. It will
come sooner or later anyway. This deviation from standard tools just causes
continuous troubles such as:
[fesco] Issue #2020: Firefox is switching from gcc to clang/llvm
https://pagure.io/fesco/issue/2020#comment-671672


> Trying to override upstream defaults in Fedora without understanding why
> upstream decided on the current defaults isn't a good idea IMHO.

You know very well Fedora already overrides upstream GCC defaults a lot:
-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 
-Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong 
-grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic 
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection

And DWZ is even a project unrelated to GCC so calling for upstream defaults
would really call for dropping DWZ.


> I totally get that it is frustrating if you worked for a long time on a
> new feature to support some DWARF constructs for lldb

It is in no way a feature as it does not bring any user visible improvement.
It is only a compatibility with marginally (0.1%) used file format with
disputable benefits.


> and your aren't able to get the patches in shape to be accepted upstream.

That is a repeated lie I have never even suggested. LLDB is fine accepting my
DWZ patches. I understand you are used to difficulty of upstreaming patches in
GNU projects but that is not the case of LLVM. In fact LLDB is a completely
different world in accepting patches than GDB where it was taking me even 10+
years to get some patches accepted. For GDB you need to learn first how to do
the ancient ineffective and bug-prone coding practices, force yourself to
really execute them to become a global maintainer and only then you manage to
get patches checked in.

You are repeatedly trying to make it look as the problem is upstreaming DWZ
support. That is not any problem. The problem is the DWZ itself as it just
isn't worth the effort of supporting it in all the consumers.

As I am repeating again and again I just find DWZ too complicated for both
production and consumption for so little gain (size reduction) it achieves.
So before I complicate the LLDB codebase by the DWZ support and make it
a catch-up game for Red Hat, Fedora and others forever (as Apple+Google is
never going to use DWZ and they know why) I am trying by this Change to save
a lot of time for everyone.

The years of engineering time I have already spent on DWZ and the years of
engineering time I will have to spend on its future maintenance and reworks
(for clang DWARF) could be better spent on improving the debugging experience.
We are no longer living in 80es where few saved bytes of size were critical
whether the debugger will be able to run or not. Apparently GNU developers
still haven't realized that change.


> But I am
> happy people now know about your patches and seem to find them useful.

"useful" means with my patches they can workaround the Fedora problem of
encumbering its debuginfo by DWZ.

And this question is not about the existing patches. That waste of engineering
time has been already done. It is about the future waste of time maintaining
compatibility with the DWZ format almost nobody (0.1%) cares about.


> You say it is difficult to support DWARF partial units as generated by
> dwz in lldb, but dwz doesn't really do anything non-standard (and GCC
> with LTO also generates partial units).

"standard" means that the DWZ specification has been accepted into DWARF-5
standard. IMO that was a mistake. It may have happened because the DWZ author
is a member of the DWARF standard committee. Apparently nobody has so far
implemented any reasonable/effective consumer for DWARF-5 DWZ otherwise they
would put some restrictions into the standard.

To make DWZ better consumable it needs to have the partial units separately
parseable. That way they can be shared at IR level and not just at DWARF level
That means:
 * DW_TAG_partial_unit should have DW_AT_language.
 * DW_TAG_partial_unit must contain only types (struct/class).
   Currently they contain for example also static constant variables but when
   you parse such independent DW_TAG_partial_unit into which dictionary you
   will register such variable? That makes 

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Mon, 28 Sep 2020 14:08:48 +0200, Mark Wielaard wrote:
> It is certainly a clever setup and makes sense if your build bottleneck
> is sending files around between different machines. But I don't think
> this is the generic Fedora packager or developer use case.

I agree and I do not propose anywhere -gsplit-dwarf. That is offtopic to this
mail thread and it may look as it is related to my -fdebug-types-section
proposal.


It would make sense possibly only for Chromium which has no debuginfo in
Fedora currently at all. And the missing debuginfo is due to DWZ because DWZ
does not support -fdebug-types-section, nobody is supporting DWARF64 and
Chromium .debug_info section without -fdebug-types-section would be larger
than 4GB, therefore technically impossible with DWARF32.

I am saying -gsplit-dwarf is probably the best solution despite right now
-fdebug-types-section is the best (smallest possible file) solution.
As with -gdwarf-5 -fdebug-types-section its .debug_info section is very close
to 4GB and it will exceed in some time anyway. Then only -gsplit-dwarf will be
possible with DWARF32 && DWARF-5.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Fri, 25 Sep 2020 16:29:26 +0200, Jan Kratochvil wrote:
> On Fri, 25 Sep 2020 12:10:22 +0200, Zbigniew Jędrzejewski-Szmek wrote:
> > - debuginfo with dwz (current approach)
> 
>  rpm size:  35186079102
> disk size: 177913332940
> 
> > - debuginfo with -fdebug-types-section
> 
>  rpm size:  37570327765
> disk size: 214927514757
> 
>  = DWZ rpm size is smaller by 6.78%
>  = DWZ on-disk size is smaller by 20.8%
...
> Also there is an opportunity for new non-DWZ optimization (orthogonal to
> DWZ/-fdebug-types-section) which can save 5.96% of *-debuginfo.rpm with
 ^^^
 = on-disk files
> clang-only draft implementation which requires no DWARF consumers modification
> and it is easier to implement than to upstream+maintain the DWZ support for
> LLDB.

Here I made a mistake. One needs to compare 5.96% to DWZ saving of 20.8%
(and not 6.78%). Therefore this "non-DWZ optimization" is not equivalent do
DWZ (but it could still save some space). This also only an estimate.

I have updated the wiki page (it does not change much on the proposal).


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Fri, 25 Sep 2020 17:40:50 +0200, Florian Weimer wrote:
> The numbers are very difficult understand because it's not clear what
> you are measuring.  Especially since as far I understand it, parts are
> not yet fully implemented, so we can't know yet if all the required data
> is there.

TL;DR nobody cares about 3% of distribution size (for Fedora; for CentOS 0%).

If really someone does then I can just post-process the DWARF files to drop
dead DIEs and the size will be the same as with DWZ without any need of DWZ.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Fri, 25 Sep 2020 17:09:54 +0200, Robbie Harwood wrote:
> So saying "Google does it" (or similar) is *not* a good argument.

So let's stick only to the numbers I sent in other mails.
In fact I do not understand why we talk about anything except the numbers.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Fri, 25 Sep 2020 12:10:22 +0200, Zbigniew Jędrzejewski-Szmek wrote:
> I'm missing some good statistics.

I have 1.6TB of statistics, ask me anything. It is calculated by my scripts:
https://git.jankratochvil.net/?p=massrebuild.git;a=tree
git clone git://git.jankratochvil.net/massrebuild


> > * DWZ advantage: On the whole Fedora distro it saves 3.3% (5GB of the
> > 157GB distribution size)
> 
> What is this comparing? Is this the size of binary rpm or the
> installation-on-disk footprint?

I am usually talking about *-debuginfo.rpm size.

Another possible number is separate *.debug files download (DWZ is then 6%
bigger than -fdebug-types-section due to the associated DWZ common files).


> I would love to see a comparison of numbers for three things:
> - raw debuginfo without dwz or -fdebug-types-section

Oops, I do not have this number, I can run new massrebuild, it takes about
4 days (depending on availability of beefy machines).


> - debuginfo with dwz (current approach)

 rpm size:  35186079102
disk size: 177913332940

> - debuginfo with -fdebug-types-section

 rpm size:  37570327765
disk size: 214927514757

 = DWZ rpm size is smaller by 6.78%
 = DWZ on-disk size is smaller by 20.8%

It is based on 22080 Fedora Rawhide packages rebuilt on 2020-08-24.


> For each of those three categories both measures (rpm size and on-disk size)
> would be useful.

Another big variable is F-34 should be hopefully in DWARF-5 (F-33 is DWARF-4)
which will change the numbers a bit (unaware which way). Currently DWZ is not
yet ported to DWARF-5 so there is no way to compare it. Also DWZ does not plan
to support LLVM DWARF-5 so that will also skew such comparison even after its
port.

For on-disk size it will all get different by F-33 btrfs compression again
which should reduce the size by about 50% (which makes any
DWZ/-fdebug-types-section differences pointless). It will obviously make the
on-disk size difference smaller (than current 20.8%).

And finally on-disk size depends a lot on which *-debuginfo packages you have
installed which varies a lot when stddev is twice the average DWZ saving.


> Could you provide numbers like this for some subset of packages
> (20-30 packages that produce debuginfo would be enough to get a good measure).

Problem of these numbers is they depend too much on the chosen set of rpms
so 20-30 packages do not say anything.
DWZ against -fdebug-types-section saves for whole Rawhide 6.35% size total.
When averaged for each package it is 5.44% (that means DWZ saves more on
bigger-than-median packages) but stddev of the saving is +/-11%.
Packages where -fdebug-types-section is smallest (by difference in bytes):
70.11: julia-1.5.0-1.fc33.src.rpm -fdebug-types-section size=866936043 
DWZ size=1236511762
74.43: nodejs-14.7.0-1.fc33.src.rpm -fdebug-types-section 
size=921485027 DWZ size=1238008099
77.84: mozjs78-78.1.0-1.fc33.src.rpm -fdebug-types-section 
size=623280098 DWZ size=800743010
Packages where DWZ is smallest (by difference in bytes):
508.93: kea-1.7.9-3.fc33.src.rpm -fdebug-types-section size=1379013840 
DWZ size=270963319
143.07: paraview-5.8.1-1.fc33.src.rpm -fdebug-types-section 
size=11462175974 DWZ size=8011695061
196.49: hpx-1.4.1-4.fc33.src.rpm -fdebug-types-section size=10981369919 
DWZ size=5588742102
All these sizes are for *-debuginfo.rpm.

The sizes depend strongly on the chosen subset of packages:
For example for ELN-like (*) distro the saving is not 6.35% but only 0.28%.
For Fedora 32 packages on my personal machines it is not 6.35% but 0.72%.

(*) I did use Fedora Rawhide subset for packages present in CentOS-8.2.

Also there is an opportunity for new non-DWZ optimization (orthogonal to
DWZ/-fdebug-types-section) which can save 5.96% of *-debuginfo.rpm with
clang-only draft implementation which requires no DWARF consumers modification
and it is easier to implement than to upstream+maintain the DWZ support for
LLDB.


> I find that 3.3% number strange — it would mean that dwz is
> essentially useless, but maybe I'm misunderstanding how it's defined.

F-32 x86_64 has 157GB total, debug/ is 82GB (6GB is *-debugsource):
6.35% * (82-6) / 157 = 3.07%
approx., the 3.3% was calculated with more exact distro size numbers.


> I think we need to get some better understanding what the effects of
> various approaches are before discussing which to pick.

Thanks for this discussion.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Fri, 25 Sep 2020 13:43:40 +0200, Neal Gompa wrote:
> On Fri, Sep 25, 2020 at 7:26 AM Ryan  wrote:
> >
> > > == Benefit to Fedora ==
> > > * Better compatibility with existing debugging and tracing tools,
> > > primarily [https://lldb.llvm.org/ LLDB].
> >
> > Thanks for your work on this Ben and Jan, Just as an interested user, use
> > of the DWZ format significantly limits Swift development on Fedora, as it
> > is impossible to debug with LLDB when using system libraries.

Currently you can use my off-trunk patchset build:
dnf copr enable jankratochvil/lldb;dnf install 
lldb-experimental;lldb-experimental


> But that's fixable since there's a patchset to make LLDB understand
> dwz, which was not submitted upstream for unstated reasons.

The reasons are stated in other mails, re-stating here:
 * the support of DWZ is complicated for effective DWARF consumers like LLDB
 * the same DWZ size reduction can be achieved just by removing dead DIEs
   and -fdebug-types-section skeletons - not needing any DWZ nor DWZ support
   in consumers at all
 * the LLDB DWZ support is now scatterd across the LLDB DWARF codebase for
   every DIE type so it will require continuous maintenance during LLDB
   development from RH; Google+Apple will probably never use DWZ themselves.
 * the LLDB DWZ support is currently implemented in LLDB DWARF/ but LLDB
   is going to replace its DWARF code to clang's DWARF code which will then
   require rewriting the LLDB DWZ code from scratch


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Fri, 25 Sep 2020 13:56:47 +0200, Jakub Jelinek wrote:
> Jan had probably problems getting his DWZ support upstreamed into LLDB

It isn't completely easy, I have already upstreamed a lot of preparatory work
for the DWZ patchset which is good for LLDB in general.

Just before upstreaming the final DWZ-specific part I am trying to provide
a better solution for Fedora as the numbers I have benchmarked do not speak
for the overcomplicated DWZ solution.


> and decided to spend his time instead non-constructively trying to kill DWZ.

Simplifying Fedora toolchains is more constructive than supporting complicated
format with no benefits.


> -fdebug-types-section in GCC is pretty much unmaintained, almost nobody is
> really using it,

According to Richard Biener from GCC -fdebug-types-section is a normally
supported GCC feature:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88878#c6


> and the debug types design is quite flawed and shouldn't
> have been added into DWARF.

So why is Google using it for everything? I have this opinion about DWZ.


> What DWZ implements is what DWARF3 and onwards have been documenting as
> DWARF compression technique, before DWARF5 for the multi-file it has been
> using an extension but that is now standardized in DWARF5.

It is specified by the standard but without .debug_names support which is
essential for effective DWARF consumers. Also not everything that got accepted
into the DWARF standard is necessarily a good thing. Besides that DWZ could
have better size reduction if:
 * it did not contain so many bugs it fails on many files
 * it did not give up on larger debuginfos due to running out of memory:
   it could implement slower alternative algorithms with on-disk temporary
   files


> Mark is actively working on DWARF5 support for dwz right now,

Mark Wielaard does not plan to support DWARF-5 features being used by LLVM.


> DWZ primarily just removes redundancies, when
> the DIEs are the same, and doesn't really matter what kind of DIEs it is,
> while -fdebug-types-section is about types only, and does significantly
> change what the debug info contains and how it can be referenced.

DWZ's only possible benefit is the DWZ common file. Without DWZ common file
for reasons listed above DWZ produces 1.6% bigger debuginfo than
-fdebug-types-section while requiring 8x more complicated consumers.

And for example for downloading of separate *.debug files DWZ is even bigger
than -fdebug-types-section exactly because of the DWZ common files which are
a double-edged sword.

Besides that it is all about few percents of size nobody cares about.


> DWZ in theory could handle what
> -fdebug-types-section produces and undo the significant size overhead it
> adds due to the large references (basically could undo the type units and
> turn them back into normal .debug_info).

Those "large references" are already considered in the numbers I provided.
If one removes the "large references" together with dead DIES (address zero)
one has the same size decrease as DWZ without any DWZ needed.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Fri, 25 Sep 2020 07:08:27 +0200, Dominique Martinet wrote:
> Jan Kratochvil wrote on Thu, Sep 24, 2020:
> > Copy-pasted it at the bottom of this mail. I do not know the talk but TL;DR
> > existing DWARF contains some dead DIEs - unused/deduplicated functions and 
> > also
> > -fdebug-types-section declarations/skeletons which can be removed or 
> > converted
> > to direct DIE references respectively. That way one could reduce the size 
> > like
> > DWZ does but without needing any new complicated support in DWARF consumers.
> 
> Ok, avoiding duplicate data makes sense there is quite a lot in there.

Duplicate data is what DWZ and -fdebug-types-section is about.

This other optimization - only draft-implemented for clang so far - removes
completely dead/unused/unusable information. That is debuginfo for functions
which were originally compiled but later removed as either unused or duplicate
with other existing functions. But their debuginfo cannot be easily removed so
it is only disabled by setting its debug info address to zero. Some approx.
calculation of possible size saving removing those debug info entries I wrote
in:

https://git.jankratochvil.net/?p=massrebuild.git;a=blob_plain;f=dwarfredundant
I have calculated the saving on Fedora Rawhide *-debuginfo.rpm as 5.96%.
That is together with removing -fdebug-types-section skeletons. It is only
approximate as it does not calculate savings on removed abbreviations, OTOH it
does not calculate absolute DIE references as replacements. One can still
expect the real saving would be slightly bigger (better).

I do not think it matters much (does it?) but if someone wants to advocate DWZ
due to its size savings then this new optimization:
 * it saves the same amount of data as DWZ
 * it needs absolutely no new support in DWARF consumers
 * it has no overhead to separate *.debug files download;
   DWZ has 6.35% size of *-debuginfo.rpm overhead due to the DWZ common files.

Sure one could apply both this dead-DIE removal AND DWZ together which would
get some approx. 6%+7%=13% of *-debuginfo.rpm reduction.


> I don't have very much there but it does work well:
> # compsize  /usr/lib/debug/
> Processed 720 files, 2232 regular extents (2239 refs), 1 inline.
> Type   Perc Disk Usage   Uncompressed Referenced  
> TOTAL   32%   74M 229M 230M   
> none   100%  644K 644K 644K   
> zstd32%   73M 229M 229M   

Nice, thanks for checking it.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Fri, 25 Sep 2020 11:36:47 +0200, Mark Wielaard wrote:
> On Thu, Sep 24, 2020 at 08:34:48PM +0200, Jan Kratochvil wrote:
> > error: Allocatable section after non-allocatable ones
> >   https://sourceware.org/bugzilla/show_bug.cgi?id=24251#c10
> 
> That isn't a longterm unfixed bug.

It is a long-term incompatible behavior, one can argue whether it is a golang
bug or DWZ bug, opinions differ.


> > error: Couldn't find DIE referenced by DW_AT_abstract_origin
> > (maybe DWZ error or maybe compiler error: Unknown DWARF DW_OP_255)
> 
> We cannot fix bugs if you don't report them. Please file bugs for the
> above issue (including how to reproduce them) and we'll take a look.

Why should I report them all? I am trying to fix Fedora by removing DWZ.

If you want to improve DWZ you have accessible Fedora build.logs for all the
years DWZ is failing there. You did nothing with it. Moreover I have even
in-company provided to you in recent months access to all the build logs on
a single machine so that you can check it even more easily. Again you did
nothing with it.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Fri, 25 Sep 2020 01:35:43 +0200, Mark Wielaard wrote:
> Replying since I am mentioned by name in this proposal and it seems to
> argue for removing a feature I am currently working on to make sure it
> works correctly with GCC11 if it switches to producing DWARF5 by
> default.

The problem is you are not working on DWARF-5 features produced by LLVM so
even your planned DWZ is still not usable for LLVM-built binaries.

In the case DWZ is used for GCC-built binaries what to do with LLVM-built
binaries?

(1) Build them without DWZ nor -fdebug-types-section.
They will get up to 2x as big.
(2) Build them with -fdebug-types-section.
Then the distro tools need to be -fdebug-types-section compatible anyway
(which they already are modulo testing).
(3) You were proposing to build them as DWARF-4.
That regresses functionality of Fedora compared to other OSes.
It is also unfair to restrict LLVM due to a deficiency of DWZ.


> On Thu, Sep 24, 2020 at 11:59:44AM -0400, Ben Cotton wrote:
> As others pointed out dwz is widely used. It is used by almost every
> distro in some form and even freedesktop.org flatpaks use dwz for
> their debuginfo.

As Debian really uses DWZ (as pointed out by Florian Weimer) I have updated
the Change text now.


> The upstream project is actively maintained.

Fedora has been waiting for DWARF-5 for 3.5 years due to DWZ. So I do not find
it as maintained. But I have removed the sentence if there is a disagreement
on it.


> Even though there are just 3 committers (including me) the project is still
> seeing ~2.5 commits a week (about 130 in a year).

Still existing bugs are not fixed for years as nobody looks at build.log's.


> > There exist several methods
> > how to make the *-debuginfo.rpms at least a bit smaller. Fedora 18
> > started using DWZ tool (from [[Features/DwarfCompressor]]) while
> > [https://gcc.gnu.org/pipermail/gcc-patches/2008-August/246281.html
> > Google implemented] the same goal in a different way called
> > -fdebug-types-section.
> 
> Note that these methods are not in conflict. Both started out as GNU
> extensions but have been standardized since.

They are in conflict as DWZ still does not support -fdebug-types-section
(after 12 years of -fdebug-types-section and 8 years of DWZ).

In binaries built with -fdebug-types-section DWZ cannot use -m DWZ common file
and in such case DWZ produces 1.6% bigger Fedora distro *-debuginfo.rpm than
simple -fdebug-types-section. That means running DWZ after
-fdebug-types-section is pointless.


> > so its support is missing in tools like
> > [https://lldb.llvm.org/ LLDB],
> 
> But you have been maintaining an out of tree patch for several years
> to support partial units and supplemental files (both of which dwz
> produces and are now standard DWARF). It would be good if you
> upstreamed those patches.

That's the question why I submitted this Change. From my point of view the DWZ
technology makes no sense and so I find a better fix to drop DWZ.
I am interesting in opinion of Fedora / FESCo.


> > or binutils readelf.
> 
> binutils readelf is used as the main tool in the dwz testsuite. It
> certainly supports both partial units and supplemental files. Also
> note the the -wK (=follow-links) option [it isn't on by default, maybe
> it should] that resolves alt links.

I was not aware of it (it is from 2017-11-15). I have updated the Change text.
Yes, it definitely should be on by default.


> I don't know of any other tool which doesn't support either partial
> units or supplemental files, since both are (now) standard DWARF.

GCC also does not support most of DWARF-5 after it is standardized for 3.5
years (only these days you started implementing better DWARF-5 support).

And the DWARF-5 standard even still does not specify .debug_names for DWZ,
that has been apparently forgotten.


> > * DWZ disadvantage: DWZ requires 8x times more complicated (LoC count)
> > support in consumers than -fdebug-types-section.
> 
> I have worked on support for both in various consumers and didn't find
> one method more difficult to support than the other. Maybe debug-types
> was actually more difficult. Because the representation changed from
> separate section in DWARF4, to intermingled with other .debug_info
> units in DWARF5. Making supporting objects that contained mixed
> versions a bit of a pain.

It is easy for primitive tools like GDB which do DWARF->IR (Internal
Representation) conversion for whole CUs (Compilation Units = per .o file) and
not per DIE (one element like a variable/type/function). That was fine in 80s
but not now with big C++ template libraries. That means on real C++ code GDB
consumes about 20GB of memory and 5 minutes of runtime to print a variable.
That does not happen much for Fedora packages as most of the packages are not
in C++ and the C++ ones are not much heavily templatized (there are
exceptions). LLDB makes many indexes of the DIEs to be effective and with DWZ
all of those need to be modified to

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Fri, 25 Sep 2020 11:01:53 +0200, Neal Gompa wrote:
> On Fri, Sep 25, 2020 at 3:08 AM Florian Weimer  wrote:
> > This is not a -dbgsym package, so it probably has been created by a
> > different procedure.  I do not know how Ubuntu distributes their -dbgsym
> > packages.  An example from Debian with .dwz paths is here:
> >
> > 
> >
> 
> Ubuntu *definitely* has it. Checking "alsa-utils" from Ubuntu 20.04
> shows dwz data.
> 
> Cf. 
> http://ddebs.ubuntu.com/pool/main/a/alsa-utils/alsa-utils-dbgsym_1.2.2-1ubuntu1_amd64.ddeb

Thanks for the info, I have already updated the Change text now.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Thu, 24 Sep 2020 20:04:22 +0200, Stephen John Smoogen wrote:
> The original language of the proposal said no other distribution used DWZ,
> and that the format was not adopted and should be removed.

I have already updated the Wiki in the meantime based on new information from
this list:

https://fedoraproject.org/wiki/Changes/DebugInfoStandardization#Detailed_Description
--
Only some of Linux distributions use existing Fedora DWZ (known are
Fedora/CentOS/RHEL, SuSE OSes, OpenMandriva, maybe others?).
[...]
Debian and Ubuntu use neither -fdebug-types-section nor DWZ.
--


> The tool is not easily maintained,

I did not mention it there but it is true there are some longterm unfixed
bugs in DWZ so that it gives up on optimization of many builds:
error: Allocatable section after non-allocatable ones
  https://sourceware.org/bugzilla/show_bug.cgi?id=24251#c10
error: Couldn't find DIE referenced by DW_AT_abstract_origin
(maybe DWZ error or maybe compiler error: Unknown DWARF DW_OP_255)


Thanks,
Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Thu, 24 Sep 2020 20:10:45 +0200, Neal Gompa wrote:
> I do not feel that this is a valid premise either, since the reason
> for no dwz support in LLDB is because nobody contributed it.
> I'm slightly surprised that Red Hat's debuginfo engineers hadn't already
> contributed support for it into LLDB.

I have implemented the support for DWZ into LLDB, it is just off-trunk now:
https://people.redhat.com/jkratoch/dwz-2020-05-31/

https://copr.fedorainfracloud.org/coprs/jankratochvil/lldb/package/lldb-experimental/
dnf copr enable jankratochvil/lldb;dnf install 
lldb-experimental;lldb-experimental

The problem is that the support is complicated as it has to affect the whole
LLDB DWARF codebase (handling of each DIE type). That is the line based on my
patchset, it is exactly calculated:

https://fedoraproject.org/wiki/Changes/DebugInfoStandardization#Detailed_Description
DWZ disadvantage: DWZ requires 8x times more complicated (LoC count) 
support in consumers than -fdebug-types-section.

And all this support is needed despite almost nobody from LLDB/LLVM users
(Android, iOS, OSX, still AFAIK Ubuntu/Debian) uses DWZ.

So when DWZ brings almost no size benefits compared to much easier to support,
faster to compile and already better supported by existing consumers (*) why
not to switch to -fdebug-types-section?

(*) such as LLDB and LLVM binutils but DWZ strings are not decoded even for
example with binutils readelf


> I wonder if the reason for that was the mistaken impression that dwz wasn't
> broadly used.

From LLDB point of view users of DWZ-built Linux distros are counted in
millions, users of Android/iOS/OSX are counted in billions. :-)


Thanks for reply,
Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Thu, 24 Sep 2020 19:16:32 +0200, Neal Gompa wrote:
> Then that certainly means that Ubuntu uses this too, since they reuse
> the dbgsym subpackage generation for the ddeb system they have now.

I am not much familiar with Debian/Ubuntu but I cannot find any use of DWZ
there:
https://packages.ubuntu.com/groovy/amd64/bluez-dbg/download
llvm-dwarfdump -color=0 
bluez-dbg_5.55-0ubuntu1_amd64/data/usr/lib/debug/.build-id/*/*.debug|grep 
DW_TAG_partial_unit

This debuginfo package has been built 2020-09-15.

(Besides that this proposal is not based on whether Debian uses DWZ or not.)


Thanks,
Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)

[Jan Kratochvil]

On Thu, 24 Sep 2020 19:01:17 +0200, Neal Gompa wrote:
> This is not true. Debian started producing -dbgsym packages and
> putting them in a separate repository years ago:
> https://wiki.debian.org/AutomaticDebugPackages
> 
> dwz is used by virtually all RPM based distributions now, including
> OpenMandriva (a clang-based distro). I know this because I implemented
> it. :)

Updated the Wiki, thanks.
https://fedoraproject.org/wiki/Changes/DebugInfoStandardization


> I do not know whether Debian has started using dwz by default because
> I haven't dug into how the -dbgsym package generation works in detail.

I do not see DWZ or -fdebug-types-section to be used for example in
bash-dbgsym_5.1~beta1-1_amd64.deb .


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: gdb says "Unexpected size of section `.reg-xstate/...` in core file

[Jan Kratochvil]

On Mon, 13 Jul 2020 18:32:10 +0200, Zbigniew Jędrzejewski-Szmek wrote:
> gdb-9.2-2.fc33.x86_64 which is the latest rawhide build... Does gdb need 
> updating?

You may rather ask at gdb -at - sourceware.org.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: gdb says "Unexpected size of section `.reg-xstate/...` in core file

[Jan Kratochvil]

On Sat, 11 Jul 2020 16:52:41 +0200, Zbigniew Jędrzejewski-Szmek wrote:
> warning: Unexpected size of section `.reg-xstate/59286' in core file.
> #0  0x7f552f8b3b95 in raise () from /lib64/libc.so.6
> (gdb)
> 
> Is this gcc, gdb, binutils, something else?

gdb is older than kernel and you have too new CPU.
https://stackoverflow.com/a/62866856/2995591

You did not say versions + CPU you use.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: out of Koji disk space

[Jan Kratochvil]

On Fri, 10 Jul 2020 20:05:07 +0200, Kevin Fenzi wrote:
> Perhaps a compromise could be to generate them for rawhide builds, but
> not stable releases and if someone needs to debug things ask them to use
> the rawhide one/debuginfo?

That is not useful, system debuginfos are there to catch bugs easily and also
to catch difficult to reproduce bugs.

If you need to change the build to troubleshoot it then you can already build
a real debug version - particularly to use -O0 -g (instead of -O2 -g as in
Fedora) and also to enable package-specific compile-time debugging options.

I agree, there is slightly easier to download a Rawhide build than to build it
yourself but in both cases you are out of luck for unreproducible bugs.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: out of Koji disk space

[Jan Kratochvil]

On Tue, 07 Jul 2020 21:11:31 +0200, Kevin Fenzi wrote:
> I am not sure what to tell you here. Perhaps you could describe the
> reason you are working on the chromium debuginfo? Is it broken? Missing?
> Less useful that normal? 

Missing. Completely.

And it is not just about enabling it (=remove its disable) because with
default Fedora building options (no -fdebug-types-section and using DWZ) the
build will fail. That is because debuginfo will exceed 4GB which is the limit
of DWARF32 format and GNU tools do not much support DWARF64.
Enable debuginfo for x86_64 and aarch64.
https://src.fedoraproject.org/rpms/chromium/pull-request/27


> All I see in this thread is that you were working on it and it increased
> disk space usage, I must have missed the background/goal here?

The goal is to have debuginfo for Chromium. Or why other Fedora packages have
debuginfo but Chromium does not?


Thanks,
Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: out of Koji disk space

[Jan Kratochvil]

On Fri, 03 Jul 2020 14:52:00 +0200, Jan Kratochvil wrote:
> On Thu, 02 Jul 2020 02:04:48 +0200, Kevin Fenzi wrote:
> > On Wed, Jul 01, 2020 at 11:22:26PM +0200, Jan Kratochvil wrote:
> > > I will file a ticket after Spot says his opinion (or not).
> > 
> > Sure. 
> 
> https://src.fedoraproject.org/rpms/chromium/pull-request/27

So Spot says:
I'm not sure that the additional build time and storage demands of
this are worthwhile in practice.

Not sure what to do when after all the compiler tools effort the debuginfo is
not worth even the time building it.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: out of Koji disk space

[Jan Kratochvil]

On Thu, 02 Jul 2020 02:04:48 +0200, Kevin Fenzi wrote:
> On Wed, Jul 01, 2020 at 11:22:26PM +0200, Jan Kratochvil wrote:
> > I will file a ticket after Spot says his opinion (or not).
> 
> Sure. 

https://src.fedoraproject.org/rpms/chromium/pull-request/27
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: out of Koji disk space

[Jan Kratochvil]

On Thu, 02 Jul 2020 02:04:48 +0200, Kevin Fenzi wrote:
> I have fixed the buildhw-x86 boxes, they all now have ~365GB now. 

Yes, it has built fine:
https://koji.fedoraproject.org/koji/taskinfo?taskID=46455864


> Sorry, I misunderstood you... I thought you were wanting to test a
> scratch build that needed more space. You are proposing a change that
> will make official chromium builds take that extra space always?

Yes (I can do scratch builds on my machine faster). But I have succeeded
getting buildhw-* only on the 4th time.


> Our very fast a64 builders do not always have that amount of space, so
> it would have to fall back to the aarch64 buildvm's in that arch. 

aarch64 needs 187G + BUILDROOT (which got deleted), it may be 205GB total.

i686 cannot build chromium with debuginfo as linker runs out of 32-bit address
space, one would have to cross-build it which is sure not worth it.


> > I will file a ticket after Spot says his opinion (or not).
> 
> Sure. 

Spot has not said anyway yet.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: out of Koji disk space

[Jan Kratochvil]

On Wed, 01 Jul 2020 17:46:23 +0200, Kevin Fenzi wrote:
> * Is this only on x86_64 that you need this? Or all arches?

Sure we should have this on all arches but I haven't tested non-x86_64 yet.
There possibly may be also some memory problems even on x86_64 (it builds on
my machine w/64GB but Koji has only 16GB + big swap).


> * you're guessing you need 160GB? Is that pretty accurate?

$ du -shc `git ls-files` `cat sources |sed 's/^.*(\(.*\)).*$/\1/'` 
chromium-83.0.4103.116-2/ BUILDROOT/
... ...
161Gchromium-83.0.4103.116/
14G BUILDROOT/
176Gtotal


> I can fix that and they would have ~300GB or so each, would that be enough?

It depends what other packages are being built on the same host, doesn't it?


> in that case you could submit and cancel until you got a buildhw-x86 ? 

I am not Chromium maintainer (Tom 'spot' Callaway is). I am not even sure Spot
will accept this change (although then why not). I do not want to talk for
Spot if he likes to do the builds this way.


> Anyhow, I'd suggest filing an infrastructure ticket with your needs and
> we can see what we can come up with either short or long term for you. 

I will file a ticket after Spot says his opinion (or not).


Thanks,
Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: out of Koji disk space

[Jan Kratochvil]

On Wed, 01 Jul 2020 15:14:09 +0200, Stephen John Smoogen wrote:
> At the moment we have about 10% of the hardware
> we shipped back in operation and there are at least 2 to 3 weeks to
> get the rest up. Getting up larger builders is going to have to wait.

Sure there is no hurry with larger builders. I watch this problem of missing
debuginfo for years.


Thanks,
Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: out of Koji disk space

[Jan Kratochvil]

On Wed, 01 Jul 2020 13:43:57 +0200, Stephen John Smoogen wrote:
> In the end we have limited resources and you are running into those
> limits. We can either have fewer builders with more disk space and a
> long queue of builds or we can have more builders with smaller disk
> space.

Good to know disk space really is a limiting factor there (as it is in general
cheap compared to CPUs+RAM).

Sure a best step would be to have multiple (two) kinds of builders with
different disk sizes and blacklist/whitelist packages to them. I am aware that
idea is obvious, is there a Koji patch needed or even administration of two
kinds of builders is too much?


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

out of Koji disk space

[Jan Kratochvil]

Hi,

I have tried to enable debuginfo for Chromium but it cannot fit the disk.
During local build it has 160GB (on x86_64) and Koji shows (moreover AFAIK
shared for multiple builders):
https://kojipkgs.fedoraproject.org//work/tasks/7099/46377099/hw_info.log
Filesystem  Size  Used Avail Use% Mounted on
/dev/vda2   138G   71G   67G  52% /

Is there some way forward?

In 2013:
https://lists.fedoraproject.org/pipermail/devel/2013-January/176673.html
Looking at space we have, we are just going to bump them all to 150 or
so. Hopefully that will keep us for a while.


Jan
diff --git a/chromium-debug.patch b/chromium-debug.patch
new file mode 100644
index 000..181521a
--- /dev/null
+++ b/chromium-debug.patch
@@ -0,0 +1,12 @@
+--- chromium-83.0.4103.116/build/config/compiler/BUILD.gn.debug
2020-06-29 19:07:24.928680424 +0200
 chromium-83.0.4103.116/build/config/compiler/BUILD.gn  2020-06-29 
19:40:23.063688281 +0200
+@@ -2277,7 +2277,8 @@ config("symbols") {
+ # flag, so we can use use -g1 for pnacl and nacl-clang compiles.
+ # gcc nacl is is_nacl && !is_clang, pnacl and nacl-clang are && is_clang.
+ if (!is_nacl || is_clang) {
+-  cflags += [ "-g2" ]
++  cflags += [ "-g3" ]
++  cflags += [ "-fdebug-types-section" ]
+ }
+ 
+ if (is_clang && !is_nacl && !use_xcode_clang) {
diff --git a/chromium.spec b/chromium.spec
index 69dcead..3d9bfff 100644
--- a/chromium.spec
+++ b/chromium.spec
@@ -45,9 +45,8 @@
 %global headlessbuilddir out/Headless
 %global remotingbuilddir out/Remoting
 
-# Debuginfo packages aren't very useful here. If you need to debug
-# you should do a proper debug build (not implemented in this spec yet)
-%global debug_package %{nil}
+# Debuginfo would run out of 32-bit 4GB DWARF so use -fdebug-types-section.
+%global _find_debuginfo_dwz_opts %{nil}
 
 # %%{nil} for Stable; -beta for Beta; -dev for Devel
 # dash in -beta and -dev is intentional !
@@ -163,7 +162,7 @@ Name:   chromium%{chromium_channel}%{nsuffix}
 Name:  chromium%{chromium_channel}
 %endif
 Version:   %{majorversion}.0.4103.116
-Release:   3%{?dist}
+Release:   3debugtypes%{?dist}
 %if %{?freeworld}
 %if %{?shared}
 # chromium-libs-media-freeworld
@@ -286,6 +285,8 @@ Patch205:   chromium-81.0.4044.92-fix-vaapi-on-intel.patch
 
 # Apply these patches to work around EPEL8 issues
 Patch300:  
chromium-76.0.3809.132-rhel8-force-disable-use_gnome_keyring.patch
+# Enable chromium-debuginfo.rpm
+Patch900:  chromium-debug.patch
 
 # Use chromium-latest.py to generate clean tarball from released build 
tarballs, found here:
 # http://build.chromium.org/buildbot/official/
@@ -877,6 +878,7 @@ udev.
 %if 0%{?rhel} == 8
 %patch300 -p1 -b .disblegnomekeyring
 %endif
+%patch900 -p1 -b .debug
 
 # Change shebang in all relevant files in this directory and all subdirectories
 # See `man find` for how the `-exec command {} +` syntax works
@@ -959,12 +961,12 @@ popd
 
 # Core defines are flags that are true for both the browser and headless.
 CHROMIUM_CORE_GN_DEFINES=""
-CHROMIUM_CORE_GN_DEFINES+=' is_debug=false'
+CHROMIUM_CORE_GN_DEFINES+=' is_debug=false symbol_level=2'
 %ifarch x86_64 aarch64
 CHROMIUM_CORE_GN_DEFINES+=' system_libdir="lib64"'
 %endif
 CHROMIUM_CORE_GN_DEFINES+=' google_api_key="%{api_key}" 
google_default_client_id="%{default_client_id}" 
google_default_client_secret="%{default_client_secret}"'
-CHROMIUM_CORE_GN_DEFINES+=' is_clang=false use_sysroot=false use_gold=false 
fieldtrial_testing_like_official_build=true use_lld=false 
rtc_enable_symbol_export=true'
+CHROMIUM_CORE_GN_DEFINES+=' is_clang=false use_sysroot=false use_gold=true 
fieldtrial_testing_like_official_build=true use_lld=false 
rtc_enable_symbol_export=true'
 %if %{freeworld}
 CHROMIUM_CORE_GN_DEFINES+=' ffmpeg_branding="ChromeOS" proprietary_codecs=true'
 %else
@@ -992,7 +994,7 @@ CHROMIUM_BROWSER_GN_DEFINES+=' is_component_ffmpeg=true 
is_component_build=true'
 %else
 CHROMIUM_BROWSER_GN_DEFINES+=' is_component_ffmpeg=false 
is_component_build=false'
 %endif
-CHROMIUM_BROWSER_GN_DEFINES+=' blink_symbol_level=0 
enable_hangout_services_extension=true'
+CHROMIUM_BROWSER_GN_DEFINES+=' blink_symbol_level=2 
enable_hangout_services_extension=true'
 CHROMIUM_BROWSER_GN_DEFINES+=' use_aura=true'
 CHROMIUM_BROWSER_GN_DEFINES+=' enable_widevine=true'
 %if %{use_vaapi}
@@ -1379,11 +1381,6 @@ ulimit -n 4096
 . /opt/rh/devtoolset-%{dts_version}/enable
 %endif
 
-# Decrease the debuginfo verbosity, so it compiles in koji
-%ifarch %{ix86}
-%global optflags %(echo %{optflags} | sed 's/-g /-g1 /')
-%endif
-
 export 
PYTHONPATH="../../third_party/pyjson5/src:../../third_party/catapult/third_party/google-endpoints"
 
 echo
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.o

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Fri, 26 Jun 2020 10:23:31 +0200, Iñaki Ucar wrote:
> On Fri, 26 Jun 2020 at 10:20, Leigh Scott  wrote:
> > Please don't make this universal for all the spins, it should be optional.
> > TBH I don't give a damn if you do it to the workstation spin, please keep 
> > your grubby hands off the cinnamon spin :-)
> 
> That escalated quickly. :)

And MATE and XFCE spin. Maybe it really matches best the user base?


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Fri, 26 Jun 2020 09:57:49 +0200, Samuel Sieb wrote:
> On 6/26/20 12:42 AM, Jan Kratochvil wrote:
> > This will be always a catch up play. To make it working each completion 
> > script
> > would need to be part of the package it is implementing completion for.
> > Additionally it would need to be integrated into the program's commandline
> > parsing as otherwise it will never be 100% matching. There were efforts to
> 
> I have no idea what you're talking about.  The bash completion scripts are
> generally supplied by the package that they are for:
> # rpm -qf /usr/share/bash-completion/completions/dnf
> dnf-4.2.21-1.fc31.noarch

$ rpm -qlp bash-completion-2.8-8.fc32.noarch.rpm|grep 
^/usr/share/bash-completion/completions/|wc -l
652


> The dnf one works fine.

It does not as I have shown. Moreover it takes so much time to do dnf command
completion and one always has to ctrl-c it anyway. That is because dnf should
use cached results updated by cron and do not contact network during
interactive cache queries. If one really wants most fresh data there is
--refresh for that.


> But if you don't like how specific ones work, then file bugs.

The bug is bash-completion itself, I do not see what does it try to solve, the
best fix is just to remove it.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Fri, 26 Jun 2020 09:40:32 +0200, Samuel Sieb wrote:
> That's only if you start vi without a file.  Otherwise, you just get the
> text of the file on the screen and the bottom line with the filename and
> cursor position.  No info at all about what just happened.

OK, I agree. So what about instead of nano to do:

cat >>~/.vimrc <:x\'\ to\ save\ 
file,\ \':q!\'\ to\ quit.
EOH

It is the most user friendly solution (plus the system still remains to be
a UNIX!).


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Fri, 26 Jun 2020 09:36:05 +0200, Samuel Sieb wrote:
> Is this something that happens to you often?

It was happening often on an Athlon computer which I retired recently.
So no longer.

Or rather it still happens for me (X1 carbon 6th with docking station) but
during switching to X when the screen remains black even when I am not using
"rhgb quiet".


> No need to be spammed for the 99.999%
> of boots that have no problem.  That's a terrible thing for users to see.

I feel blind when I do not see what is happening and I feel scared it will
lock up again and I will be unable to debug it. Besides that it is much more
pretty to see what is happening and it makes the waiting time shorter.

All pros and no con. Yes, it is a personal preference. Yes, I understand most
of computer users prefer "rhgb quiet". I have some doubts most of _Fedora_
users really prefer it.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Fri, 26 Jun 2020 09:38:06 +0200, Samuel Sieb wrote:
> On 6/26/20 12:27 AM, Jan Kratochvil wrote:
> > Some replies also do not like this change, I am not alone. Still it looks 
> > like
> > there area really more votes for this change than against it. Fine with me.
> 
> From all the replies here, yours is the only one I've seen that's really
> against it.

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/6OPZA57OBV3FPJPGNRYWBROL3WR2B4NG/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Fri, 26 Jun 2020 03:31:10 +0200, Samuel Sieb wrote:
> But regardless, that's something to fix in the dnf bash completion scripts,
> not a reason to completely disable completion as the earlier poster said.

TL;DR it regresses the original bash completion feature.

This will be always a catch up play. To make it working each completion script
would need to be part of the package it is implementing completion for.
Additionally it would need to be integrated into the program's commandline
parsing as otherwise it will never be 100% matching. There were efforts to
unify commandline parsing between programs but that never happened:
GNU getopt_long, glib GOption, 
https://en.wikipedia.org/wiki/Getopt#In_other_languages

I have shown just 'dnf' but if you really want I can find arbitrary number of
other such programs and bugs where the completion script implements things
differently than the program itself.

This is why IMHO bash-completion is only an experimental unfinished feature
and it should not be the default.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Fri, 26 Jun 2020 01:34:19 +0200, Samuel Sieb wrote:
> I agree with your points, but pressing ESC only reverses the "rhgb" part,
> the kernel output is still "quiet".

If the kernel really locks up it is locked up and no keys work anymore.
Without "rhgb quiet" one can make a photo of the screen.

A better solution would be to use serial console all the time but the ethernet
one does not work during crashes and physical serial hardware is no longer
available in modern computers.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Fri, 26 Jun 2020 02:25:24 +0200, Matthew Miller wrote:
> But, if you don't like our offerings that are targetted that way, I suggest
> you make a spin or remix that has all of defaults _you_ want.

So FESCo has decided and there is no point of discussing this Change, that is
how it will be and any resistance is futile? The subject even says it is
a "Change _proposal_".

Some replies also do not like this change, I am not alone. Still it looks like
there area really more votes for this change than against it. Fine with me.

BTW this change (contrary to other ones I mentioned) is not such a problem as
it is overridable in $HOME/.bashrc so it does not really need a distro spin.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Fri, 26 Jun 2020 03:48:56 +0200, Adam Williamson wrote:
> 3. provides a handy reference of key combos you can use to get help,
> save the file, and exit. Yes, you have to know that ^O means "ctrl+O",
> but figuring that out is a lot easier than working out how to drive vi
> from scratch.

After starting 'vi' there is:
~type  :help  orfor on-line help

And after :help there is:
Close this window:  Use ":q".
   Get out of Vim:  Use ":qa!" (careful, all changes are lost!).
  WHAT  PREPENDEXAMPLE
  Normal mode command  :help x
  Visual mode command v_   :help v_u
  Insert mode command i_   :help i_

Isn't that even more helpful than the '^' means 'ctrl' magic?


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Thu, 25 Jun 2020 21:58:28 +0200, Jonathan Wakely wrote:
> try 'ls $HOME/' without the direxpand
> option set, and tell me what on earth that is for.

It works as expected, what is the problem? Using ctrl-e instead of direxpand
but keeping there $HOME/ may be what one wants.


> the only thing I absolutely insist on
> changing on a new Fedora install is to disable abrt. No, I don't want
> to report it to bugzilla every time a unit test or quick hacked
> together piece of code terminates abnormally.

ABRT primarily crashes machine used for development.
I gave up on reopening the Bug after it always gets EOL-closed unfixed:
-fsanitize=address locks up abrt-hook-ccpp
https://bugzilla.redhat.com/show_bug.cgi?id=1164548


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Thu, 25 Jun 2020 21:21:37 +0200, Chris Adams wrote:
> I'm not sure why you think end-users can't use a free OS.

First steps of end-users is to install Chrome, Spotify and VirtualBox.
So there is left no advantage of a Free OS.


> I've run with SELinux enabled for years, rarely if ever causes problems
> for typical stuff.

Sooner or later something does not work. I do not want to open this can of
worms whether SELinux yes or not, it may be a good idea but IMNSHO it is not
for a development machine.


> I have no idea why you'd remove a useful tool like bash-completion.  It
> has lots of things useful for developers.

I agree the idea of bash-completion is nice. But it is so severly incomplete
it is more a burden than help. Unfortunately I no longer remember all the bugs
I faced before I started removing it years ago but a simple one is:

$ wget -O somepackage.rpm https://...
$ dnf install som

$ dnf install sombok-


> Unless you are doing kernel development, why do you care what the kernel
> messages say?  On my systems, they go by too fast to read anyway.

When it locks up (during updating firmware on my Athlon machine) I see just
a black screen. When I reboot without rhgb/quiet the problem is not
reproducible as it happens only rarely. There are many reasons why kernels
sometimes fail to boot, why to give up on troubleshooting?


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 33 System-Wide Change proposal: Make nano the default editor

[Jan Kratochvil]

On Thu, 25 Jun 2020 19:18:59 +0200, Ben Cotton wrote:
> In contrast, Nano offers the kind of graphical text editing experience
> that people are used to,

This is another step trying to make Fedora end-user friendly while the only
effect is making it hostile to developers. As Fedora will never be used by
end-users as it conflicts with Fedora's foundation Freedom. With each such
step it takes more and more effort to make a new Fedora installation usable by
a developer (setenforce 0, dnf remove bash-completion, remove rhgb quiet etc.).


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: It seems that gdb cannot find debuginfo on f32

[Jan Kratochvil]

On Sun, 07 Jun 2020 17:22:33 +0200, Barry Scott wrote:
> python3-debuginfo.x86_64 3.8.3-1.fc32 
>   
> @updates-debuginfo
...
> Reading symbols from /usr/bin/python3.8...
> Reading symbols from .gnu_debugdata for /usr/bin/python3.8...
> (No debugging symbols found in .gnu_debugdata for /usr/bin/python3.8)
> Missing separate debuginfos, use: dnf debuginfo-install 
> python3-3.8.3-1.fc32.x86_64
> (gdb) 
> 
> Am I doing something wrong?

I do not see the problem, you can verify:

$ ls -l /usr/lib/debug/usr/bin/python3.8-3.8.3-1.fc32.x86_64.debug
-r--r--r-- 1 root root 17800 May 19 10:31 
/usr/lib/debug/usr/bin/python3.8-3.8.3-1.fc32.x86_64.debug
$ rpm -qf /usr/lib/debug/usr/bin/python3.8-3.8.3-1.fc32.x86_64.debug
python3-debuginfo-3.8.3-1.fc32.x86_64
$ rpm -V python3-debuginfo-3.8.3-1.fc32.x86_64
$ _


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Packagers with no corresponding valid bugzilla accounts

[Jan Kratochvil]

On Sat, 13 Jun 2020 22:10:36 +0200, Pierre-Yves Chibon wrote:
> jkratoch

This account is set as "inactive" in FAS. Couldn't you regenerate this list
with inactive accounts excluded?

I would like to remove this old account but FAS admin interface cannot do
that, I can only set it as "inactive".

I am using active account "jankratochvil" instead. In the past I was told it
is easier for Fedora admins to create a new account than to rename my old
account.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Supporting hibernation in Workstation ed., draft 1

[Jan Kratochvil]

On Sat, 30 May 2020 00:58:26 +0200, Chris Murphy wrote:
> The Fedora Workstation working group recognizes hibernation can be
> useful, but due to impediments it's currently not practical to support
> it.

TL;DR let's go the s2idle way as that is the only one which may work. But for
me it resumes immediately again (similar to the "suspend" Bugs below).
Also I had to do 'echo freeze >/sys/power/state' for s2idle as systemctl has
no such option (nor there is any GUI option for s3idle AFAIK).

--

The problem is "suspend" depends on computer BIOS, sometimes it does not work
and due to the BIOS it is difficult/impossible to debug:
Kernel fails to suspend due to rmi_smbus
https://bugzilla.redhat.com/show_bug.cgi?id=1480602
Cannot successfully suspend, machine starts again after a few seconds
https://bugzilla.redhat.com/show_bug.cgi?id=1781566
Bugs in "suspend" are also mentioned in the first article about "S3":

https://lists.fedoraproject.org/archives/list/desk...@lists.fedoraproject.org/thread/V5MOCX23KU45J3WXUN6TCGEJYQLXQYUL/

I agree hibernation also sometimes does not work for me (in 20% cases it just
suddenly reboots during resuming); X1 Carbon 6th gen (Red Hat one).

Having both suspend & resume was useful one had a chance at least one of them
works sometimes. If the 'out of the box' installations will no longer offer
the hibernation option it means less compatibility with hardware.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: late generation of assemble code

[Jan Kratochvil]

On Sun, 24 May 2020 05:21:05 +0200, Paul Dufresne via devel wrote:
> The idea was to push code generation as near as possible of code execution.
> Because at execution time, you know what are the specific features of the
> CPU, and what is used to most often by the user of the program.

In Free Software you have that already - it is called the source code.
If you want host-specific optimizations use Gentoo Linux.

But that approach (incl. the possible LLVM JIT) has its own kind of
disadvantages such as unreproducibility of compilation-specific problems
elsewhere - no way to have meaningful build-ids, ABRT retrace server etc.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Getting security updates out to users sooner

[Jan Kratochvil]

On Fri, 17 Apr 2020 06:55:10 +0200, Michel Alexandre Salim wrote:
> For kernel updates this is probably not a good idea. Given that updates
> potentially introduce regressions, being able to distinguish updates with
> known CVEs that we do need to roll out immediately, versus other updates we
> can do more compatibility testing on, is critical.

Even when there is a kernel regression a -1 vote gets immediately overvoted by
the +1s of majority so the update gets pushed anyway. So I do not see what is
the purpose of the voting at all. As an example:

https://bodhi.fedoraproject.org/updates/FEDORA-2020-3cd64d683c#comment-1258825
 = kernel-5.5.6-201.fc31


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: The Chromium Dilemma

[Jan Kratochvil]

On Mon, 13 Apr 2020 18:43:07 +0200, Tom Callaway wrote:
> The linker said: error adding symbols: Malformed archive. Searching leads
> me to translate that error to "too many open files". See:
> https://github.com/OSSystems/meta-browser/issues/194
> 
> Apparently, gold does not have this issue, but I have not tested.

gold is also limited by 'ulimit -S -n', I had to raise it while building LLDB
(using -DLLVM_USE_LINKER=gold).


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Why does gdb dlopen() librpm instead of linking to it?

[Jan Kratochvil]

On Fri, 03 Jan 2020 22:45:52 +0100, Neal Gompa wrote:
> Can someone please explain why gdb dlopen()'s librpm instead of just
> doing proper compile-time linking?

Because when it was written (2008) it was common to transfer /usr/bin/gdb
binary across OS versions (maybe incl. RHEL) as GDB had only very few shared
library dependencies (no Python that time) which were ABI compatible.
After implementing the rpm functionality the DT_NEEDED of librpm was the only
reason why one could no longer use /usr/bin/gdb across OSes as librpm major
version changes very often. Therefore I changed the DT_NEEDD to dlopen().

Nowadays there are tons of DT_NEEDED dependencies of /usr/bin/gdb so one does
not even think about the possibility using the binary on some other OS.

You are right dlopen() could be changed to DT_NEEDED back. But then rather
this whole librpm dependency patch is being rewritten some other ways (and it
is longer done by me who wrote it in that 2008). That librpm dependency is
painful anyway as librpm authors do not like it to be used by 3rd party apps
and they prefer popen()ing some rpm commands instead:
rpmdb locking broken by other-arch rpmquery
https://bugzilla.redhat.com/show_bug.cgi?id=486423#c1


> Best I can tell, our patches to GDB support both modes, and it seems to be
> unnecessarily painful for us to have to track the soversion like that
> instead of just letting it pick it up.

The positive side is that GDB handles gracefully if the librpm (of proper
version) is missing. But the gdb.spec correctly fails to build it.

gdb.spec could auto-detect the current librpm version.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Gcc/annobin changes in F32

[Jan Kratochvil]

On Mon, 30 Dec 2019 00:58:24 +0100, Michael Cronenworth wrote:
> warning: Could not find DWO CU 
> CMakeFiles/kodi-xrandr.dir/xbmc-xrandr.c.dwo(0xc444049b4814c563) referenced 
> by CU at offset 0x0 [in module 
> /builddir/build/BUILDROOT/kodi-18.5-1.fc32.x86_64/usr/lib64/kodi/kodi-xrandr]

Unaware why it builds in F-31 but in the log there is:

-- Checking to see if CXX compiler accepts flag -gsplit-dwarf - yes
...
[10/1538] /usr/bin/cc ... -gsplit-dwarf ...

-gsplit-dwarf generates DWO files which are not supported for RPM builds
(due to DWZ). Try to turn it off somehow in the package's cmake/configure,
I haven't investigated how for this package.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

fstrim and LUKS [Re: Fedora 32 System-Wide Change proposal: Enable fstrim.timer by default]

[Jan Kratochvil]

On Thu, 19 Dec 2019 22:42:02 +0100, Ben Cotton wrote:
> == Summary ==
> Enabling fstrim.timer will cause fstrim.service to execute weekly,
> which in turn executes `/usr/sbin/fstrim --fstab --verbose --quiet`

This is AFAIK not enough for LUKS drives, will it be supported for LUKS?


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Disallow Empty Password By Default

[Jan Kratochvil]

On Mon, 25 Nov 2019 22:59:18 +0100, Samuel Sieb wrote:
> I don't understand why this change is
> necessary at all.  It only affects local logins and if someone wants to have
> an empty password, why make it so difficult?  It's their choice.  It's not
> like Fedora has any default logins with empty passwords, the user has to
> make their own.

Yes, what about firewalled virtual machines for development testing purposes?
There any password makes no sense.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Build Python 3 to statically link with libpython3.8.a for better performance

[Jan Kratochvil]

On Thu, 07 Nov 2019 22:36:44 +0100, Jan Kratochvil wrote:
> On Thu, 07 Nov 2019 15:59:41 +0100, Victor Stinner wrote:
> > I cannot explain why inlining cannot be done more often in libpython.
> > 
> > I cannot explain why PLT is needed when a libpython function calls a 
> > libpython function.
> 
> Could you re-run the benchmark with shared library but with
> -fno-semantic-interposition? I have run it locally but it takes a lot of time.

nbody python3-3.7.5-1.fc30.x86_64: Mean +- std dev: 217 ms +- 2 ms
nbody -fno-semantic-interposition: Mean +- std dev: 203 ms +- 3 ms - 6.9%
nbody static linkage claim:-27%

So -fno-semantic-interposition does help but it is not the whole static gain.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Build Python 3 to statically link with libpython3.8.a for better performance

[Jan Kratochvil]

On Thu, 07 Nov 2019 15:59:41 +0100, Victor Stinner wrote:
> I cannot explain why inlining cannot be done more often in libpython.
> 
> I cannot explain why PLT is needed when a libpython function calls a 
> libpython function.

Could you re-run the benchmark with shared library but with
-fno-semantic-interposition? I have run it locally but it takes a lot of time.


Thanks,
Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Build Python 3 to statically link with libpython3.8.a for better performance

[Jan Kratochvil]

On Wed, 06 Nov 2019 11:49:18 +0100, Vít Ondruch wrote:
> > we can achieve a
> > performance gain of 5% to 27% depending on the workload.
> 
> Where are these number coming from? And what is the reason for the
> performance hit for dynamically linked Python?

Yes, it looks suspicious. -fPIC was a performance hit for i686 but it no
longer affects x86_64.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: inotify-tools with dead upstream cannot be fixed in Fedora

[Jan Kratochvil]

On Thu, 05 Sep 2019 14:43:48 +0200, Stephen Gallagher wrote:
> This isn't required, no. That being said, if this package is abandoned
> upstream then it probably makes sense to contact the Debian maintainer
> and establish a fork where our projects can at least share patches. If
> upstream comes back to life, we can merge them back together then.

I have been given co-maintainership of the Fedora package so I have finally
imported 2 crasher fixing patches from Debian into Fedora. So I find the issue
resolved from my point of view.

I have now also sent mail to the Debian maintaner about a new possible
upstream fork.

Thanks for the response.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: inotify-tools with dead upstream cannot be fixed in Fedora

[Jan Kratochvil]

On Mon, 02 Sep 2019 23:33:32 +0200, Richard W.M. Jones wrote:
> On Sun, Sep 01, 2019 at 09:33:29PM +0200, Jan Kratochvil wrote:
> > Fix stack overflow in: `inotifytools_replace_filename`
> > https://bugzilla.redhat.com/show_bug.cgi?id=1741472
> > 
> > Not sure how to fix this crashing bug. Upstream is dead while Fedora package
> > maintainer requires to upstream the fix first. A Catch 22.
> > 
> > Debian has the crasher fixed by an off-trunk simple patch, Fedora cannot?
> 
> I would start by pulling in the Debian fix into the Fedora package.

This is what I tried in the Bug above and I failed. So I have submitted
a request to become co-maintainer with no response so far.

(Besides that the Debian fix compared to my fix is a hack, not properly coded
from programming point of view. But I agree the Debian fix does work.)


> Then talk to the Debian folk and see if you can set up a simple git
> repo on one of the many free hosting services to collect these patches
> "upstream".

There is already upstream but its maintainer does not respond. So do you mean
to make a new upstream? Is it really required to push 2 characters of a fix to
Fedora package?


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

inotify-tools with dead upstream cannot be fixed in Fedora

[Jan Kratochvil]

Hi,

Fix stack overflow in: `inotifytools_replace_filename`
https://bugzilla.redhat.com/show_bug.cgi?id=1741472

Not sure how to fix this crashing bug. Upstream is dead while Fedora package
maintainer requires to upstream the fix first. A Catch 22.

Debian has the crasher fixed by an off-trunk simple patch, Fedora cannot?


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

[Jan Kratochvil]

On Sun, 11 Aug 2019 20:54:28 +0200, Chris Murphy wrote:
> and likely experiences data loss and possibly even file system
> corruption as a direct consequence of having to force power off on the
> machine because for all practical purposes normal control has been
> lost.

Not really, this is what journaling filesystem is there for.

But then there still can be an application-level data corruptions if an
application does not handle its sudden termination properly.
Which should be rare but IIRC I did see it for example with Firefox.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

[Jan Kratochvil]

On Sun, 11 Aug 2019 17:50:17 +0200, Chris Murphy wrote:
> I don't follow. You're saying RelWithDebInfo is never suitable for a
> local build?

Most of the time. What is your use case for it?


> isn't relevant to getting a successful build.

With powerful enough machine everything is possible.  Just be aware
RelWithDebInfo is the most resource demanding option compared to Release and
Debug and at the same time it is the least useful one for local builds.


> In file included from Source/JavaScriptCore/config.h:32,
>  from 
> Source/JavaScriptCore/llint/LLIntSettingsExtractor.cpp:26:
> Source/JavaScriptCore/runtime/JSExportMacros.h:32:10: fatal error: 
> wtf/ExportMacros.h: No such file or directory

You are reinventing the wheel Fedora packager has already done for this
package.  I guess you are missing some dependency.  If you have a problem
stick to the proven build (unless it is temporarily FTBFS which this package
is not now).  I think Fedora recommends mock for such rebuild but I find mock
inconvenient for local development so I use (I have some scripts for that):
dnf download --source webkit2gtk3
mkdir webkit2gtk3-2.24.3-1.fc30.src
cd webkit2gtk3-2.24.3-1.fc30.src
rpm2cpio ../webkit2gtk3-2.24.3-1.fc30.src.rpm|cpio -id
function rpmbuildlocal { time MAKEFLAGS= rpmbuild --define "_topdir 
$PWD" --define "_builddir $PWD" --define "_rpmdir $PWD" --define "_sourcedir 
$PWD" --define "_specdir $PWD" --define "_srcrpmdir $PWD" --define 
"_build_name_fmt %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm" "$@"; rmdir 
&>/dev/null BUILDROOT; }
# Is the .src.rpm rebuild still needed?  
https://bugzilla.redhat.com/show_bug.cgi?id=1210276
rpmbuildlocal -bs *.spec
sudo dnf builddep webkit2gtk3-2.24.3-1.fc30.src.rpm
rm webkit2gtk3-2.24.3-1.fc30.src.rpm
rpmbuildlocal -bc webkit2gtk3.spec 2>&1|tee log
# or -bb or what do you want.
It has built fine for me here now.


> Let's take another argument. If the user manually specifies 'ninja -j
> 64' on this same system, is that sabotage?

For untrusted users Linux has given up for that, it is too big can of worms.
Use virtual machine (KVM) with specified resources (memory size).  Nowadays it
should be also possible with less overhead by using Docker containers.

If you mean some local builds of your own causing runaway then
(1) Turn off swap as RAM is cheap enough today.
If something really runs out of the RAM it gets killed by kernel OOM.
(2) Have the swap on NVMe, it from my experience does not kill the machine.
(3) Use some reasonable ulimits in your ~/.bash_profile.
(4) When the machine is really unresponsible login there from a different box
and kill the culprits. From my own experience the machine is still able to
accept new SSH connection, despite a bit slowly.
But yes, I agree this problem has AFAIK no perfect solution.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

[Jan Kratochvil]

On Fri, 09 Aug 2019 23:50:43 +0200, Chris Murphy wrote:
> $ cmake -DPORT=GTK -DCMAKE_BUILD_TYPE=RelWithDebInfo -GNinja

RelWithDebInfo is -O2 -g build.  That is not suitable for debugging, for
debugging you should use -DCMAKE_BUILD_TYPE=Debug (that is -g).
RelWithDebInfo is useful for final rpm packages but those are build in Koji.

Debug build will have smaller debug info so the problem may go away.

If it does not go away then tune the parallelism. Low -j makes the build
needlessly slow during compilation phase while high -j (up to about #cpus
+ 2 or so) will make the final linking phase with debug info to run out of
memory. This is why LLVM has separate "-j" for the linking phase but that is
implemented only in LLVM CMakeLists.txt files:
https://llvm.org/docs/CMake.html
LLVM_PARALLEL_LINK_JOBS
So that you leave the default -j high but set LLVM_PARALLEL_LINK_JOBS to 1 or 2.

Other options for faster build times are also LLVM specific:
-DLLVM_USE_LINKER=gold (maybe also lld now?)
 - as ld.gold or ld.lld are faster than ld.bfd
-DLLVM_USE_SPLIT_DWARF=ON
 - Linking phase no longer deals with the huge debug info

Which should be applicable for other projects by something like (untested!):
-DCMAKE_C_FLAGS="-gsplit-dwarf"
-DCMAKE_CXX_FLAGS="-gsplit-dwarf"
-DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=gold -Wl,--gdb-index"
-DCMAKE_SHARED_LINKER_FLAGS="-fuse-ld=gold -Wl,--gdb-index"

(That gdb-index is useful if you are really going to debug it using GDB as
I expect you are going to do when you want RelWithDebInfo and not Release; but
then I would recommend Debug in such case anyway as debugging optimized code
is very difficult.)


> is there a practical way right now of enforcing CPU
> and memory limits on unprivileged applications?

$ help ulimit
  -mthe maximum resident set size
  -uthe maximum number of user processes
  -vthe size of virtual memory

One can also run it with 'nice -n19', 'ionice -c3'
and/or "cgclassify -g '*':hammock" (config attached).

But after all I recommend just more memory, it is cheap nowadays and I find
64GB just about the right size.


Jan
mount {
cpu = /cgroup/cpu;
memory  = /cgroup/memory;
blkio   = /cgroup/blkio;
}

group hammock {
perm {
task {
uid = jkratoch;
gid = jkratoch;
}
admin {
uid = jkratoch;
gid = jkratoch;
}
}
cpu {
cpu.shares = 2;
}
memory {
memory.limit_in_bytes = 2G;
}
blkio {
blkio.weight = 100;
}
}
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Valgrind and Fedora debuginfo: best GUI

[Jan Kratochvil]

On Thu, 06 Dec 2018 11:59:35 +0100, Germano Massullo wrote:
> This seems to not happen with Valgrind

I would first suggest using compiler option -fsanitize=address instead of
valgrind.


Jan Kratochvil
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Reliability test for hard drives and SSD

[Jan Kratochvil]

On Wed, 08 Aug 2018 02:11:08 +0200, Andrey Ponomarenko wrote:
> 2) The -get-inventory-id option creates a new inventory id (or 'group') to
> mark new probes by this id. You've created too many inventory ids per day
> and received an error when trying to create more. Usually it's enough to
> have only one inventory id.
> 
> bf0a7f04b4 is a probe id (10-digit), not the inventory id. Your inventory id
> is 8557d9ad (8-digit).

How did you find out the 8-digit id?


> The command:
> 
> hw-probe -get-inventory-id
> 
> creates a new group/inventory id (G) each time.

#  -inventory-id ID
#  Set inventory ID of the probe. You can get this ID
#  by the -get-inventory-id option.
#  
#  -get-inventory-id
#  Get inventory ID.

From the help it looks to me -get-inventory-id should return what has been set
by -inventory-id. The text "Get inventory ID." does not suggest me it should
create anything new.


Thanks,
Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/G66OOKUYWVRNXWJJPTO7WJ53W77EKYEI/

Re: Reliability test for hard drives and SSD

[Jan Kratochvil]

Hello,

On Tue, 07 Aug 2018 10:21:05 +0200, Andrey Ponomarenko wrote:
> sudo dnf config-manager --add-repo 
> https://download.opensuse.org/repositories/home:/linuxbuild/Fedora_Rawhide/home:linuxbuild.repo
> sudo dnf config-manager --set-enabled home_linuxbuild
> sudo dnf install hw-probe

that looks better, thanks. Still it is from non-Fedora build but then it
contains only a text script anyway.


> It's better to use lightweight all-in-one AppImage (F15 and higher) or Snap
> (F26 and higher):
> 
> https://github.com/linuxhw/hw-probe#appimage
> https://github.com/linuxhw/hw-probe#snap

I do not like recommending such insecure behavior - to run untrusted binaries
with root permissions - without even warning it is insecure.


> Usually people don't create more than two probes per computer, so receiving
> updates is not necessary.

It is insecure to keep non-updated software installed/available on your
computer.


Anyway it somehow does not work for me, in the past I registered
https://linux-hardware.org/?probe=bf0a7f04b4

But now I cannot update the info trying to prevent a duplicate:
# hw-probe   -get-inventory-id
Group ID: 8557d9ad
# hw-probe   -inventory-id bf0a7f04b4
# hw-probe   -get-inventory-id
Group ID: 4d2703f5
# hw-probe   -inventory-id bf0a7f04b4
# hw-probe   -get-inventory-id
Group ID: 07746fb5
# hw-probe   -get-inventory-id
Group ID: 077ce20e
# hw-probe   -inventory-id 0xbf0a7f04b4
# hw-probe   -get-inventory-id
Group ID: 7b448de5
# hw-probe   -get-inventory-id
Error request
# hw-probe   -get-inventory-id
Error request


Jan Kratochvil
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/IQT24S5F43DIAFNK6CIUSYPJPQMGJBB4/

Re: Why are we shipping debug builds of pythons?

[Jan Kratochvil]

On Mon, 06 Aug 2018 17:30:02 +0200, David Malcolm wrote:
> On Mon, 2018-08-06 at 13:15 +0200, Jan Kratochvil wrote:
> > That confirms the whole OS "Checked Build" variant would solve even this
> > problem.
> 
> Jan:  In my view, it's not a problem, it's a feature.  An OS-wide
> change wouldn't turn on "--with-pydebug" for Python, unless it was
> directly coded into the specfile.

Packages would be recommended to turn on such flags for that variant of OS,
such as for example for glib2:
--enable-debug --enable-gc-friendly --disable-mem-pools 
etc.
At least this is what IIUC Microsoft does for their Checked Build.


> The change to the ABI is due to two new fields with the base struct for
> python objects.

That is not a problem for a Checked Build OS as all the packages are compiled
for the different ABI.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/WOIBNF2ELBQYTQZDO4BAC2HMYJBDO4K2/

Re: Why are we shipping debug builds of pythons?

[Jan Kratochvil]

On Mon, 06 Aug 2018 10:57:31 +0200, Petr Viktorin wrote:
> On 08/05/18 14:01, Jan Kratochvil wrote:
> > That is all together messy. This is why Microsoft has their debug build of
> > their whole OS - Windows - called Checked Build:
> > 
> > https://docs.microsoft.com/en-us/windows-hardware/drivers/devtest/checked-and-free-build-differences
> 
> The main issue with --with-pydebug is that it changes the ABI, i.e. all
> extensions need to be re-built specifically for it (and debug extensions
> outside the standard library aren't usually packaged in Fedora). That makes
> it much less useful than if it just used less optimizations.

That confirms the whole OS "Checked Build" variant would solve even this
problem.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/LFTRUAGMLOFIZIRLG7UL2D2KG3LVKOIX/

Re: Reliability test for hard drives and SSD

[Jan Kratochvil]

On Sun, 05 Aug 2018 14:47:06 +0200, Andrey Ponomarenko wrote:
> I've just created Fedora package for hw-probe. See
> https://github.com/linuxhw/hw-probe/blob/master/INSTALL.md#install-on-fedora.

Installing a package out of repository for its maintenance is not great.
I tried to find a repository for it but what is Build vs. Publish
vs. Use for Build?
https://build.opensuse.org/repositories/home:linuxbuild/hw-probe
COPR would be better for Fedora, until it is in standard Fedora distro.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/YUNWP4MDBSES5NHQQKTVUHFCJEW4F7CL/

Re: Why are we shipping debug builds of pythons?

[Jan Kratochvil]

On Sun, 05 Aug 2018 13:39:58 +0200, Charalampos Stratakis wrote:
> Here we are referring to the python2-debug or python3-debug builds, which
> are just extra python builds that are compiled with the --with-pydebug flag

Not just --with-pydebug:

--
%if %{with debug_build}
BuildPython debug \
  "--without-ensurepip --with-pydebug" \
  "-O0"
%endif # with debug_build

BuildPython optimized \
  "--without-ensurepip %{optimizations_flag}" \
  ""
--

Many packages have their *-debug variants. And then some packages (like
kernel) switch even their standard builds to debug mode but only in Rawhide.
Despite all that the *-debug package one occasionally needs is then missing
and one still has to build it locally.

That is all together messy. This is why Microsoft has their debug build of
their whole OS - Windows - called Checked Build:

https://docs.microsoft.com/en-us/windows-hardware/drivers/devtest/checked-and-free-build-differences


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/VNQAZGQHXM764P3LJPKN5SA43QBGXRKN/

Re: Guideline change: glibc malloc as the C/C++/Rust allocator

[Jan Kratochvil]

On Thu, 26 Jul 2018 21:14:14 +0200, Carlos O'Donell wrote:
> Bug 1430223 - In some conditions, tcmalloc memalign will segfault
> https://bugzilla.redhat.com/show_bug.cgi?id=1430223

It looks as a tcmalloc bug which could be fixed; it also has been probably
fixed in the meantime as stated there.


> I think a key point here is to reduce the number of allocators being
> used by the distribution so we can keep the quality high and help
> our users when they have problems.

So why glibc greated an N+1 allocator (by DJ Delorie) instead of just
importing/using tcmalloc (which is license-compatible with glibc)?


On Thu, 26 Jul 2018 22:34:09 +0200, Florian Weimer wrote:
> https://bugzilla.redhat.com/show_bug.cgi?id=1237260
 = gperftools: tcmalloc breaks ABI on x86_64

You (Florian Weimer) have not provided any countercase (*) why small
allocations under alignment size (16 bytes) still should be aligned to 16 bytes.
This is why tcmalloc upstream closed it:
https://github.com/gperftools/gperftools/issues/724

(*) https://github.com/gperftools/gperftools/issues/724#issuecomment-147369562
I do not see how that can be useful in practice, do you?


> https://bugzilla.redhat.com/show_bug.cgi?id=1303323
 = tcsh: interposed malloc is not ABI-compliant due to lack of alignment

This was a bug in tcsh custom allocator. (That is unrelated to tcmalloc.)

 
> http://www.erahm.org/2016/03/24/minimum-alignment-of-allocation-across-platforms/

Looking at its source it looks to me mozjemalloc still in use by Fedora
Firefox still has only 8-byte alignment.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/YNZXINTXTMQ4SGFU6LT5A4FDQ4ANXGLS/

Re: Guideline change: glibc malloc as the C/C++/Rust allocator

[Jan Kratochvil]

On Thu, 26 Jul 2018 17:43:06 +0200, Daniel P. Berrangé wrote:
> # dnf repoquery --whatrequires 'libjemalloc.so.2()(64bit)'
> 389-ds-base-devel-0:1.4.0.11-2.fc28.x86_64
> blender-1:2.79b-2.fc28.x86_64
> blender-1:2.79b-3.fc28.x86_64
> blenderplayer-1:2.79b-2.fc28.x86_64
> blenderplayer-1:2.79b-3.fc28.x86_64
> jemalloc-devel-0:5.0.1-5.fc28.x86_64
> neovim-0:0.2.2-1.fc28.x86_64
> neovim-0:0.3.0-2.fc28.x86_64
> redis-0:4.0.10-1.fc28.x86_64
> redis-0:4.0.9-1.fc28.x86_64
> varnish-0:5.2.1-4.fc28.x86_64

firefox on x86* is using its internal fork of jemalloc (mozjemalloc).


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/IQ474EQXGAHPYRPPBOTLRAUVREQUKA2D/

Re: [HEADS UP] Removal of GCC from the buildroot

[Jan Kratochvil]

On Wed, 11 Jul 2018 18:26:23 +0200, Josh Stone wrote:
> So I hear you like compile-time safety...
> 
> No, I don't seriously want to get into a language comparison here,
> except to say that it's reasonable for the world to expand beyond C/C++,

There is no C/C++ language.  There are two orthogonal languages, C and C++.
(And some people say C++11 and C++03 are also orthogonal.)


Jan Kratochvil
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/2H7QIJJZAEK7424E3K36CL4CZZEN6YVE/

Re: Packages which needlessly use %defattr

[Jan Kratochvil]

On Wed, 04 Jul 2018 12:18:00 +0200, Zbigniew Jędrzejewski-Szmek wrote:
> What about just doing a mass specfile update now? I think asking
> individual maintainers to fix their packages isn't worth their
> time. It's a safe change,

Is it? I haven't found whether this change is compatible with CentOS-6 (and 7)
which could break them if you just build the Fedora Rawhide .spec for them.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/N7CSJ5AEZT2Q4O3CJ4MD4NJBJN3OD3V3/

Re: Call for users of darkserver

[Jan Kratochvil]

On Thu, 01 Mar 2018 19:30:28 +0100, Pierre-Yves Chibon wrote:
> On Fri, Feb 16, 2018 at 03:04:11PM +0100, Jan Kratochvil wrote:
> > On Fri, 16 Feb 2018 11:02:25 +0100, Pierre-Yves Chibon wrote:
> > It seems to me interactive crash core file analysis is just not a goal of 
> > ABRT
> > (which is what was the goal of darkserver).
> 
> Did it ever meet this goal?

I do not know.


> I never quite understood how it works, being far from this field, so I'm happy
> to be enlighten on the topic.

Darkserver itself will just give you a rpm build for given build-id.  So with
a core file (containing the build-ids) and a small shell script you can
reconstruct system (chroot) where you can load the core file for more thorough
crash analysis.


> > And currently after my reassignment in Red Hat I currently do not have any
> > ABRT bugreports to investigate so I am currently not involved in this
> > darkserver/ABRT build-id crash reproducibility (I may be again later).
> 
> Would you know if anyone in your old team would be interested by this?

It is not specific to GDB team.  I am curious how all the Fedora package
developers deal with ABRT-filed Bugs where the backtrace contained therein is
not sufficient to understand the problem but one may be able to understand it
accessing more data with GDB using the core file.

The backtrace contained in the ABRT bugreport can rarely really debug the
problem.  Sure even the core file is not always sufficient but it gives more
chance to debug the crash (which requires an darkserver-like functionality).

But then I usually cannot get even just the core file from ABRT users so this
is why I ask how other package (*) maintainers deal with the ABRT bugreports.

(*) It may be confusing I am GDB package maintainer when GDB is used for the
crash analysis.  But here I talk about GDB as any other program (like bash,
firefox etc.) as even GDB crashes like any other program (bash, firefox, etc.).


> I'm trying to assess if there is actually an interest for it.
> You said you were interested by it but aren't working on this topic anymore, I
> don't think anybody else than you noticed it not working over the course of 
> last
> year.

Yes, because I think nobody knows one could troubleshoot ABRT-bugreported
problems better than with the information filed by ABRT into Bugzilla.


> So I wonder if we shouldn't just call it a day,

It won't change the current state of bugs troubleshooting.
But Fedora should not forget the Bugs troubleshooting could be improved.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Call for users of darkserver

[Jan Kratochvil]

On Fri, 16 Feb 2018 11:02:25 +0100, Pierre-Yves Chibon wrote:
> So what do you advice as course of action here, should we fix darkserver or 
> move
> its functionality to ABRT?

"move its functionality to ABRT"


> The later is tempting to me if it's just a few lines of code added on an app 
> we
> already maintain.

One problem of ABRT is that it cannot (or at least not easily) provide the
core file to Bug Assignee.  Users always fail to figure out how when I ask
them for that.  I do not know if it is just not user friendly enough or if the
core files gets deleted from disk too quickly.

And developers (that is me) could not use ABRT so far as it OOM-killed machine
on a first segfaulted program:
-fsanitize=address locks up abrt-hook-ccpp
https://bugzilla.redhat.com/show_bug.cgi?id=1164548
It is EOLed again but maybe it works now on F-27 after my quick local
re-check, I will check it later once I have access to an F-27 VM (travelling
now).

Then the ABRT bugreport does not contain build-ids of the shared libraries
involved, only build-ids of frames from a backtrace ("core_backtrace") which
is not sufficient to reproduce the same memory layout.

It seems to me interactive crash core file analysis is just not a goal of ABRT
(which is what was the goal of darkserver).

And currently after my reassignment in Red Hat I currently do not have any
ABRT bugreports to investigate so I am currently not involved in this
darkserver/ABRT build-id crash reproducibility (I may be again later).


Regards,
Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Call for users of darkserver

[Jan Kratochvil]

On Wed, 14 Feb 2018 11:02:52 +0100, Pierre-Yves Chibon wrote:
> If there is little interest for this project, we will likely decommission it 
> in
> the coming weeks (say end of March).

The darkserver project ws initiated AFAIK by me as there is always a problem
how to reproduce an ABRT bugreport.

The darkserver usability problems were related to each other,
a chicken-and-egg problem:
 * It never really started working.  When I tried to use it once upon few
   months when I found time to process some ABRT bugreports which were not
   obvious enough darkserver failed and after contacting Kushal Das
   (darkserver author) he found some new software or data bug why it did not
   work that time.
 * There was never a tool making it convenient enough to reconstruct the
   tree of files based on their build-ids.
 * There was never enough users (was there any besides me?) that started using
   darkserver, because of the two problems above.

So I believe darkserver would be great but not in its current state of
functionality.

Also I believe ABRT project already contains most of the infrastructure and
code required, I believe darkserver could be rather just few lines of code
added to the ABRT project - that is to interactively run the crashed program
with all matching versions of libraries - not just getting the non-interative
core file backtrace (which ABRT submits to Bugzilla).


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Call for users of darkserver

[Jan Kratochvil]

On Wed, 14 Feb 2018 15:55:20 +0100, Pierre-Yves Chibon wrote:
> darkserver itself doesn't store anything.

Darkserver was at least in the past using debuginfo storage from ABRT server
which is much more rich than what Koji server keeps.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide buildroot broken?

[Jan Kratochvil]

On Sun, 04 Feb 2018 22:14:47 +0100, Doug Ledford wrote:
> cc1: fatal error: inaccessible plugin file plugin/annobin.so expanded 
> from short plugin name annobin: No such file or directory
...
> So, where is the gcc plugin annobin?

In my fresh Rawhide buildroot
https://koji.fedoraproject.org/koji/buildrootinfo?buildrootID=11315754
I got
annobin-3.1-2.fc28.x86_64.rpm
which contains
/usr/lib/gcc/x86_64-redhat-linux/7/plugin/annobin.so.0.0.0
while latest annobin build
annobin-3.3-1.fc28
already contains
/usr/lib/gcc/x86_64-redhat-linux/8/plugin/annobin.so.0.0.0
which is needed for gcc v8 in the buildroot I got
gcc-8.0.1-0.9.fc28.x86_64.rpm
I am only curious why fresh Rawhide buildroots still have so old annobin when
annobin-3.3-1.fc28 has been built on Jan 30th - 5 days ago.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: gdb: No symbol table info available

[Jan Kratochvil]

On Sun, 10 Dec 2017 13:43:25 +0100, Richard Shaw wrote:
> I pretty much did the same thing with valgrind but the output of your
> script is much more concise and easier to read.

Fedora GDB should print the dnf debuginfo-install command.  When it does not?
It uses a similar script logic internally.

$ gdb ./core.24421 
...
Missing separate debuginfo for the main executable file
Try: dnf --enablerepo='*debug*' install 
/usr/lib/debug/.build-id/f9/9b1ecdd9f6408e76bfe14659e94efa1b79d949
Core was generated by `xclock'.

which installs xorg-x11-apps-debuginfo and then:

$ gdb ./core.24421 
...
Missing separate debuginfo for linux-vdso.so.1
Try: dnf --enablerepo='*debug*' install 
/usr/lib/debug/.build-id/4a/506b85d2f1ffcabd8ae0a5839480ea18930d6e
...
Missing separate debuginfos, use: dnf debuginfo-install 
libxkbfile-1.0.9-6.fc27.x86_64

for example (I have most of the debuginfos already installed, otherwise it
would say more of them).

Although now in F-27 after:
https://fedoraproject.org/wiki/Changes/SubpackageAndSourceDebuginfo

the debuginfo-install command installs only *-debuginfo and not the
*-debugsource counterpart which is then missing for GDB.  That is now a DNF
bug I guess.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org