Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Ed Maste]

On 11 February 2015 at 21:39, Glen Barber  wrote:
> Hi,
>
> Within the next 24 hours, I will merge the release-install-debug branch
> into head, which will enable building and installing stripped debugging
> files by default.
>
> In general, this should have no significant impact, but any fallout will
> be addressed as soon as possible after the merge.
>
> Those that do not want debugging files built/installed by default should
> add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
> UPDATING.

I want to pick this change up again. There are a few separate parts
under discussion:

1. Release build changes to support standalone debug
2. Installer changes
3. Switching debug files on by default
4. Separating into two or more debug dist sets

Glen's proposed merge from earlier this year included #1 through #3,
but was held up based on discussion in this thread. I propose that we
merge #1 and #2 now, since they're effectively a no-op if debug files
aren't enabled, but facilitate further testing.

Separating the debug dist sets (into runtime and binaries, or even
more fine-grained) requires coordination with the packaged base work,
and I suggest that we hold off on that project for now.

I'd like to flip the default shortly after and continue refining it
with it enabled - we need to start finding and fix issues now in order
to have it ready for 11.0. I have a review open for the default
switch, in https://reviews.freebsd.org/D4018.

In my test just now on a powerful server the cpu (user+system) time
increased from 15260s to 15760s. Wall clock time increased by under
40s (20:08.83 to 20:47.49). The objdir was 2.4GB without debug and
4.3GB with.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[John Baldwin]

On Tuesday, October 27, 2015 06:09:37 PM Ed Maste wrote:
> On 11 February 2015 at 21:39, Glen Barber  wrote:
> > Hi,
> >
> > Within the next 24 hours, I will merge the release-install-debug branch
> > into head, which will enable building and installing stripped debugging
> > files by default.
> >
> > In general, this should have no significant impact, but any fallout will
> > be addressed as soon as possible after the merge.
> >
> > Those that do not want debugging files built/installed by default should
> > add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
> > UPDATING.
> 
> I want to pick this change up again. There are a few separate parts
> under discussion:
> 
> 1. Release build changes to support standalone debug
> 2. Installer changes
> 3. Switching debug files on by default
> 4. Separating into two or more debug dist sets
> 
> Glen's proposed merge from earlier this year included #1 through #3,
> but was held up based on discussion in this thread. I propose that we
> merge #1 and #2 now, since they're effectively a no-op if debug files
> aren't enabled, but facilitate further testing.

Sounds good.

> Separating the debug dist sets (into runtime and binaries, or even
> more fine-grained) requires coordination with the packaged base work,
> and I suggest that we hold off on that project for now.

Agreed.

> I'd like to flip the default shortly after and continue refining it
> with it enabled - we need to start finding and fix issues now in order
> to have it ready for 11.0. I have a review open for the default
> switch, in https://reviews.freebsd.org/D4018.

All ahead full!

I do think that packaged base is going to force us to revisit things
quite a bit in that you won't necessarily just install the equivalent
of base.txz all the time.  Long term I do think that is the solution
for folks wanting more fine-grained control over disk space as well
as the ability to "fault in" symbols on demand.

For whatever we are going to do for the graybeards like myself that
will still use make, I think that the only knob you would need for 4
is runtime vs everything else where runtime is libs + rtld.  Anything
finer-grained than that should be handled by pkg(8) instead.  For my
part I already set WITH_DEBUG_FILES=yes in /etc/src.conf on just about
all of my boxes and VMs (head or stable).

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Ed Maste]

On 12 February 2015 at 10:29, Steve Kargl
s...@troutmask.apl.washington.edu wrote:

 Does this feature take into account the optimization level
 that one may set via CFLAGS?  Conventional wisdom used to
 claim that debuggers did not fair too well with code that
 was compiled with -O2 (or -Os and/or other well-known gcc
 options -fomit-frame-pointer, etc.)  The common advice was
 to recompile one's code with -g or '-g -O'.  Does lldb cope
 better with optimized binaries?

The WITH_DEBUG_FILES option builds debug regardless of what
optimization level is set. I'm not quite sure in which way
WITH_DEBUG_FILES would take account of the optimization level.

It's true that debugging is easier if the target is built with -O0.
If I have a reproducible crash as a developer, I would rebuild with
-O0. That may not be feasible for end users though, and doesn't help
in the case of a crash that can't be reproduced.

That said, debugging is still possible with -O2, it just loses
information or makes control flow hard to follow in some (perhaps
many) cases. Type and source line information is still available.  It
may not be possible to read the value of some variables at some times
during execution, and stepping can be confusing. But it's usually
possible to get a useful backtrace and find necessary variables.

All of my systems are built with -O2 and WITH_DEBUG_FILES, and
generally speaking I have a positive experience with postmortem
debugging of core files.

LLDB does handle optimized binaries better than the ancient version of
GDB we have in the base system. Contemporary GDB versions also do so.

Also note that for userland DTrace support we build with -g as part of
the process of generating CTF information, so enabling userland DTrace
also implies an increase in OBJDIR size.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Andreas Nilsson]

On Thu, Feb 12, 2015 at 10:12 AM, Konstantin Belousov kostik...@gmail.com
wrote:

 On Thu, Feb 12, 2015 at 02:39:12AM +, Glen Barber wrote:
  Hi,
 
  Within the next 24 hours, I will merge the release-install-debug branch
  into head, which will enable building and installing stripped debugging
  files by default.
 
  In general, this should have no significant impact, but any fallout will
  be addressed as soon as possible after the merge.
 
  Those that do not want debugging files built/installed by default should
  add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
  UPDATING.

 Is there a knob to generate and install debug symbols for the libraries
 and rtld only ?  I remember that there was some.

 IMO having debugging information for all binaries is just an overkill.
 Nobody debugs ls on regular basis, while debug information for C runtime
 is needed often.


I like the idea of having the debug files available.

How would this work with regards to freebsd-update? Would the debug files
earn a separate component slot?

Best regards
Andeas
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[David Chisnall]

On 12 Feb 2015, at 04:49, Glen Barber g...@freebsd.org wrote:
 
 I would like to see it enabled by default for 11.0-RELEASE.

For everyone concerned about size, remember that the goal for 11.0-RELEASE is 
to use pkg for the base system, which makes it much easier to:

- Install some debug info later, on demand.
- Split debug info into separate packages so only the relevant bits can be 
installed when debugging a particular problem.

I probably don't want debug info for everything to be installed on my systems 
by default and upgraded all of the time, but I really do want to be able to 
just run a single pkg ins command to get the symbols for whatever it is that 
I'm experiencing problems with.

David

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Konstantin Belousov]

On Thu, Feb 12, 2015 at 02:39:12AM +, Glen Barber wrote:
 Hi,
 
 Within the next 24 hours, I will merge the release-install-debug branch
 into head, which will enable building and installing stripped debugging
 files by default.
 
 In general, this should have no significant impact, but any fallout will
 be addressed as soon as possible after the merge.
 
 Those that do not want debugging files built/installed by default should
 add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
 UPDATING.

Is there a knob to generate and install debug symbols for the libraries
and rtld only ?  I remember that there was some.

IMO having debugging information for all binaries is just an overkill.
Nobody debugs ls on regular basis, while debug information for C runtime
is needed often.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Ed Maste]

On 11 February 2015 at 23:43, Alfred Perlstein alf...@freebsd.org wrote:

 Agreed, SMALL platforms should turn off at decision of those platform
 maintainers.  Won't fight to keep debug default on for arm, that's for sure.

Actually, it's even more important that we build and store debug files
for ARM releases. Consider a user running FreeBSD on a Raspberry Pi,
using an image they downloaded and dd'd onto an SD card. The advice
rebuild with debug data and try to reproduce the issue is more
absurd there than on a big amd64 machine.

All of this concerns what is on the installation media and in the
release package sets, though, and not necessarily the default setting
of the knob. It is technically possible for the release builds to
enable debug files but leave the default off in the tree, although
there are disadvantages of doing so.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Ed Maste]

On 12 February 2015 at 08:21, Glen Barber g...@freebsd.org wrote:
 On Thu, Feb 12, 2015 at 11:12:19AM +0200, Konstantin Belousov wrote:

 IMO having debugging information for all binaries is just an overkill.
 Nobody debugs ls on regular basis, while debug information for C runtime
 is needed often.


 I thought there was one for libraries and rtld only, but do not see any
 trace of it now.

No, there was never a separate knob.  I first added standalone debug
support to bsd.lib.mk, so there was a time that WITH_DEBUG_FILES=yes
provided .debug files for libraries but not binaries.  Note that rtld
uses bsd.prog.mk and so it initially did not have debug data.  We
could introduce knobs like WITH_LIBRARY_DEBUG_FILES and
WITH_BINARY_DEBUG_FILES for bsd.lib.mk / bsd.prog.mk,  but we'd need
an additional special case to build rtld debug to make a runtime
debug package.

The primary goal of this work is building debug data for releases.
The debug files will not be included on the install media, but will be
available for installation afterwards in order to debug a crash.  For
that packaging we want a 1:1 relationship between base system packages
and -dbg packages.  We can work backwards from there to help decide
what the knobs should be.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Glen Barber]

On Thu, Feb 12, 2015 at 11:12:19AM +0200, Konstantin Belousov wrote:
 On Thu, Feb 12, 2015 at 02:39:12AM +, Glen Barber wrote:
  Hi,
  
  Within the next 24 hours, I will merge the release-install-debug branch
  into head, which will enable building and installing stripped debugging
  files by default.
  
  In general, this should have no significant impact, but any fallout will
  be addressed as soon as possible after the merge.
  
  Those that do not want debugging files built/installed by default should
  add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
  UPDATING.
 
 Is there a knob to generate and install debug symbols for the libraries
 and rtld only ?  I remember that there was some.
 
 IMO having debugging information for all binaries is just an overkill.
 Nobody debugs ls on regular basis, while debug information for C runtime
 is needed often.
 

I thought there was one for libraries and rtld only, but do not see any
trace of it now.

Perhaps Ed can provide more information on this.

Glen



pgpPwz28hopzr.pgp
Description: PGP signature

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Glen Barber]

On Thu, Feb 12, 2015 at 09:27:53AM -0500, Ed Maste wrote:
 On 11 February 2015 at 23:43, Alfred Perlstein alf...@freebsd.org wrote:
 
  Agreed, SMALL platforms should turn off at decision of those platform
  maintainers.  Won't fight to keep debug default on for arm, that's for sure.
 
 Actually, it's even more important that we build and store debug files
 for ARM releases. Consider a user running FreeBSD on a Raspberry Pi,
 using an image they downloaded and dd'd onto an SD card. The advice
 rebuild with debug data and try to reproduce the issue is more
 absurd there than on a big amd64 machine.
 
 All of this concerns what is on the installation media and in the
 release package sets, though, and not necessarily the default setting
 of the knob. It is technically possible for the release builds to
 enable debug files but leave the default off in the tree, although
 there are disadvantages of doing so.
 

We cannot set non-default options for release builds, because we lose
reproducibility in the by default sense.

Glen



pgpTJ8x2GJg_o.pgp
Description: PGP signature

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Ed Maste]

On 11 February 2015 at 23:23, Ian Lepore i...@freebsd.org wrote:

 I can't imagine that anybody is going to be happy with an installed
 system size increase from 520 to 2520 MB no matter how much it helps
 debugging, especially considering the the typical installation media is
 in the 2-8 GB range (with lots of 2 and 4 GB cards out there because
 that's what vendors bundle with the boards).

This change will have almost(*) no effect on the size of a default
installation from FreeBSD releases, or the size of those releases (CD
or memstick images). The plan is that the debug files will be
available as package sets that can be installed on demand, but in
order to do so the debug data needs to be built and stored at the time
the release is built.

I say almost no effect because the installed binaries grow by a
trivial amount with the addition of a .gnu_debuglink section and note.
For illustration here are the file sizes of ls, files in the objdir
unless otherwise noted:

# no debug
ls44465
ls (installed)31920

# with debug
ls.full   89401
ls.debug  60145
ls40344
ls (installed)32016
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Steve Kargl]

On Thu, Feb 12, 2015 at 02:39:12AM +, Glen Barber wrote:
 Hi,
 
 Within the next 24 hours, I will merge the release-install-debug branch
 into head, which will enable building and installing stripped debugging
 files by default.
 
 In general, this should have no significant impact, but any fallout will
 be addressed as soon as possible after the merge.
 
 Those that do not want debugging files built/installed by default should
 add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
 UPDATING.
 

Does this feature take into account the optimization level
that one may set via CFLAGS?  Conventional wisdom used to 
claim that debuggers did not fair too well with code that
was compiled with -O2 (or -Os and/or other well-known gcc
options -fomit-frame-pointer, etc.)  The common advice was
to recompile one's code with -g or '-g -O'.  Does lldb cope
better with optimized binaries?

-- 
Steve
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Glen Barber]

On Thu, Feb 12, 2015 at 02:39:12AM +, Glen Barber wrote:
 Within the next 24 hours, I will merge the release-install-debug branch
 into head, which will enable building and installing stripped debugging
 files by default.
 

Given the feedback received, I will not be merging this branch to head
today, pending additional changes.

Glen



pgpAPdY4TvTLn.pgp
Description: PGP signature

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Glen Barber]

On Wed, Feb 11, 2015 at 08:56:00PM -0700, Ian Lepore wrote:
 How much time will this add to a build?
 

I did not keep the build log files from the initial universe build with
DEBUG_FILES enabled, so cannot yet provide a granular overview of the
build times at the moment unfortunately.  I will provide this next time
a heads-up about this merge is sent, since I will not be merging this
branch today.

This is the comparison of the resulting userland from 'installworld'.
Note, this does not take into account 'installkernel', since there is no
change there, that I can tell.

   Target  MK_DEBUG_FILES=yes  MK_DEBUG_FILES=no
   amd64/amd64:  400M257M
   arm/arm:  317M206M
   arm/armeb:262M185M
   arm/armv6:316M205M
   arm/armv6hf:  316M205M
   i386/i386:325M208M
   mips/mips:276M195M
   mips/mips64:  282M200M
   mips/mips64el:280M198M
   mips/mipsel:  277M195M
   mips/mipsn32: 273M194M
   pc98/i386:321M207M
   powerpc/powerpc:  285M198M
   powerpc/powerpc64:362M250M
   sparc64/sparc64:  277M195M

Glen



pgpUgchL9nHZ0.pgp
Description: PGP signature

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Glen Barber]

On Thu, Feb 12, 2015 at 04:30:58PM +, Glen Barber wrote:
 On Wed, Feb 11, 2015 at 08:56:00PM -0700, Ian Lepore wrote:
  How much time will this add to a build?
  
 
 I did not keep the build log files from the initial universe build with
 DEBUG_FILES enabled, so cannot yet provide a granular overview of the
 build times at the moment unfortunately.  I will provide this next time
 a heads-up about this merge is sent, since I will not be merging this
 branch today.
 
 This is the comparison of the resulting userland from 'installworld'.
 Note, this does not take into account 'installkernel', since there is no
 change there, that I can tell.
 

Sigh.  The previous numbers were wrong, because I used 'du -hs' instead
of 'du -hsA' on a lz4-compressed ZFS dataset.  The corrected numbers
are:

 Target  MK_DEBUG_FILES=yes  MK_DEBUG_FILES=no
 amd64/amd64:   647M   374M
 arm/arm:   465M   270M
 arm/armeb: 351M   232M
 arm/armv6: 463M   266M
 arm/armv6hf:   462M   266M
 i386/i386: 477M   274M
 mips/mips: 414M   279M
 mips/mips64:   458M   307M
 mips/mips64el: 458M   307M
 mips/mipsel:   414M   279M
 mips/mipsn32:  406M   276M
 pc98/i386: 470M   272M
 powerpc/powerpc:   386M   252M
 powerpc/powerpc64: 566M   368M
 sparc64/sparc64:   409M   262M

Glen



pgpb3dqE8RqPW.pgp
Description: PGP signature

HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Glen Barber]

Hi,

Within the next 24 hours, I will merge the release-install-debug branch
into head, which will enable building and installing stripped debugging
files by default.

In general, this should have no significant impact, but any fallout will
be addressed as soon as possible after the merge.

Those that do not want debugging files built/installed by default should
add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
UPDATING.

This work was sponsored by The FreeBSD Foundation.

Love FreeBSD?  You can help support The FreeBSD Project with a donation
to The FreeBSD Foundation!  https://www.freebsdfoundation.org/donate/

Glen



pgpRrd4UjjD6u.pgp
Description: PGP signature

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Ed Maste]

On 11 February 2015 at 21:39, Glen Barber g...@freebsd.org wrote:

 Within the next 24 hours, I will merge the release-install-debug branch
 into head, which will enable building and installing stripped debugging
 files by default.

 In general, this should have no significant impact, but any fallout will
 be addressed as soon as possible after the merge.

 Those that do not want debugging files built/installed by default should
 add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
 UPDATING.

Note that the debug files do consume a reasonably large amount of disk
space in both the OBJDIR and in the installed location under
/usr/lib/debug. Users with limited disk space will probably want to
disable them.  As an example, the installed debug data on my laptop is
about 2GB.

-Ed
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Jia-Shiun Li]

On Thu, Feb 12, 2015 at 12:11 PM, Glen Barber g...@freebsd.org wrote:

 The major benefit is that all debugging data that we need to properly
 debug application crashes in the base system will be available
 out-of-box.

 There is a trade-off here, in both directions.  For arm, for example,
 the trade-off is that the default installed userland would grow, however
 when there is a PR regarding an application crash, the tools to diagnose
 the issue are there by default (we do not need to ask that the utility
 is rebuilt with debugging options enabled, and then recreate the crash).

 I considered making this an opt-in thing for arm, but given the above
 rationale, thought it would be more beneficial for the opposite route.
 If you feel necessary, however, we can turn this off by default for now
 for arm.


Is this default value supposed to go to future releases or only kept in
 -current for development  debugging purpose?

For releases and resource-critical platforms, it'd be nice to build them all
at once but only populate when needed after installation.

-Jia-shiun.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Glen Barber]

On Thu, Feb 12, 2015 at 04:11:58AM +, Glen Barber wrote:
 On Wed, Feb 11, 2015 at 08:56:00PM -0700, Ian Lepore wrote:
  On Wed, 2015-02-11 at 22:21 -0500, Ed Maste wrote:
   On 11 February 2015 at 21:39, Glen Barber g...@freebsd.org wrote:
   
Within the next 24 hours, I will merge the release-install-debug branch
into head, which will enable building and installing stripped debugging
files by default.
   
In general, this should have no significant impact, but any fallout will
be addressed as soon as possible after the merge.
   
Those that do not want debugging files built/installed by default should
add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
UPDATING.
   
   Note that the debug files do consume a reasonably large amount of disk
   space in both the OBJDIR and in the installed location under
   /usr/lib/debug. Users with limited disk space will probably want to
   disable them.  As an example, the installed debug data on my laptop is
   about 2GB.
  
  Seriously?  2GB is bigger than the entire filesystem on many ARM boards
  that do useful work.  Not to mention how long it will take to write all
  that to an sdcard (it already takes a long time to installworld/kernel
  to an sdcard and it's only 400MB).
  
  Just what are these files, and what use will the average user make of
  them?
  
  What use will I make of them, that is going to justify that every one of
  my couple-dozen build sandboxes will now be 4gb larger (a copy in obj
  and a copy in the nfs root that things install to)?
  
  How much time will this add to a build?
  
  Yeah yeah, I can update a couple dozen src.conf files to eliminate them,
  and that's not the biggest hassle in the world (but it's also not
  nothing).  It seems like this is a heavy enough load that it needs to
  justify its existance.
  
 
 The major benefit is that all debugging data that we need to properly
 debug application crashes in the base system will be available
 out-of-box.
 
 There is a trade-off here, in both directions.  For arm, for example,
 the trade-off is that the default installed userland would grow, however
 when there is a PR regarding an application crash, the tools to diagnose
 the issue are there by default (we do not need to ask that the utility
 is rebuilt with debugging options enabled, and then recreate the crash).
 
 I considered making this an opt-in thing for arm, but given the above
 rationale, thought it would be more beneficial for the opposite route.
 If you feel necessary, however, we can turn this off by default for now
 for arm.
 

One thing to note in addition, is that the 2GB increase is for amd64.

I'm running a universe build with WITHOUT_DEBUG_FILES=1 now, and will
report back with a more accurate breakdown of what can be expected with
regard to filesystem expansion with this change.

Glen



pgpBNWEZDGTey.pgp
Description: PGP signature

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Glen Barber]

On Thu, Feb 12, 2015 at 12:46:18PM +0800, Jia-Shiun Li wrote:
 On Thu, Feb 12, 2015 at 12:11 PM, Glen Barber g...@freebsd.org wrote:
 
  The major benefit is that all debugging data that we need to properly
  debug application crashes in the base system will be available
  out-of-box.
 
  There is a trade-off here, in both directions.  For arm, for example,
  the trade-off is that the default installed userland would grow, however
  when there is a PR regarding an application crash, the tools to diagnose
  the issue are there by default (we do not need to ask that the utility
  is rebuilt with debugging options enabled, and then recreate the crash).
 
  I considered making this an opt-in thing for arm, but given the above
  rationale, thought it would be more beneficial for the opposite route.
  If you feel necessary, however, we can turn this off by default for now
  for arm.
 
 
 Is this default value supposed to go to future releases or only kept in
  -current for development  debugging purpose?
 

I would like to see it enabled by default for 11.0-RELEASE.

 For releases and resource-critical platforms, it'd be nice to build them all
 at once but only populate when needed after installation.
 

Yes, this would be the case for those tracking the releng/ branch in
SVN.

Glen



pgpFEkn4gxo4e.pgp
Description: PGP signature

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Glen Barber]

On Wed, Feb 11, 2015 at 08:56:00PM -0700, Ian Lepore wrote:
 On Wed, 2015-02-11 at 22:21 -0500, Ed Maste wrote:
  On 11 February 2015 at 21:39, Glen Barber g...@freebsd.org wrote:
  
   Within the next 24 hours, I will merge the release-install-debug branch
   into head, which will enable building and installing stripped debugging
   files by default.
  
   In general, this should have no significant impact, but any fallout will
   be addressed as soon as possible after the merge.
  
   Those that do not want debugging files built/installed by default should
   add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
   UPDATING.
  
  Note that the debug files do consume a reasonably large amount of disk
  space in both the OBJDIR and in the installed location under
  /usr/lib/debug. Users with limited disk space will probably want to
  disable them.  As an example, the installed debug data on my laptop is
  about 2GB.
 
 Seriously?  2GB is bigger than the entire filesystem on many ARM boards
 that do useful work.  Not to mention how long it will take to write all
 that to an sdcard (it already takes a long time to installworld/kernel
 to an sdcard and it's only 400MB).
 
 Just what are these files, and what use will the average user make of
 them?
 
 What use will I make of them, that is going to justify that every one of
 my couple-dozen build sandboxes will now be 4gb larger (a copy in obj
 and a copy in the nfs root that things install to)?
 
 How much time will this add to a build?
 
 Yeah yeah, I can update a couple dozen src.conf files to eliminate them,
 and that's not the biggest hassle in the world (but it's also not
 nothing).  It seems like this is a heavy enough load that it needs to
 justify its existance.
 

The major benefit is that all debugging data that we need to properly
debug application crashes in the base system will be available
out-of-box.

There is a trade-off here, in both directions.  For arm, for example,
the trade-off is that the default installed userland would grow, however
when there is a PR regarding an application crash, the tools to diagnose
the issue are there by default (we do not need to ask that the utility
is rebuilt with debugging options enabled, and then recreate the crash).

I considered making this an opt-in thing for arm, but given the above
rationale, thought it would be more beneficial for the opposite route.
If you feel necessary, however, we can turn this off by default for now
for arm.

Glen



pgpt9httNnHjl.pgp
Description: PGP signature

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Ian Lepore]

On Thu, 2015-02-12 at 04:11 +, Glen Barber wrote:
 On Wed, Feb 11, 2015 at 08:56:00PM -0700, Ian Lepore wrote:
  On Wed, 2015-02-11 at 22:21 -0500, Ed Maste wrote:
   On 11 February 2015 at 21:39, Glen Barber g...@freebsd.org wrote:
   
Within the next 24 hours, I will merge the release-install-debug branch
into head, which will enable building and installing stripped debugging
files by default.
   
In general, this should have no significant impact, but any fallout will
be addressed as soon as possible after the merge.
   
Those that do not want debugging files built/installed by default should
add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
UPDATING.
   
   Note that the debug files do consume a reasonably large amount of disk
   space in both the OBJDIR and in the installed location under
   /usr/lib/debug. Users with limited disk space will probably want to
   disable them.  As an example, the installed debug data on my laptop is
   about 2GB.
  
  Seriously?  2GB is bigger than the entire filesystem on many ARM boards
  that do useful work.  Not to mention how long it will take to write all
  that to an sdcard (it already takes a long time to installworld/kernel
  to an sdcard and it's only 400MB).
  
  Just what are these files, and what use will the average user make of
  them?
  
  What use will I make of them, that is going to justify that every one of
  my couple-dozen build sandboxes will now be 4gb larger (a copy in obj
  and a copy in the nfs root that things install to)?
  
  How much time will this add to a build?
  
  Yeah yeah, I can update a couple dozen src.conf files to eliminate them,
  and that's not the biggest hassle in the world (but it's also not
  nothing).  It seems like this is a heavy enough load that it needs to
  justify its existance.
  
 
 The major benefit is that all debugging data that we need to properly
 debug application crashes in the base system will be available
 out-of-box.
 
 There is a trade-off here, in both directions.  For arm, for example,
 the trade-off is that the default installed userland would grow, however
 when there is a PR regarding an application crash, the tools to diagnose
 the issue are there by default (we do not need to ask that the utility
 is rebuilt with debugging options enabled, and then recreate the crash).
 
 I considered making this an opt-in thing for arm, but given the above
 rationale, thought it would be more beneficial for the opposite route.
 If you feel necessary, however, we can turn this off by default for now
 for arm.
 
 Glen
 

I can't imagine that anybody is going to be happy with an installed
system size increase from 520 to 2520 MB no matter how much it helps
debugging, especially considering the the typical installation media is
in the 2-8 GB range (with lots of 2 and 4 GB cards out there because
that's what vendors bundle with the boards).

-- Ian


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Ian Lepore]

On Wed, 2015-02-11 at 22:21 -0500, Ed Maste wrote:
 On 11 February 2015 at 21:39, Glen Barber g...@freebsd.org wrote:
 
  Within the next 24 hours, I will merge the release-install-debug branch
  into head, which will enable building and installing stripped debugging
  files by default.
 
  In general, this should have no significant impact, but any fallout will
  be addressed as soon as possible after the merge.
 
  Those that do not want debugging files built/installed by default should
  add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
  UPDATING.
 
 Note that the debug files do consume a reasonably large amount of disk
 space in both the OBJDIR and in the installed location under
 /usr/lib/debug. Users with limited disk space will probably want to
 disable them.  As an example, the installed debug data on my laptop is
 about 2GB.

Seriously?  2GB is bigger than the entire filesystem on many ARM boards
that do useful work.  Not to mention how long it will take to write all
that to an sdcard (it already takes a long time to installworld/kernel
to an sdcard and it's only 400MB).

Just what are these files, and what use will the average user make of
them?

What use will I make of them, that is going to justify that every one of
my couple-dozen build sandboxes will now be 4gb larger (a copy in obj
and a copy in the nfs root that things install to)?

How much time will this add to a build?

Yeah yeah, I can update a couple dozen src.conf files to eliminate them,
and that's not the biggest hassle in the world (but it's also not
nothing).  It seems like this is a heavy enough load that it needs to
justify its existance.

-- Ian


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Glen Barber]

On Wed, Feb 11, 2015 at 09:23:01PM -0700, Ian Lepore wrote:
 On Thu, 2015-02-12 at 04:11 +, Glen Barber wrote:
  On Wed, Feb 11, 2015 at 08:56:00PM -0700, Ian Lepore wrote:
   On Wed, 2015-02-11 at 22:21 -0500, Ed Maste wrote:
On 11 February 2015 at 21:39, Glen Barber g...@freebsd.org wrote:

 Within the next 24 hours, I will merge the release-install-debug 
 branch
 into head, which will enable building and installing stripped 
 debugging
 files by default.

 In general, this should have no significant impact, but any fallout 
 will
 be addressed as soon as possible after the merge.

 Those that do not want debugging files built/installed by default 
 should
 add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted 
 in
 UPDATING.

Note that the debug files do consume a reasonably large amount of disk
space in both the OBJDIR and in the installed location under
/usr/lib/debug. Users with limited disk space will probably want to
disable them.  As an example, the installed debug data on my laptop is
about 2GB.
   
   Seriously?  2GB is bigger than the entire filesystem on many ARM boards
   that do useful work.  Not to mention how long it will take to write all
   that to an sdcard (it already takes a long time to installworld/kernel
   to an sdcard and it's only 400MB).
   
   Just what are these files, and what use will the average user make of
   them?
   
   What use will I make of them, that is going to justify that every one of
   my couple-dozen build sandboxes will now be 4gb larger (a copy in obj
   and a copy in the nfs root that things install to)?
   
   How much time will this add to a build?
   
   Yeah yeah, I can update a couple dozen src.conf files to eliminate them,
   and that's not the biggest hassle in the world (but it's also not
   nothing).  It seems like this is a heavy enough load that it needs to
   justify its existance.
   
  
  The major benefit is that all debugging data that we need to properly
  debug application crashes in the base system will be available
  out-of-box.
  
  There is a trade-off here, in both directions.  For arm, for example,
  the trade-off is that the default installed userland would grow, however
  when there is a PR regarding an application crash, the tools to diagnose
  the issue are there by default (we do not need to ask that the utility
  is rebuilt with debugging options enabled, and then recreate the crash).
  
  I considered making this an opt-in thing for arm, but given the above
  rationale, thought it would be more beneficial for the opposite route.
  If you feel necessary, however, we can turn this off by default for now
  for arm.
  
  Glen
  
 
 I can't imagine that anybody is going to be happy with an installed
 system size increase from 520 to 2520 MB no matter how much it helps
 debugging, especially considering the the typical installation media is
 in the 2-8 GB range (with lots of 2 and 4 GB cards out there because
 that's what vendors bundle with the boards).
 

Absolutely understood.  As mentioned in a recent reply (before seeing
this email), I'll provide a closer estimate of what to expect soon.
Again, if you want this turned off for arm, that's fine.  I would,
however, like to see it enabled by default across the board eventually
(even for -RELEASE builds).

Glen



pgpmuUD4E36_z.pgp
Description: PGP signature

Re: HEADS-UP: Enabling WITH_DEBUG_FILES by default

[Alfred Perlstein]

On 2/11/15 8:28 PM, Glen Barber wrote:

On Wed, Feb 11, 2015 at 09:23:01PM -0700, Ian Lepore wrote:

On Thu, 2015-02-12 at 04:11 +, Glen Barber wrote:

On Wed, Feb 11, 2015 at 08:56:00PM -0700, Ian Lepore wrote:

On Wed, 2015-02-11 at 22:21 -0500, Ed Maste wrote:

On 11 February 2015 at 21:39, Glen Barber g...@freebsd.org wrote:


Within the next 24 hours, I will merge the release-install-debug branch
into head, which will enable building and installing stripped debugging
files by default.

In general, this should have no significant impact, but any fallout will
be addressed as soon as possible after the merge.

Those that do not want debugging files built/installed by default should
add 'WITHOUT_DEBUG_FILES=1' to src.conf(5).  This will also be noted in
UPDATING.


Note that the debug files do consume a reasonably large amount of disk
space in both the OBJDIR and in the installed location under
/usr/lib/debug. Users with limited disk space will probably want to
disable them.  As an example, the installed debug data on my laptop is
about 2GB.


Seriously?  2GB is bigger than the entire filesystem on many ARM boards
that do useful work.  Not to mention how long it will take to write all
that to an sdcard (it already takes a long time to installworld/kernel
to an sdcard and it's only 400MB).

Just what are these files, and what use will the average user make of
them?

What use will I make of them, that is going to justify that every one of
my couple-dozen build sandboxes will now be 4gb larger (a copy in obj
and a copy in the nfs root that things install to)?

How much time will this add to a build?

Yeah yeah, I can update a couple dozen src.conf files to eliminate them,
and that's not the biggest hassle in the world (but it's also not
nothing).  It seems like this is a heavy enough load that it needs to
justify its existance.



The major benefit is that all debugging data that we need to properly
debug application crashes in the base system will be available
out-of-box.

There is a trade-off here, in both directions.  For arm, for example,
the trade-off is that the default installed userland would grow, however
when there is a PR regarding an application crash, the tools to diagnose
the issue are there by default (we do not need to ask that the utility
is rebuilt with debugging options enabled, and then recreate the crash).

I considered making this an opt-in thing for arm, but given the above
rationale, thought it would be more beneficial for the opposite route.
If you feel necessary, however, we can turn this off by default for now
for arm.

Glen



I can't imagine that anybody is going to be happy with an installed
system size increase from 520 to 2520 MB no matter how much it helps
debugging, especially considering the the typical installation media is
in the 2-8 GB range (with lots of 2 and 4 GB cards out there because
that's what vendors bundle with the boards).



Absolutely understood.  As mentioned in a recent reply (before seeing
this email), I'll provide a closer estimate of what to expect soon.
Again, if you want this turned off for arm, that's fine.  I would,
however, like to see it enabled by default across the board eventually
(even for -RELEASE builds).




YES

Thanks Glen, this is a great step forward for Intel platform.

Agreed, SMALL platforms should turn off at decision of those platform 
maintainers.  Won't fight to keep debug default on for arm, that's for sure.


Our Arm stuff runs Linux anyhow. :(

-Alfred
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org