> On Oct. 14, 2014, 9:06 p.m., Timothy St. Clair wrote:
> > configure.ac, line 281
> > <https://reviews.apache.org/r/26426/diff/1/?file=714874#file714874line281>
> >
> > Is there a reason you want to leave debug symbols out of optimized
> > builds?
> >
> > cmake has the pattern correct imho:
> > Release
> > Debug
> > ReleaseWithDebug
> >
> > A ReleaseWithDebug allows packagers, such as myself, to build
> > w/debugsymbols that are stripped out into a .debuginfo package which can be
> > used by developers for tracing "When bears attack". Granted that it is
> > tenuous debugging at best, but it's better then nothing.
> >
> > So I think we want all three modes, stripping all debug information is
> > not really idea.
>
> Cody Maloney wrote:
> My main motivation is to shrink the size of libmesos. Yesterday sugis in
> #mesos had one which was 213M. For the buildbot internally, full debian
> packages (which are compressed) of mesos weign in at 165M a piece (Yes,
> stripping post-build would help a lot, but why build it to begin with?). Most
> of this is debug info. Also, we build a bunch of different ways, and when
> libmesos is as big as it is, a decent amount of time ends up being spent on
> disk I/O reading / writing all the debug info when we are really just trying
> to ensure it builds on all the different platforms (Not to mention storage
> and file size shipping things around the network to centralized repositories).
>
> The simple toggle between debug and release, removing the legacy logic
> gets us most of the benefit.
>
> If you have a good place to point me for what is needed to get a
> 'ReleaseWithDebug' info build up and running I can definitely work on adding
> that as well.
>
> Cody Maloney wrote:
> It is also entirely possible to specify custom CXXFLAGS + CFLAGS with
> this patch to get the old "optimized debug" build. The flags set by
> --enable-debug (or not having it), to get back the "optimized debug" build
> from before.
>
> Timothy St. Clair wrote:
> I suppose rpm builds default CXXFLAGS to '-O2 -g ...', so I can buy the
> argument of just making it easier on the average person.
>
> Ben Mahler wrote:
> Re-opening because conflating debug and optimization under a single flag
> seems a bit confusing to me.
>
> In practice, we run mesos **with optimizations** (for free performance
> win), and **with debugging symbols** (for meaningful backtraces and core
> dumps), which was previously the default but now requires the setting of
> CXXFLAGS to obtain. :(
>
> For development, I think by default we need debugging symbols turned on.
> Otherwise most people will forget to configure with `--enable-debug` and
> consequently will need to recompile everything when they encounter a
> backtrace or need to debug. CI jobs will encounter this issue as well.
>
> If there are use cases that merit no debugging symbols, seems nice to
> make that explicit given it runs the risk of making debugging difficult.
>
> Thoughts?
>
> Cody Maloney wrote:
> Note that the two were conflated under one flag previously. This just
> changes how we do it to be a little more standard way of conflating them.
> Generally people talk about `Debug` or `Relase` builds. Not the specific
> compiler flags that get switched. The general switch is -O2 -> -O0 + -g and
> back (Possibly with a couple extra warning flags). Autotools projects which
> implement a debug vs. release build almost always do '--enable-debug' / the
> default is 'optimized' build.
>
> https://gcc.gnu.org/wiki/DebugFission - that artical has some good
> explanations why debug info gets problematic as programs get larger and
> larger, which is only going to get worse over time. Already mesos debug info
> is over 100MB. As the codebase grows, this will get worse than it already is,
> increasing the minimum machine specs you need to develop / compile Mesos.
>
> Compiler implementors generally give that '-O2' should be used to
> generate an optimal binary. '-g' should be used when you are planning to
> attach GDB and step through a program line by line. General development
> generally people want nice backtraces, which is inbetween those. See
> **Backtraces**. The section **Coredumps** talks about how we can give all the
> info in coredumps without running full debug builds ('-g') everywhere.
>
> **Backtraces** should generally be reasonably meaningful as long as you
> don't strip the binary. Most of the symbols are still there (You just lose
> inlined things). You don't get file names and line numbers, but all the
> functions in mesos should be uniquely named (C++'s ODR rule), so you get out
> a mangled C++ name which you can demangle with `c++filt` (Shipped with GCC)
> and you will get the full namespace prefixed name of the C++ function. Find
> the function with your favorite text editor search, life is happy.
>
> Pretty much every binary which is shipped on your machine via a
> distribution is sent like this. The backtraces + coredumps that are gathered
> come from this.
>
> [Clang has a option
> -gline-tables-only](http://clang.llvm.org/docs/UsersManual.html#cmdoption-gline-tables-only)
> which adds just the file name and line numbers to all functions in the debug
> info (Which can be done with minimal space overhead. [Gcc has
> '-g1'](https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html#Debugging-Options),
> which would mean that you would get line number + file names in back traces
> with less overhead.
>
> Those options would make it so what people see printed by the build bot
> is exactly what they see now. I think that should be an easily accessible
> level, but not default build.
>
>
> **Coredumps**: It is possible to make coredumps just as inspectable
> without including debug information in all the shipping binaries. In fact,
> you generally probably don't want to run gdb on the binary on the production
> host anyways. For this, what typically happens is that
>
> 1. People actually only look at the backtrace to figure out what went
> wrong
> 2. You can generate debug information which lives in seperate files from
> the target executable (-gsplit-dwarf on new versions of GCC, requires a
> relatively new toolchain. I believe there is a way to do it on older tooling
> where you split it out later, but am not familiar with it). This debug info
> would generally live in a '-dbg' package. Which you can install / run on your
> local machine to inspect whatever variables remain directly, as well as
> global state.
>
>
> **OVERALL**
> I don't see anywhere where this makes debugging considerably more
> difficult than it is already. Yes, you can't attach a GDB to any build of
> mesos without thinking about it. I think most of the time most developers
> aren't attaching GDB.
>
> CI jobs I think can be easily switched over where they care. There aren't
> that many of them around. I agree it is something people need to be aware of,
> but not a hard level of change. Most CI systems don't let people connect in
> and step through the program with GDB to where it failed a test. It just
> gives a backtrace. In the case of a buildbot it isn't hard to set CXXFLAGS to
> just the level you want to get that ('-g1'). No major loss there with this
> change.
>
> The only case this makes considerably more difficult is that if you want
> to attach GDB to inspect variabels you need to have specified --enable-debug
> at configure time. I don't think that is a major loss, or something which is
> in the standard compile/test loop of most most Mesos contributors /
> developers (Although we can collect evidence if desired)
Thanks for being thorough! Having `-g1` and `-gline-tables-only` sound good to
me for release builds, any reason not to include those?
As for local development, I would _personally_ be fine with optimization turned
off and `-g1` and `-gline-tables-only`, if the smaller debug information has a
measurable improvement on compile speed or memory consumption (let's measure!).
I tend to only need backtrace, or at most gdb thread backtraces from a dump.
These two use-cases seem to suggest leaving the existing `--disable-optimize`
and merely tweaking the debug flags to avoid dumping all debug information into
release builds.
How about for this change, we tackle the following:
Default is Release: -O2 -g1 (or -gline-tables-only)
--disable-optimize: -O0 -g1 (or -gline-tables-only)
In a subsequent patch, we can add an orthogonal debug flag for those who want
to the full set of debug information:
--enable-debug: turns on -g
Thoughts? It seems we can acheive Debug / Release builds even if we don't map
the distinction to a single binary configure flag.
- Ben
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56573
-----------------------------------------------------------
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now
> built in release (no debug info, optimized build). If '--enable-debug' is
> specified to configure, than optimization will be turned off, and debug info
> will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can
> use in code to see if mesos is built with debugging to enable extra
> assertions / checks. For release builds we may want to set 'NDEBUG' which
> removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible.
> Before some things like using clang would forcibly enable debug info in all
> cases
> 2) libmesos.so and the other binaries which get packaged up for use in
> distributions shrink considerably without manually stripping post-build
> (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get
> passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
