morningman opened a new pull request, #66960:
URL: https://github.com/apache/doris/pull/66960
### What problem does this PR solve?
Issue Number: close #66715
Related PR: #66510 (the BE build-time series this closes out)
Problem Summary:
BE compiles every TU with `-g -gdwarf-5`, and on this codebase that debug
info is not a rounding error — it is **95% of the bytes the compiler writes**.
Measured over the 7426 objects of a full Release build: `__text` 150 MB,
`.debug_*` **4.01 GB**. Type-level DWARF has to describe, per TU, every type
that TU touched, so it scales with the header closure rather than with the code.
Developers iterating locally rarely need variable-level DWARF; line tables
are enough for stack traces, `perf` and `addr2line`.
So: an opt-in env switch, off by default.
```bash
DORIS_DEV_DEBUG_INFO=line-tables ./build.sh --be
```
It appends `-gline-tables-only` **after** the existing `-g -gdwarf-5` (clang
lets the last debug-info flag win). Appending rather than replacing is the
point: with the switch unset, the `if` block contributes nothing, so the
command line is unchanged and release/CI builds, ninja state and ccache keys
are untouched. An unrecognized value is a configure FATAL, and the switch
requires clang (gcc has no `-gline-tables-only`).
**Measured** — macOS arm64, clang 20.1.8, `./build.sh --compile-bench -j 6`,
PCH on, ccache off, build dir recreated from scratch every run, same commit,
three back-to-back runs on one machine (default → line-tables → default, so
machine drift is measured rather than assumed):
| | default (run 1) | `line-tables` | default (run 3) | effect vs. mean of
the two defaults |
|---|---|---|---|---|
| cold build wall | 12m02s | **10m34s** | 11m52s | **-11.6%** |
| build phase wall | 11m28s | 10m03s | 11m22s | -11.9% |
| sum of TU wall | 3821s | 3341s | 3781s | -12.1% |
| sum of TU cpu | 3774s | 3281s | 3733s | -12.6% |
Machine drift between the two default runs: **-1.3%** — an order of
magnitude below the effect.
Artifacts (default vs. `line-tables`, same 7426 objects):
| | default | `line-tables` | delta |
|---|---|---|---|
| object files on disk | 4.25 GB | 0.99 GB | **-76.8%** |
| build dir | 9.4 GB | 2.8 GB | -70.3% |
| type-level `.debug_*` in objects | 4.01 GB | 0.51 GB | -87.2% |
| heaviest TU peak RSS | 3798 MB | 2618 MB | -31% |
| `doris_be` `__text` | 133,744,872 B | 133,744,872 B | **identical** |
Per-file: 207 TUs improved, 16 regressed (all ≤ +2.6s, clustered in one
target — scheduling noise). The two heaviest TUs, the `Exec` unity batches, go
1m01s → 39.5s and 1m00s → 38.0s.
**Disclosure / blind spots:**
- All numbers are macOS/clang20; Linux is covered only by this PR's CI. The
switch is clang-only by construction, and the gcc guard was not exercised
locally (no gcc toolchain on the bench machine) — it is a static `NOT
COMPILER_CLANG` check.
- **No link-time win on macOS**: ld64 leaves DWARF in the objects and builds
a debug map, so `doris_be` is 336 MB either way (`__text` byte-identical, 176 B
of file difference) and the link is 0.9s → 0.8s. On Linux ELF the linker *does*
copy `.debug_*` into the binary, so the binary itself should shrink there — not
measured, so no number is claimed.
- The switch reaches BE plus the contrib targets that inherit the directory
options; `clucene` (347 TUs) and `apache-orc` (48 TUs) set their own flags and
keep full debug info.
- What you give up: variable-level DWARF. Stack traces, `perf` and
`addr2line` still resolve to `file:line`; `print some_var` in gdb/lldb does
not. It is a dev-loop knob, not a debugging configuration.
- Toggling the switch changes the command line, so it forces a full rebuild
of that build dir and shares no ccache entries with default builds. Pick one
per build dir.
- Orthogonal to `STRIP_DEBUG_INFO=ON`, which splits debug info off **after**
linking (saves binary/deploy size, saves no compile time — the DWARF was
already generated). This switch avoids generating it at all.
### Release note
None
### Check List (For Author)
- Test
- [x] Manual test (add detailed scripts or steps below)
- The three paired cold benchmark builds described above (`build.sh
--compile-bench -j 6`), same commit, same machine, back to back, with the third
run quantifying drift.
- Verified the flag reaches the compiler: 7545 of 7940 entries in
the generated `compile_commands.json` carry `-gline-tables-only` (the remainder
are the two contrib targets that override flags).
- Verified emitted code is unaffected: `doris_be` `__text`
byte-identical between the default and `line-tables` builds.
- Verified the unknown-value path: `DORIS_DEV_DEBUG_INFO=full` stops
configure with `CMake Error at CMakeLists.txt:412 (message): Unknown
DORIS_DEV_DEBUG_INFO value: full (supported: line-tables)`.
- Default path: the switch adds compile options only from inside the
`if`, so an unset variable leaves the flags exactly as they are on master; the
two default runs above are that path.
- Behavior changed:
- [x] No. Default builds are unchanged; the switch is opt-in and
developer-only.
- Does this need documentation?
- [x] No. `build.sh -h` documents the switch next to `STRIP_DEBUG_INFO`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
