Your message dated Fri, 19 Jul 2024 02:23:13 -0400
with message-id <[email protected]>
and subject line Re: Redundant build when making packed_resources
has caused the Debian Bug report #1072317,
regarding Redundant build when making packed_resources
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
1072317: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1072317
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Package: chromium
Version: 125.0.6422.141-1
Severity: minor
This is something I've noticed in the course of the build that has been
annoying me for some time, and I thought I'd write it up.
Here is an excerpt from the build log which illustrates the problem:
[...]
[61738/61742] AR obj/content/web_test/libweb_test_renderer.a
[61739/61742] AR obj/content/web_test/libweb_test_browser.a
[61740/61742] AR obj/content/shell/libcontent_shell_app.a
[61741/61742] LINK ./content_shell
[61742/61742] LINK ./chrome
cp out/Release/chrome out/Release/chromium
cp out/Release/content_shell out/Release/chromium-shell
[...]
debian/rules override_dh_auto_build-indep
make[1]: Entering directory '/tmp/chromium-125.0.6422.141'
gn gen out/Release --args="clang_use_chrome_plugins=false ..."
Done. Made 20173 targets from 3449 files in 10398ms
ninja -j8 -C out/Release packed_resources
ninja: Entering directory `out/Release'
[1/30013] CXX
obj/base/allocator/partition_allocator/src/partition_alloc/allocator_base/file_util_posix.o
[2/30013] CXX
obj/base/allocator/partition_allocator/src/partition_alloc/allocator_base/time_override.o
[3/30013] CXX
obj/base/allocator/partition_allocator/src/partition_alloc/allocator_base/stack_trace_linux.o
[...]
It's not clear why, but the "gn gen" operation in the
override_dh_auto_build-indep target renders a significant chunk of the
existing build tree stale, and the packed_resources build then has to
recompile a bunch of things that are distant dependencies of the
requested resource files.
I can't say at what version this started happening, but I do seem to
recall at one point the packed_resources build needing only a three-
digit number of steps.
Would it be reasonable to do something like
override_dh_auto_build-indep:
- gn gen out/Release --args="$(defines)"
+ test -f out/Release/build.ninja || gn gen out/Release
--args="$(defines)"
ninja -j$(njobs) -C out/Release packed_resources
so that it doesn't run "gn gen" again, if not necessary?
--Daniel
--- End Message ---
--- Begin Message ---
I finally took the time to dig into this, and found that the problem is
much different from what it originally looked like.
The second "gn gen" is not, in fact, doing any harm. If you make
packed_resources right after making chrome et al., it'll still take >30K
steps to get there.
The problem lies in ninja(1) itself, specifically with how it
canonicalizes the paths of dependencies found in the *.d files
written out by the compiler. The canonicalization logic it uses is
just a straight string operation on the pathname, that does not check
whether any components of the path are symlinks (for performance
reasons). Combined with the /bin -> /usr/bin symlink gifted to us
from usrmerge, this causes the .ninja_deps database to record
incorrect paths for various system header files (starting with
/include/ instead of /usr/include/). And then the next time ninja
runs, it says, oh, these headers don't exist, so I'm gonna remake
everything that depends on them.
The whole sordid tale is explained in
https://github.com/ninja-build/ninja/issues/1330
and remains unresolved as of this writing.
Further testing confirmed that this occurs when the compiler is invoked
as /bin/clang*, but not as /usr/bin/clang*, because Clang apparently
uses $(dirname $0) as the starting point (!) for paths written into
dependency files.
Finally, /usr/bin is usually placed before /bin in PATH, so the compiler
should always be invoked with the former prefix rather than the latter.
But as I build Chromium and Firefox repeatedly, I also use ccache(1)---
and for some reason, even though ccache claims to iterate through PATH
to find the requested compiler, it was locating said compiler in /bin.
The workaround on my side, then, was to set CCACHE_PATH=/usr/bin. If
ccache is not being used, then simply having a normal PATH with
/usr/bin before /bin is enough. This is a gotcha to watch out for with
ninja + usrmerge, but otherwise does not necessitate any change to the
chromium package.
Apologies for the noise,
--Daniel
--- End Message ---
