[gentoo-dev] Usage of dev-utils/ninja in ebuilds
For those unaware, dev-util/ninja is a make-replacement created by one
of the Chromium guys at Google. Its focus is on making incremental
builds of large software faster.
In the latest chromium ebuild (chromium-29.0.1516.3), we are using ninja
instead of make. ninja auto-detects the number of jobs to run by
default; in the chromium ebuild, we override this by passing -j
$(makeopts_jobs) instead.
Another place ninja is used cmake-utils.eclass, where it may be used
optionally in place of make depending on the CMAKE_MAKEFILE_GENERATOR
variable. The ninja_src_make function in cmake-utils.eclass calls ninja
with ${MAKEOPTS} or not depending on the value of CMAKE_VERBOSE. This is
a bit inconsistent and could cause problems because ninja only accepts a
limited subset of GNU make's command line options.
I'm wondering if we should create a more global function for calling
ninja in a consistent way. Maybe we should introduce a NINJAOPTS
variable for users to set in make.conf. Should we create a new eclass
for this, or just stick it in eutils?
Thoughts?
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] Usage of dev-utils/ninja in ebuilds
On Sat, 25 May 2013 14:48:30 -0400 Mike Gilbert flop...@gentoo.org wrote: For those unaware, dev-util/ninja is a make-replacement created by one of the Chromium guys at Google. Its focus is on making incremental builds of large software faster. I've no idea how this would work out, so I'm asking some questions from different perspective to form a better idea; this may step aside your discussion but on the other hand could help other unaware developers. If I missed earlier ML discussions on this by not paying attention or being too new, feel free to point me to those; no need to repeat. :) From an user perspective I was wondering if ninja in the Portage tree makes use of this faster incremental builds feature. Should I expect faster builds when trying this out? From a developer perspective, aside from the above feature, are there any other features that make ninja interesting to use in cmake ebuilds? When should it be used? From a maintainer's perspective, shouldn't the user be able to choose whether ninja or make is used and not the developer? Or does ninja really work out for the majority in a way nobody would complain? From an upstream perspective, would it be problematic if people use make instead of ninja; or wouldn't it translate into problems upstream has to resolve? (To me, using ninja instead of make sounds like using llvm instead of gcc; correct me if I'm wrong, I'm fairly new to ninja.) I'm wondering if we should create a more global function for calling ninja in a consistent way. Maybe we should introduce a NINJAOPTS variable for users to set in make.conf. Should we create a new eclass for this, or just stick it in eutils? If we don't allow the user to pick which make system is preferred, which I tried to suggest above; then I would prefer that we introduce a NINJAOPTS variable like you suggest to go alongside MAKEOPTS. In the case the user is able to choose which make system is preferred, then MAKEOPTS should probably apply to that make system; unless we allow maintainer's to override the user, then splitting them is back in favor. But well, this is based on preliminary thoughts without really properly knowing what the advantages and disadvantages here are. Hopefully you can shed some light here, thanks in advance. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Usage of dev-utils/ninja in ebuilds
On 05/25/2013 03:17 PM, Tom Wijsman wrote: From an user perspective I was wondering if ninja in the Portage tree makes use of this faster incremental builds feature. Should I expect faster builds when trying this out? No, you will not see any significant speed increase because we always start with an empty build directory in an ebuild. Nothing has changed in that regard. You may see small (~60 seconds) difference when compiling Chromium due to faster target dependency resolution. From a maintainer's perspective, shouldn't the user be able to choose whether ninja or make is used and not the developer? Or does ninja really work out for the majority in a way nobody would complain? As a maintainer, it is nice to know exactly how the software is being built in case of a bug report. This also helps if we need to report an issue upstream. I should clarify: ninja is not a *drop-in* replacement for make; it does not parse Makefiles. Instead, it uses its own configuration format, which may be generated using GYP or CMake. The choice of build system should be transparent to the user. However, I have no strong personal objection to making it configurable. From an upstream perspective, would it be problematic if people use make instead of ninja; or wouldn't it translate into problems upstream has to resolve? (To me, using ninja instead of make sounds like using llvm instead of gcc; correct me if I'm wrong, I'm fairly new to ninja.) So long as GYP supports emitting makefiles as well as ninja build files, I don't think this will be a problem. For Gentoo, I believe phajdan.jr simply chose to switch to ninja to more closely mimic the normal upstream build process. I'm wondering if we should create a more global function for calling ninja in a consistent way. Maybe we should introduce a NINJAOPTS variable for users to set in make.conf. Should we create a new eclass for this, or just stick it in eutils? If we don't allow the user to pick which make system is preferred, which I tried to suggest above; then I would prefer that we introduce a NINJAOPTS variable like you suggest to go alongside MAKEOPTS. In the case the user is able to choose which make system is preferred, then MAKEOPTS should probably apply to that make system; unless we allow maintainer's to override the user, then splitting them is back in favor. Another thought would be to mimic scons-utils.eclass, which utilizes a filtered copy of MAKEOPTS if SCONSOPTS is not defined. But well, this is based on preliminary thoughts without really properly knowing what the advantages and disadvantages here are. I believe the only real advantage to using ninja for building chromium is to follow upstream's default build configuration. It should be pretty transparent to the end user. signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Usage of dev-utils/ninja in ebuilds
On 5/25/13 9:17 PM, Tom Wijsman wrote: On Sat, 25 May 2013 14:48:30 -0400 Mike Gilbert flop...@gentoo.org wrote: For those unaware, dev-util/ninja is a make-replacement created by one of the Chromium guys at Google. Its focus is on making incremental builds of large software faster. I've no idea how this would work out, so I'm asking some questions from different perspective to form a better idea; this may step aside your discussion but on the other hand could help other unaware developers. If I missed earlier ML discussions on this by not paying attention or being too new, feel free to point me to those; no need to repeat. :) Ninja is simpler than make and strives to be as essential as possible, for those that know exotic build system is a close relative to plan9 mk. Ninja doesn't make any effort to make its build rules human readable, as said they have to be generated by something else, not sure how easy/hard would be debug the generated file. Being what it is used by default by upstream, probably it makes sense using it for building chromium. Using it on cmake shouldn't make much, if any, difference speed wise and would just make cmake build require yet another building block so I wouldn't use it if there aren't clear advantages in doing so (somebody could benchmark so we can see how it fares) lu
Re: [gentoo-dev] Usage of dev-utils/ninja in ebuilds
On 5/25/13 12:45 PM, Mike Gilbert wrote: You may see small (~60 seconds) difference when compiling Chromium due to faster target dependency resolution. You may also see faster builds in general because ninja is designed for very high degree of parallelization as compared to make. From a maintainer's perspective, shouldn't the user be able to choose whether ninja or make is used and not the developer? Or does ninja really work out for the majority in a way nobody would complain? Why would the user want to choose the build system? The choice of build system should be transparent to the user. However, I have no strong personal objection to making it configurable. The build system of choice actually is completely transparent to the user... From an upstream perspective, would it be problematic if people use make instead of ninja; or wouldn't it translate into problems upstream has to resolve? (To me, using ninja instead of make sounds like using llvm instead of gcc; correct me if I'm wrong, I'm fairly new to ninja.) So long as GYP supports emitting makefiles as well as ninja build files, I don't think this will be a problem. For Gentoo, I believe phajdan.jr simply chose to switch to ninja to more closely mimic the normal upstream build process. One part is indeed that the upstream default changed to ninja, but another one is that makefile support will be going away. The faster we can switch to ninja and fix any bugs resulting from that, the better. Let me repeat: Chromium project will be dropping support for generating makefiles, making ninja the only supported build system on Linux. Paweł signature.asc Description: OpenPGP digital signature
