For a long time I've been annoyed when building rustc because it
thinks it owns the machine and will schedule one job per CPU. On a
reasonably-well specified machine that is only being used for
compiling rust that is not too much of a problem.
But on lesser machines (such as the laptop where I'm writing this
while building rusts - 8 cores, 6.7GB RAM according to 'top' because
some is used for the graphics), or where you are trying to use a
graphical browser or do other things, at best the desktop can become
unresponsive from time to time, at worst it might swap badly and
become very unresponsive.
Until now, the only solution for minimising swap that I have found
was to take cores offline. But it turns out that rust, or rather
cargo, has now been adapted to use -j <N> or --jobs <N> like 'make'.
The bast that I can find for that is that it only works for packages
which use cargo to build them (e.g. cbindgen, perhaps librsvg).
Unfortunately, for building rustc itself that approach does not
work. Putting --jobs <N> in the RUSTFLAGS is not recognized.
Exporting it in CARGOFLAGS is recognized, but after a few minutes
rust fails with:
running:
"/scratch/working/rustc-1.42.0-src/build/x86_64-unknown-linux-gnu/stage0/bin/cargo"
"build" "-Zconfig-profile" "--target" "x86_64-unknown-linux-gnu" "--jobs" "4"
"-Zbinary-dep-depinfo" "-j" "8" "-v" "--release" "--features" "panic-unwind
backtrace compiler-builtins-c" "--manifest-path"
"/scratch/working/rustc-1.42.0-src/src/libtest/Cargo.toml" "--message-format"
"json-render-diagnostics"
error: The argument '--jobs <N>' was provided more than once, but cannot be
used multiple times
USAGE:
cargo build --features <FEATURES>... --jobs <N> --manifest-path <PATH>
--message-format <FMT>... --release --target <TRIPLE> -Z <FLAG>... --verbose
However, there seemed to be a partial solution. Googling for this
sort of rust problem is doubly hard - mismatches for a rust game and
for corrosion on iron and steel, but the worse problem is they
expect people to use rustup for the binary, therefore almost all
compile questions are about persuading it to use *more* CPU. But I
eventually I found a page showing the available options for ./x.py
and there it mentioned --jobs <N>.
With that, I've got all cores of this laptop online using --jobs 4
and although at times more than 4 C++ jobs seemed to be active in
'top', the loadavg was around 4 to 5 (with 'top' and other terms
also active. The box was very quickly into swap, and reached 0.7GB
swap usage.
Unfortunately, now the main rust compilations have started, up to 8
jobs are running at a time and the system is marginally unresponsive
although not using any more swap. /me swears, I thought I'd cracked
this. Perhaps the envvar CARGO_BUILD_JOBS=4 might help (I tried
that and then discarded it when I saw 8 C++ jobs running).
Hmm, swap is now up to 2.0GB - not yet sorted, but perhaps a step
on the way. And no, I'm not going to stop this compile to see if
the envvar is accepted - I'll wait until I've got tiem to try that
on a faster machine :)
ĸen
--
He died at the console, of hunger and thirst.
Next day he was buried, face-down, nine-edge first.
- the perfect programmer
--
http://lists.linuxfromscratch.org/listinfo/blfs-support
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page
