On 10/29/24 13:54, Andrew wrote:
Ideally, yes.
Not certain that 100% of make environments have that exact command available
(e.g. Command Prompt, PowerShell).
Which is why I showed how to do it on Mac and BSD, yes.
Linux 2.0 shipped SMP support in 1996 (28 years ago), Intel shipped its
first SMP laptop processor in 2006 (18 years ago), and the first phone
with multiple processors came out in 2011 (13 years ago).
All of course too recent for Posix, but hopefully someday...
It's also more efficient to query an OS API than to create a new process and
parse the I/O streams.
Which is why I was suggesting that "make -j" detect it, via whatever
mechanism the other commands are using, so you don't have to specify it
on the command line.
The current behavior of gmake is that -j without a number (I.E. the next
argument is not a number, which is gnu/terrible UI design but decades
established by now) will forkbomb the system:
-j [jobs], --jobs[=jobs]
Specifies the number of jobs (commands) to run simultaneously. If
there is more than one -j option, the last one is effective. If
the -j option is given without an argument, make will not limit
the number of jobs that can run simultaneously. When make invokes
a sub-make, all instances of make will coordinate to run the spec‐
ified number of jobs at a time; see the section PARALLEL MAKE AND
THE JOBSERVER for details.
The forkbomb usually triggers the OOM killer in my experience even on
pretty big systems, so this default value is unlikely to be that widely
used in the wild.
Defaulting to "the number of processors you actually have" seems to make
more sense. Or maybe capital -J if somebody actually wants to keep the
forkbomb behavior for some reason? That's not yet used in the man page.
But you can also just -j 99999 if you really _want_ to turn the kernel
build into a forkbomb for some reason, and doing -j should complete the
build either way modulo the OOM killer.
I've seen a lot more "we hardwired the number of parallel instances our
magic build machine had 12 years ago" in build scripts, and keep
replacing them with -j $(nproc) and get nasty emails from homebrew
users, and send them the very small shell script that implements nproc
there (which homebrew somehow doesn't bother to do because hey, no
standard)...
I propose attempting to query first, and apply a default of 4 as a fallback.
Does "can't detect" mean "detected less than 4"?
*shrug* No strong objection I guess...
Note that make implementations such as GNU currently interleave the output
streams
from different concurrent tasks, making them harder to read.
I posted this to linux-kernel twenty two years ago, where linux weekly
news picked it up:
https://static.lwn.net/2002/0117/a/blueberry.php3
Almost 23 now. The general idea was adopted by linux's kbuild, and wound
up fairly widely distributed since. I've even seen cmake do a multi-line
curses thing, which I first noticed because it broke the log output.
(They now detect whether output is to a tty and provide radically
different output depending on whether you | tee log.txt or not, so
that's nice.)
Recommend striping outputs in dedicated ncurses style bands, like Docker.
Please don't.
Make combines imperative and declarative code in the same context, has
WAY more significant whitespace than python (which doesn't require hard
tabs), has entire schools of thought about how NOT to use it:
https://accu.org/journals/overload/14/71/miller_2004/
...and currently has about as many projects vying to replace it as cvs
had before "git" emerged as the new standard. The advantage of make is
it exists, has known behavior, and at least at a surface level is fairly
simple. Trying to get fancy will most likely encourage faster migration
away from it, to projects which are _not_ standardized and usually don't
even have multiple compatible-ish implementations.
To be honest, 90% of the reason make still exists is "cc -j" isn't a
thing, nor does the compiler have a partial build option checking if
file.c is newer than file.o, so "cc *.c" is a single-threaded build that
takes forever and always does a full rebuild. Otherwise just about
everything "make" does would make more sense as a fairly small shell script.
My point is if it still exists because of -j then it would be nice if -j
had intelligent default behavior.
Rob
