On Thu, Oct 31, 2024, 15:49 Rob Landley via austin-group-l at The Open Group <[email protected]> wrote:
> On 10/29/24 13:54, Andrew wrote: > > Ideally, yes. > > > > Not certain that 100% of make environments have that exact command > available (e.g. Command Prompt, PowerShell). > > Which is why I showed how to do it on Mac and BSD, yes. > > Linux 2.0 shipped SMP support in 1996 (28 years ago), Intel shipped its > first SMP laptop processor in 2006 (18 years ago), and the first phone > with multiple processors came out in 2011 (13 years ago). > > All of course too recent for Posix, but hopefully someday... > > > It's also more efficient to query an OS API than to create a new process > and parse the I/O streams. > > Which is why I was suggesting that "make -j" detect it, via whatever > mechanism the other commands are using, so you don't have to specify it > on the command line. > > The current behavior of gmake is that -j without a number (I.E. the next > argument is not a number, which is gnu/terrible UI design but decades > established by now) will forkbomb the system: > > -j [jobs], --jobs[=jobs] > Specifies the number of jobs (commands) to run simultaneously. If > there is more than one -j option, the last one is effective. If > the -j option is given without an argument, make will not limit > the number of jobs that can run simultaneously. When make invokes > a sub-make, all instances of make will coordinate to run the spec‐ > ified number of jobs at a time; see the section PARALLEL MAKE AND > THE JOBSERVER for details. > > The forkbomb usually triggers the OOM killer in my experience even on > pretty big systems, so this default value is unlikely to be that widely > used in the wild. > > Defaulting to "the number of processors you actually have" seems to make > more sense. Or maybe capital -J if somebody actually wants to keep the > forkbomb behavior for some reason? That's not yet used in the man page. > But you can also just -j 99999 if you really _want_ to turn the kernel > build into a forkbomb for some reason, and doing -j should complete the > build either way modulo the OOM killer. > > I've seen a lot more "we hardwired the number of parallel instances our > magic build machine had 12 years ago" in build scripts, and keep > replacing them with -j $(nproc) and get nasty emails from homebrew > users, and send them the very small shell script that implements nproc > there (which homebrew somehow doesn't bother to do because hey, no > standard)... > > > I propose attempting to query first, and apply a default of 4 as a > fallback. > > Does "can't detect" mean "detected less than 4"? > > *shrug* No strong objection I guess... > > > Note that make implementations such as GNU currently interleave the > output streams > > from different concurrent tasks, making them harder to read. > > I posted this to linux-kernel twenty two years ago, where linux weekly > news picked it up: > > https://static.lwn.net/2002/0117/a/blueberry.php3 > > Almost 23 now. The general idea was adopted by linux's kbuild, and wound > up fairly widely distributed since. I've even seen cmake do a multi-line > curses thing, which I first noticed because it broke the log output. > (They now detect whether output is to a tty and provide radically > different output depending on whether you | tee log.txt or not, so > that's nice.) > technically, i think that's actually the ninja backend, the same as Android's build system uses as soong's backend? cmake either calls make or ninja as the "assembler" to its "compiler". > Recommend striping outputs in dedicated ncurses style bands, like Docker. > > Please don't. > > Make combines imperative and declarative code in the same context, has > WAY more significant whitespace than python (which doesn't require hard > tabs), has entire schools of thought about how NOT to use it: > > https://accu.org/journals/overload/14/71/miller_2004/ > > ...and currently has about as many projects vying to replace it as cvs > had before "git" emerged as the new standard. The advantage of make is > it exists, has known behavior, and at least at a surface level is fairly > simple. Trying to get fancy will most likely encourage faster migration > away from it, to projects which are _not_ standardized and usually don't > even have multiple compatible-ish implementations. > > To be honest, 90% of the reason make still exists is "cc -j" isn't a > thing, nor does the compiler have a partial build option checking if > file.c is newer than file.o, so "cc *.c" is a single-threaded build that > takes forever and always does a full rebuild. Otherwise just about > everything "make" does would make more sense as a fairly small shell > script. > > My point is if it still exists because of -j then it would be nice if -j > had intelligent default behavior. > > Rob > >
