Re: Enabling the jobserver
On Tue, 11 Jun 2024 at 09:55, Eli Zaretskii wrote: > Thanks. I wonder how they distinguish between MinGW (a.k.a. "native") > port of GNU Make, and MSYS/Cygwin ports. The latter implement the > jobserver and the associated command-line arguments as on Unix. > Distinguishing only by name is dangerous: the MinGW port of Make on my > system is called "make.exe", not "mingw32-make.exe" (and that's what > people who install from ezwinports site get). There's a way to detect > the kind of the executable reliably (Emacs implements it, see > w32-application-type), but it's not entirely trivial. GCC doesn't attempt to distinguish. It doesn't look inside the name in the MAKE environment variable: just passes it to 'pex_one' (the exec/spawn wrapper in libiberty) to find and execute it. If the makefile is constructed to be system agnostic, maybe this could work. I think the real situation is that "native" Make is not supported.
Re: Enabling the jobserver
> From: Richard Copley > Date: Tue, 11 Jun 2024 09:30:13 +0100 > Cc: psm...@gnu.org, help-make@gnu.org > > On Tue, 11 Jun 2024 at 07:31, Eli Zaretskii wrote: > > > > Just curious: why does GCC need to know about Make's jobserver? > > (Since you say "GCC source", I presume you are not talking about the > > GCC build scripts, but GCC itself.) > > Its link-time optimization has some jobs to perform in parallel, and > attempts to do so by writing out a makefile and executing the program > named in the environment variable "MAKE" (or by executing "make", if > none). At the linking stage, when using link-time optimization, one > often gets the following message: > > lto-wrapper.exe: warning: using serial compilation of 2 LTRANS jobs > lto-wrapper.exe: note: see the '-flto' option documentation for more > information > > The "-flto" documentation does not directly explain the error or what > to do about it, so one has a fun puzzle. (Or one can ignore the > warning, as I have been doing for many years already. It doesn't do > any harm.) > > Passing "-flto=jobserver" does not work, in the first instance because > the "--jobserver-auth=" argument in MAKEARGS is overvalidated. GCC > ignores it (and removes it from MAKEARGS) unless it is a pair of > numbers or contains "fifo:". > > But passing "-flto=n" (where n is a number which should be used as the > argument to the "-j" option in the inferior make) does not work > either. > > One can set "export MAKE:=$(MAKE)" in a target-specific variable in > the top-level makefile to pass "mingw32-make" (the installed name of > native Windows GNU Make in its MSYS2 package) through to GCC, and then > GCC creates a temporary makefile and executes "mingw32-make", but an > error message is printed: > > The system cannot find the path specified. > mingw32-make[1]: [C:\Users\buster\AppData\Local\Temp\cc4moXuq.mk:3: > C:\Users\buster\AppData\Local\Temp\ccCEYc1I.ltrans0.ltrans.o] Error 1 > (ignored) > > The file that could not be found is presumably > "$(prefix)/lib/gcc/x86_64-w64-mingw32/14.1.0/collect2.exe" (another > layer of wrapping for the linker). I don't know why it isn't found. > Perhaps it is not on the path. By this time the temporary makefile has > been deleted, and for now, I have lost the motivation to debug > further. Thanks. I wonder how they distinguish between MinGW (a.k.a. "native") port of GNU Make, and MSYS/Cygwin ports. The latter implement the jobserver and the associated command-line arguments as on Unix. Distinguishing only by name is dangerous: the MinGW port of Make on my system is called "make.exe", not "mingw32-make.exe" (and that's what people who install from ezwinports site get). There's a way to detect the kind of the executable reliably (Emacs implements it, see w32-application-type), but it's not entirely trivial. But this quickly becomes way out off-topic here, I guess.
Re: Enabling the jobserver
On Tue, 11 Jun 2024 at 09:30, Richard Copley wrote: > > On Tue, 11 Jun 2024 at 07:31, Eli Zaretskii wrote: > > (Since you say "GCC source", I presume you are not talking about the > > GCC build scripts, but GCC itself.) Yes, I'm using GCC and GNU Make to build a project of my own. I'm not building GCC.
Re: Enabling the jobserver
On Tue, 11 Jun 2024 at 07:31, Eli Zaretskii wrote: > > > From: Richard Copley > > Date: Mon, 10 Jun 2024 23:29:28 +0100 > > Cc: help-make@gnu.org > > > > Having passed that hurdle, I see that GCC does not handle the > > named-semaphore jobserver in native GNU Make on Windows. (I had to > > read GCC's source to see it, since the error in that case simply says > > "'--jobserver-auth=' is not present in MAKEFLAGS". This subproblem > > appears to be unintentional and I have reported it on the GCC > > bugtracker.) None of that is a Make problem, of course. > > Just curious: why does GCC need to know about Make's jobserver? > (Since you say "GCC source", I presume you are not talking about the > GCC build scripts, but GCC itself.) Its link-time optimization has some jobs to perform in parallel, and attempts to do so by writing out a makefile and executing the program named in the environment variable "MAKE" (or by executing "make", if none). At the linking stage, when using link-time optimization, one often gets the following message: lto-wrapper.exe: warning: using serial compilation of 2 LTRANS jobs lto-wrapper.exe: note: see the '-flto' option documentation for more information The "-flto" documentation does not directly explain the error or what to do about it, so one has a fun puzzle. (Or one can ignore the warning, as I have been doing for many years already. It doesn't do any harm.) Passing "-flto=jobserver" does not work, in the first instance because the "--jobserver-auth=" argument in MAKEARGS is overvalidated. GCC ignores it (and removes it from MAKEARGS) unless it is a pair of numbers or contains "fifo:". But passing "-flto=n" (where n is a number which should be used as the argument to the "-j" option in the inferior make) does not work either. One can set "export MAKE:=$(MAKE)" in a target-specific variable in the top-level makefile to pass "mingw32-make" (the installed name of native Windows GNU Make in its MSYS2 package) through to GCC, and then GCC creates a temporary makefile and executes "mingw32-make", but an error message is printed: The system cannot find the path specified. mingw32-make[1]: [C:\Users\buster\AppData\Local\Temp\cc4moXuq.mk:3: C:\Users\buster\AppData\Local\Temp\ccCEYc1I.ltrans0.ltrans.o] Error 1 (ignored) The file that could not be found is presumably "$(prefix)/lib/gcc/x86_64-w64-mingw32/14.1.0/collect2.exe" (another layer of wrapping for the linker). I don't know why it isn't found. Perhaps it is not on the path. By this time the temporary makefile has been deleted, and for now, I have lost the motivation to debug further.
Re: Enabling the jobserver
> From: Richard Copley > Date: Mon, 10 Jun 2024 23:29:28 +0100 > Cc: help-make@gnu.org > > Having passed that hurdle, I see that GCC does not handle the > named-semaphore jobserver in native GNU Make on Windows. (I had to > read GCC's source to see it, since the error in that case simply says > "'--jobserver-auth=' is not present in MAKEFLAGS". This subproblem > appears to be unintentional and I have reported it on the GCC > bugtracker.) None of that is a Make problem, of course. Just curious: why does GCC need to know about Make's jobserver? (Since you say "GCC source", I presume you are not talking about the GCC build scripts, but GCC itself.)
Re: Enabling the jobserver
> If you provide a limit you'll see different behavior, e.g.: > > make -j5 I see, thank you. It might not hurt to mention this in the manual, obvious though it may seem. Having passed that hurdle, I see that GCC does not handle the named-semaphore jobserver in native GNU Make on Windows. (I had to read GCC's source to see it, since the error in that case simply says "'--jobserver-auth=' is not present in MAKEFLAGS". This subproblem appears to be unintentional and I have reported it on the GCC bugtracker.) None of that is a Make problem, of course.
Re: Enabling the jobserver
On Mon, 2024-06-10 at 22:14 +0100, Richard Copley wrote:
> What is the correct way to cause jobserver information to be passed
> to
> child processes? I tried this Makefile:
>
> all:
> @echo Jobserver present? $(filter jobserver,$(.FEATURES))
> +@sh -c "echo \"make toplevel MAKEFLAGS=${MAKEFLAGS}\""
> +@sh -c "echo \"child toplevel MAKEFLAGS=$${MAKEFLAGS}\""
> +$(MAKE) recursive
>
> recursive:
> +@sh -c "echo \"make recursive MAKEFLAGS=${MAKEFLAGS}\""
> +@sh -c "echo \"child recursive MAKEFLAGS=$${MAKEFLAGS}\""
>
> # Makefile ends
>
> The actual output is:
>
> make -j
This is your problem.
When you run "make -j" with no limit, then make doesn't ever create a
jobserver. Why create a jobserver when you have an unlimited number of
jobs to run?
In this case the "-j" flag just tells make to run everything it
possibly can with no limits.
If you provide a limit you'll see different behavior, e.g.:
make -j5
