Re: Should make handle non-blocking jobserver and use socket pair instead?
On Thu, 2019-08-15 at 17:26 +0300, Juha-Matti Tilli wrote: > GNU make expects the jobserver to be blocking, whereas a program > based on an event-loop architecture should really be using non- > blocking file descriptors. Well, GNU make invented the jobserver concept, and GNU make is not an event-loop architecture (at least not in the way you mean). The jobserver pipe is specifically and intentionally designed to be blocking, so that when a program needs to get a token it will block in the OS until there is one available, rather than having to maintain its own unnecessary sleep loop to wait. > I see two solutions: > > 1. Create a socket pair instead of a pipe so that I can use > MSG_DONTWAIT on a blocking socket I'm not interested in rewriting the jobserver to use a socket instead of a pipe. > 2. Use a select/poll + read pair, and immediately before the read, > turn on a 10 millisecond interval timer just in case some sub-make > won the race to acquire the token This also seems like a big change that would greatly increase the complexity of the jobserver. Also, note that there may be other users of the jobserver. I don't know if this ever happened but I had conversations with the GCC and binutils folks, discussing the possibility of multithreaded compilers or linkers that would interoperate with GNU make's jobserver feature to control how many threads they would use. Might I suggest a third option? In your program, create the jobserver as blocking as it is defined to be, and implement another method of integrating it with your event loop. For example, you could create a separate thread that would read the blocking jobserver descriptor and notify your event loop when a token was available. ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: gmake-4.2.90 patch
On Tue, 2019-09-03 at 10:27 -0400, Dmitry Goncharov wrote: > /* Define to 1 if you can safely include both and . */ > /* #undef TIME_WITH_SYS_TIME */ This is the reason for your problem. On my system, this is defined. What version of GNU/Linux are you using? What this undefined value means is that as far as autoconf is concerned, it's not valid to include both and at the same time on the same system. In this situation, if HAVE_SYS_TIME_H is defined (which it is for you) then will be included but not . The fact that you can add an include of into file.c and have it not fail shows that in fact, it IS OK to have both these headers included at the same time, which means that the configure check got it wrong. Can you email me (privately please, not to the entire list) your config.log file, or go through it to find the errors that were generated when configure attempted to determine whether both these headers can be included? Thanks! ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: gmake-4.2.90 regression (error message missing)
On Tue, 2019-09-03 at 10:54 -0400, Dmitry Goncharov wrote: > You were planning to use stat, right? Are you planning to stat the > interperter specified in shebang as well? > Some of the races can be avoided by opening the file before > posix_spawn and then using fstat on the fd. > Alternatively that posix_spawn check from configure.ac can be executed > at runtime to support cross compilation. I'm going to use a trivial best-effort approach, that should be sufficient to allow the regression tests to pass. I don't plan to care overly much about race conditions, weird interpreter/shebang issues, etc. If a simple attempt to detect and emulate the old-school fork/exec behavior doesn't pan out, I'll just show the failure as exit 127 like we do now. I don't think it's worthwhile to spend more effort on it than that, or to throw out posix_spawn() completely simply because some error messages are not as clear as they used to be--especially these type of errors which are due to fundamental failures in the build environment (trying to invoke non- executable files or non-existent files), not everyday things like failing compilations or whatever. ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: gmake-4.2.90 regression (error message missing)
You were planning to use stat, right? Are you planning to stat the interperter specified in shebang as well? Some of the races can be avoided by opening the file before posix_spawn and then using fstat on the fd. Alternatively that posix_spawn check from configure.ac can be executed at runtime to support cross compilation. regards, Dmitry On Mon, Sep 2, 2019 at 9:44 AM Paul Smith wrote: > > On Sun, 2019-09-01 at 15:33 -0400, Dmitry Goncharov via Bug reports and > discussion for GNU make wrote: > > This patch executes posix_spawnp at configuration time and then > > disables posix_spawn if posix_spawn returns 0 on an attempt to exec a > > missing file. > > This solution won't work for cross compilation. > > Thanks Dmitry. I guessed at this issue during testing: > > https://lists.gnu.org/archive/html/bug-make/2019-08/msg00063.html > > I have a different solution in mind that doesn't require using > configure tests and won't have issues with cross-compilation: I'm just > handling the 127 exit code and generating the appropriate error message > directly. > ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: gmake-4.2.90 patch
Sure, Paul. Here is the error message on redhat. /usr/bin/gcc -DHAVE_CONFIG_H -Isrc -I../src -Ilib -I../lib -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DLOCALEDIR=\"/usr/local/share/locale\"-m64 -g -O0 -MT src/file.o -MD -MP -MF $depbase.Tpo -c -o src/file.o ../src/file.c &&\ mv -f $depbase.Tpo $depbase.Po ../src/file.c: In function 'file_timestamp_sprintf': ../src/file.c:887: warning: initialization makes pointer from integer without a cast ../src/file.c:891: error: dereferencing pointer to incomplete type ../src/file.c:891: error: dereferencing pointer to incomplete type ../src/file.c:891: error: dereferencing pointer to incomplete type ../src/file.c:892: error: dereferencing pointer to incomplete type ../src/file.c:892: error: dereferencing pointer to incomplete type ../src/file.c:892: error: dereferencing pointer to incomplete type gmake[1]: *** [src/file.o] Error 1 Here is config.h /* src/config.h. Generated from config.h.in by configure. */ /* src/config.h.in. Generated from configure.ac by autoheader. */ /* CPU and C ABI indicator */ #ifndef __i386__ /* #undef __i386__ */ #endif #ifndef __x86_64_x32__ /* #undef __x86_64_x32__ */ #endif #ifndef __x86_64__ #define __x86_64__ 1 #endif #ifndef __alpha__ /* #undef __alpha__ */ #endif #ifndef __arm__ /* #undef __arm__ */ #endif #ifndef __armhf__ /* #undef __armhf__ */ #endif #ifndef __arm64_ilp32__ /* #undef __arm64_ilp32__ */ #endif #ifndef __arm64__ /* #undef __arm64__ */ #endif #ifndef __hppa__ /* #undef __hppa__ */ #endif #ifndef __hppa64__ /* #undef __hppa64__ */ #endif #ifndef __ia64_ilp32__ /* #undef __ia64_ilp32__ */ #endif #ifndef __ia64__ /* #undef __ia64__ */ #endif #ifndef __m68k__ /* #undef __m68k__ */ #endif #ifndef __mips__ /* #undef __mips__ */ #endif #ifndef __mipsn32__ /* #undef __mipsn32__ */ #endif #ifndef __mips64__ /* #undef __mips64__ */ #endif #ifndef __powerpc__ /* #undef __powerpc__ */ #endif #ifndef __powerpc64__ /* #undef __powerpc64__ */ #endif #ifndef __powerpc64_elfv2__ /* #undef __powerpc64_elfv2__ */ #endif #ifndef __riscv32__ /* #undef __riscv32__ */ #endif #ifndef __riscv64__ /* #undef __riscv64__ */ #endif #ifndef __riscv32_ilp32__ /* #undef __riscv32_ilp32__ */ #endif #ifndef __riscv32_ilp32f__ /* #undef __riscv32_ilp32f__ */ #endif #ifndef __riscv32_ilp32d__ /* #undef __riscv32_ilp32d__ */ #endif #ifndef __riscv64_ilp32__ /* #undef __riscv64_ilp32__ */ #endif #ifndef __riscv64_ilp32f__ /* #undef __riscv64_ilp32f__ */ #endif #ifndef __riscv64_ilp32d__ /* #undef __riscv64_ilp32d__ */ #endif #ifndef __riscv64_lp64__ /* #undef __riscv64_lp64__ */ #endif #ifndef __riscv64_lp64f__ /* #undef __riscv64_lp64f__ */ #endif #ifndef __riscv64_lp64d__ /* #undef __riscv64_lp64d__ */ #endif #ifndef __s390__ /* #undef __s390__ */ #endif #ifndef __s390x__ /* #undef __s390x__ */ #endif #ifndef __sh__ /* #undef __sh__ */ #endif #ifndef __sparc__ /* #undef __sparc__ */ #endif #ifndef __sparc64__ /* #undef __sparc64__ */ #endif /* Define if building universal (internal helper macro) */ /* #undef AC_APPLE_UNIVERSAL_BUILD */ /* Define to 1 if the `closedir' function returns void instead of `int'. */ /* #undef CLOSEDIR_VOID */ /* Define to one of `_getb67', `GETB67', `getb67' for Cray-2 and Cray-YMP systems. This function is required for `alloca.c' support on those systems. */ /* #undef CRAY_STACKSEG_END */ /* Define to 1 if using `alloca.c'. */ /* #undef C_ALLOCA */ /* Define to 1 for DGUX with . */ /* #undef DGUX */ /* Define to 1 if translation of program messages to the user's native language is requested. */ #define ENABLE_NLS 1 /* Use high resolution file timestamps if nonzero. */ #define FILE_TIMESTAMP_HI_RES 1 /* Define to a C preprocessor expression that evaluates to 1 or 0, depending whether the gnulib module strerror shall be considered present. */ #define GNULIB_STRERROR 1 /* Define to 1 when the gnulib module getloadavg should be tested. */ #define GNULIB_TEST_GETLOADAVG 1 /* Define to 1 when the gnulib module strerror should be tested. */ #define GNULIB_TEST_STRERROR 1 /* Define to 1 if you have 'alloca' after including , a header that may be supplied by this distribution. */ #define HAVE_ALLOCA 1 /* Define to 1 if you have and it should be used (not on Ultrix). */ #define HAVE_ALLOCA_H 1 /* Define to 1 if you have the `atexit' function. */ #define HAVE_ATEXIT 1 /* Use case insensitive file names */ /* #undef HAVE_CASE_INSENSITIVE_FS */ /* Define to 1 if you have the Mac OS X function CFLocaleCopyCurrent in the CoreFoundation framework. */ /* #undef HAVE_CFLOCALECOPYCURRENT */ /* Define to 1 if you have the Mac OS X function CFPreferencesCopyAppValue in the CoreFoundation framework. */ /* #undef HAVE_CFPREFERENCESCOPYAPPVALUE */ /* Define to 1 if you have the clock_gettime function. */ #define HAVE_CLOCK_GETTIME 1 /* Define if the GNU dcgettext() function is already present or preinstalled. */ #define HAVE_DCGETTEXT 1 /* Define to 1 if you have t
Re: GNU make 4.2.90 release candidate available
On Wed, 2019-08-28 at 17:35 -0400, Dennis Clarke wrote: > > This also disables every extension over C, especially POSIX. > > Is this a *bad* thing or a *good* thing ? What is the actual language > dialect that GNU Make wishes to be compliant with is the real question. Like all GNU software, it is written to work with a minimum version but uses autoconf facilities to take advantage of the newest capabilities it can detect... that's kind of autoconf's raison d'être. So, it is not useless to test building with all improvements turned off because it lets us know whether our attempts to support very old environments are still working properly. But I would not recommend using such options when trying to compile an actual, useful GNU make for everyday use, as it will be full of workarounds and hacks and won't take advantage of modern POSIX features which give better behavior and performance. ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: GNU make 4.2.90 release candidate available
On Tue, 2019-09-03 at 10:25 +0200, Robert Pluim wrote: > The good news is that this affects only building from git, the > pre-release tarball builds fine with just './configure', and passes > all but one test from 'make check'. Yes, I do rely on GCC (and GNU make) when performing maintainer builds. You could just set AM_CFLAGS to the empty string, if you wanted to. I assume that would work but have not tested it. Thanks! ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #56701] Do not allow -j without a number
> On Mon, 26 Aug 2019 08:35:15 -0400 (EDT), "Paul D. Smith" > said: Paul> Update of bug #56701 (project make): Paul> Item Group: Bug => Enhancement Paul> ___ Paul> Follow-up Comment #1: Paul> Actually -j without a number is useful: it's used in conjunction with the -l Paul> option to allow parallelism to be limited by system load rather than an Paul> explicit number of outstanding jobs. Paul> I implemented a change which requires the -l option to be provided if -j is Paul> given without an argument, else you get an error. Paul> However that's a large backward-compatibility change so I'm not sure about it. Paul> Just as an example, I had to modify quite a number of tests in the GNU make Paul> regression test suite after making this change. Of course, it's quite Paul> reasonable to say that the usages in the regression test suite are not Paul> appropriate to "real world" usages. Paul> I'll need to think about this. If anyone has opinions on whether this would Paul> be a good change and/or how much breakage it would cause please let me know. Paul> I'm changing this to an enhancement because the current behavior is (a) Paul> documented, (b) useful, and (c) how make has worked for 30+ years. The Paul> question is can we find a way to avoid the downsides, and is the cost in Paul> backward-compatibiity worth it. Please donʼt do this. When I type 'make -j', I mean it. If I get it wrong and kill my box, thatʼs on me, make should not be handholding here. Robert ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: GNU make 4.2.90 release candidate available
> On Mon, 02 Sep 2019 12:28:52 -0400, Paul Smith said: Paul> My first guess is that this another of a series of unpleasantnesses Paul> caused by clang trying to "pretend" to be GCC, and failing. Paul> I understand why the Clang devs do this: they want to take advantage of Paul> the built-in userbase for GCC software. But it's extremely painful for Paul> portability since clang is NOT GCC. Paul> If you run "./configure CC=clang" instead of allowing it to find the Paul> "gcc" wrapper that Apple installs, does it work properly? No, clang gets invoked with the same flags, and fails the same way. Paul> This behavior (invoking GCC using these flags) is part of the automake Paul> package, not GNU make itself, so it will be very painful for us to work Paul> around. Yes. Plus removing '-C' from AM_CFLAGS reveals that there are other flags that clang doesnʼt like. The good news is that this affects only building from git, the pre-release tarball builds fine with just './configure', and passes all but one test from 'make check'. Robert ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
