bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin

[Sergei Trofimovich]

On Fri, 19 Apr 2024 00:33:52 -0700
Paul Eggert  wrote:

> On 2024-04-18 14:52, Sergei Trofimovich wrote:
> > $ clang simple.c -o simple && echo 42 | ./simple
> > 1: ino=3009428657538693161
> > 2: ino=3009428657538693161
> > 3: ino=1568241705
> > 
> > Note how stat() and fstat() don't agree on inode.
> > 
> > Apparently it's documented in
> > https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/fstat.2.html
> > as
> > 
> >BUGS
> >   Applying fstat to a socket (and thus to a pipe) returns a zero'd 
> > buffer,
> >   except for the blocksize field, and a unique device and inode number. 
> >  
> 
> The BUGS note simply means that a pipe has a unique inode number, which 
> is what we want. So that's not indicating any problem.
> 
> 
> Oh, I see the problem now.  For a socket or pipe, macOS fstat returns 
> the full 64-bit inode number, whereas macOS stat returns only the low 
> order 32 bits.  In your example, 3009428657538693161 % (2**32) == 
> 1568241705.
> 
> This is a kernel bug in macOS. Can you report it or otherwise arrange to 
> have the kernel bug fixed? I expect that you have better connections 
> with Apple than I do. A proposed patch (relative to xnu-10063.101.15) is 
> attached; I have not tested it as I don't use macOS. Thanks.

I reported it via https://www.apple.com/feedback/macos.html

> Also, I am documenting this macOS bug in Gnulib by installing the second 
> attached patch to Gnulib, and am cc'ing this email to bug-gnulib.

Thank you, Paul!

-- 

  Sergei

Re: bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin

[Sergei Trofimovich]

On Fri, 19 Apr 2024 00:33:52 -0700
Paul Eggert  wrote:

> On 2024-04-18 14:52, Sergei Trofimovich wrote:
> > $ clang simple.c -o simple && echo 42 | ./simple
> > 1: ino=3009428657538693161
> > 2: ino=3009428657538693161
> > 3: ino=1568241705
> > 
> > Note how stat() and fstat() don't agree on inode.
> > 
> > Apparently it's documented in
> > https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/fstat.2.html
> > as
> > 
> >BUGS
> >   Applying fstat to a socket (and thus to a pipe) returns a zero'd 
> > buffer,
> >   except for the blocksize field, and a unique device and inode number. 
> >  
> 
> The BUGS note simply means that a pipe has a unique inode number, which 
> is what we want. So that's not indicating any problem.
> 
> 
> Oh, I see the problem now.  For a socket or pipe, macOS fstat returns 
> the full 64-bit inode number, whereas macOS stat returns only the low 
> order 32 bits.  In your example, 3009428657538693161 % (2**32) == 
> 1568241705.
> 
> This is a kernel bug in macOS. Can you report it or otherwise arrange to 
> have the kernel bug fixed? I expect that you have better connections 
> with Apple than I do. A proposed patch (relative to xnu-10063.101.15) is 
> attached; I have not tested it as I don't use macOS. Thanks.

I reported it via https://www.apple.com/feedback/macos.html

> Also, I am documenting this macOS bug in Gnulib by installing the second 
> attached patch to Gnulib, and am cc'ing this email to bug-gnulib.

Thank you, Paul!

-- 

  Sergei

bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin

[Sergei Trofimovich]

On Tue, 16 Apr 2024 22:37:58 +0100
Sergei Trofimovich  wrote:

> On Tue, 16 Apr 2024 12:33:44 +0100
> Pádraig Brady  wrote:
> 
> > On 16/04/2024 01:19, Alejandro Colomar wrote:  
> > > Hi!
> > > 
> > > I don't own a Darwin system, so I can't help much reproduce.  However,
> > > I've received a bug report to the Linux man-pages, that our build
> > > system (GNUmakefile-based), which ends up calling
> > > 
> > >   ... | install /dev/stdin $@
> > > 
> > > doesn't work on Darwin.  Here's the original bug report:
> > > <https://github.com/NixOS/nixpkgs/pull/300797>.
> > > 
> > > Here are the reported error messages:
> > > 
> > > ...
> > > INSTALL 
> > > /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addseverity.3
> > > INSTALL 
> > > /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/adjtime.3
> > > install: skipping file '/dev/stdin', as it was replaced while being copied
> > > make: *** [share/mk/install/man.mk:54: 
> > > /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addmntent.3]
> > >  Error 1
> > > make: *** Waiting for unfinished jobs
> > > install: skipping file '/dev/stdin', as it was replaced while being copied
> > > make: *** [share/mk/install/man.mk:54: 
> > > /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/acosh.3]
> > >  Error 1
> > > install: skipping file '/dev/stdin', as it was replaced while being copied
> > > install: skipping file '/dev/stdin', as it was replaced while being copied
> > > install: skipping file '/dev/stdin', as it was replaced while being copied
> > > ...
> > > 
> > > I don't see why install(1) should fail to read /dev/stdin under any
> > > POSIX system
> > 
> > What version of darwin is this? I can't repro on Darwin 21.6.0 (MacOSX 
> > 12.6).
> > The issue seems to be that /dev/stdin returns a varying inode which 
> > install(1) doesn't like currently  
> 
> The system I have locally is this one:
> 
> $ uname -a
> Darwin tests-iMac.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Jun 22 
> 19:37:08 PDT 2021; root:xnu-4903.278.70~1/RELEASE_X86_64 x86_64

I debugged it a bit more locally and it looks like it's a known
limitation of fstat()/stat() mismatch on FIFOs on Darwin. Example
program:

$ cat simple.c
#include 
#include 
#include 
#include 
#include 

int main() {
  struct stat s;
  int fd;
  int r;

  fd = open("/dev/fd/0", O_RDONLY);
  assert(fd > 0);
  r = fstat(fd, );
  assert(r == 0);
  fprintf(stderr, "1: ino=%llu\n", s.st_ino);
  close(fd);

  fd = open("/dev/fd/0", O_RDONLY);
  assert(fd > 0);
  r = fstat(fd, );
  assert(r == 0);
  fprintf(stderr, "2: ino=%llu\n", s.st_ino);
  close(fd);

  r = stat("/dev/fd/0", );
  assert(r == 0);
  fprintf(stderr, "3: ino=%llu\n", s.st_ino);
  close(fd);
}

Running:

$ clang simple.c -o simple && echo 42 | ./simple
1: ino=3009428657538693161
2: ino=3009428657538693161
3: ino=1568241705

Note how stat() and fstat() don't agree on inode.

Apparently it's documented in
https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/fstat.2.html
as

  BUGS
 Applying fstat to a socket (and thus to a pipe) returns a zero'd buffer,
 except for the blocksize field, and a unique device and inode number.

Perhaps coreutils should avoid inode checks on FIFOs and sockets. They
are already volatile by nature.

-- 

  Sergei

bug#70411: [bug] install(1) fails to read /dev/stdin on Darwin

[Sergei Trofimovich]

On Tue, 16 Apr 2024 12:33:44 +0100
Pádraig Brady  wrote:

> On 16/04/2024 01:19, Alejandro Colomar wrote:
> > Hi!
> > 
> > I don't own a Darwin system, so I can't help much reproduce.  However,
> > I've received a bug report to the Linux man-pages, that our build
> > system (GNUmakefile-based), which ends up calling
> > 
> > ... | install /dev/stdin $@
> > 
> > doesn't work on Darwin.  Here's the original bug report:
> > .
> > 
> > Here are the reported error messages:
> > 
> > ...
> > INSTALL 
> > /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addseverity.3
> > INSTALL 
> > /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/adjtime.3
> > install: skipping file '/dev/stdin', as it was replaced while being copied
> > make: *** [share/mk/install/man.mk:54: 
> > /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/addmntent.3]
> >  Error 1
> > make: *** Waiting for unfinished jobs
> > install: skipping file '/dev/stdin', as it was replaced while being copied
> > make: *** [share/mk/install/man.mk:54: 
> > /nix/store/3s28l9ijlkmsq8256zdxjvl173gkn37c-man-pages-6.7/share/man/man3/acosh.3]
> >  Error 1
> > install: skipping file '/dev/stdin', as it was replaced while being copied
> > install: skipping file '/dev/stdin', as it was replaced while being copied
> > install: skipping file '/dev/stdin', as it was replaced while being copied
> > ...
> > 
> > I don't see why install(1) should fail to read /dev/stdin under any
> > POSIX system  
> 
> What version of darwin is this? I can't repro on Darwin 21.6.0 (MacOSX 12.6).
> The issue seems to be that /dev/stdin returns a varying inode which 
> install(1) doesn't like currently

The system I have locally is this one:

$ uname -a
Darwin tests-iMac.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Jun 22 
19:37:08 PDT 2021; root:xnu-4903.278.70~1/RELEASE_X86_64 x86_64

-- 

  Sergei

[PATCH] kexec-tools: purgatory: fix build on `binutils-2.42`

[Sergei Trofimovich]

`binutils-2.42` introduced stricter checks on what `.arch` can be used
in 64-bit mode and started failing the build as:

$ as-2.42 --64 -o entry32-16-debug.o entry32-16-debug.s
purgatory/arch/i386/entry32-16-debug.S: Assembler messages:
purgatory/arch/i386/entry32-16-debug.S:28: Error: 64bit mode not supported 
on `i386'.

The change moves `.code32` before `.arch 386` as suggested in
https://sourceware.org/PR31319

Signed-off-by: Sergei Trofimovich 
---
 purgatory/arch/i386/entry32-16-debug.S | 2 +-
 purgatory/arch/i386/entry32-16.S   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/purgatory/arch/i386/entry32-16-debug.S 
b/purgatory/arch/i386/entry32-16-debug.S
index 5167944..297d6f5 100644
--- a/purgatory/arch/i386/entry32-16-debug.S
+++ b/purgatory/arch/i386/entry32-16-debug.S
@@ -25,10 +25,10 @@
.globl entry16_debug_pre32
.globl entry16_debug_first32
.globl entry16_debug_old_first32
+   .code32
.arch i386
.balign 16
 entry16_debug:
-   .code32
/* Compute where I am running at (assumes esp valid) */
call1f
 1: popl%ebx
diff --git a/purgatory/arch/i386/entry32-16.S b/purgatory/arch/i386/entry32-16.S
index c051aab..7a84565 100644
--- a/purgatory/arch/i386/entry32-16.S
+++ b/purgatory/arch/i386/entry32-16.S
@@ -20,10 +20,10 @@
 #undef i386
.text
.globl entry16, entry16_regs
+   .code32
.arch i386
.balign 16
 entry16:
-   .code32
/* Compute where I am running at (assumes esp valid) */
call1f
 1: popl%ebx
-- 
2.43.0


___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[sr #110990] AS_IF / AC_CHECK_LIB generates invalid shell on editline-1.17.1

[Sergei Trofimovich]

Follow-up Comment #3, sr#110990 (group autoconf):

Proposed `editline` change to adapt to `autoconf-2.72` as
https://github.com/troglobit/editline/pull/64


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #110990] AS_IF / AC_CHECK_LIB generates invalid shell on editline-1.17.1

[Sergei Trofimovich]

Follow-up Comment #2, sr#110990 (group autoconf):

Yeah, adding extra quotes around the second argument of `AS_IF()` does work.
Thank you!


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #110990] AS_IF / AC_CHECK_LIB generates invalid shell on editline-1.17.1

[Sergei Trofimovich]

URL:
  <https://savannah.gnu.org/support/?110990>

 Summary: AS_IF / AC_CHECK_LIB generates invalid shell on
editline-1.17.1
   Group: Autoconf
   Submitter: slyfox
   Submitted: Sat 23 Dec 2023 06:24:04 PM UTC
Priority: 5 - Unprioritized
Severity: 3 - Normal
  Status: None
 Privacy: Public
 Assigned to: None
Originator Email: 
 Open/Closed: Open
 Discussion Lock: Any
Operating System: GNU/Linux


___

Follow-up Comments:


---
Date: Sat 23 Dec 2023 06:24:04 PM UTC By: Sergei Trofimovich 
Noticed seeming autoconf-2.72 regression when was building
https://github.com/troglobit/editline/blob/master/configure.ac#L92 in
`nixpkgs`.

Complete reproducer:


dnl cat configure.ac
AC_INIT(editline, 1.17.1, https://github.com/troglobit/editline/issues)

AC_CONFIG_SRCDIR([a.c])

AC_PROG_CC

AC_ARG_ENABLE(termcap,
   AS_HELP_STRING([--enable-termcap], [Use termcap library to query terminal
size.]))

# Check for a termcap compatible library if enabled
AS_IF([test "x$enable_termcap" = "xyes"],
   AC_CHECK_LIB(terminfo, tgetent, , [
  AC_CHECK_LIB(termcap, tgetent, , [
 AC_CHECK_LIB(tinfo, tgetent, , [
AC_CHECK_LIB(curses, tgetent, , [
   AC_CHECK_LIB(ncurses, tgetent, , [
  AC_MSG_ERROR([Cannot find a termcap capable library, try
installing Ncurses.])])
   ])
])
 ])
  ])
)

# Generate all files
AC_OUTPUT


Triggering:


$ autoreconf -ifv && touch a.c && ./configure

autoreconf: export WARNINGS=
autoreconf: Entering directory '.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: configure.ac: not using Intltool
autoreconf: configure.ac: not using Gtkdoc
autoreconf: running:
/nix/store/78q16yv4zyrary0xw9m6q1yi8i9h33bl-autoconf-2.72/bin/autoconf
--force
autoreconf: configure.ac: not using Autoheader
autoreconf: configure.ac: not using Automake
autoreconf: Leaving directory '.'
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
./configure: line 3476: syntax error near unexpected token `;;'
./configure: line 3476: ` ;;'









___

Reply to this item at:

  <https://savannah.gnu.org/support/?110990>

___
Message sent via Savannah
https://savannah.gnu.org/

[PATCH] tests: fix build against upcoming `gcc-14` (`-Werror=calloc-transposed-args`)

[Sergei Trofimovich]

`gcc-14` added a new `-Wcalloc-transposed-args` warning recently. It
detected minor infelicity in `calloc()` API usage in `elfutils`:

elfstrmerge.c: In function 'main':
elfstrmerge.c:450:32: error:
  'calloc' sizes specified with 'sizeof' in the earlier argument and not in 
the later argument [-Werror=calloc-transposed-args]
  450 |   newscnbufs = calloc (sizeof (void *), newshnums);
  |^~~~
elfstrmerge.c:450:32: note: earlier argument should specify number of 
elements, later size of each element

Signed-off-by: Sergei Trofimovich 
---
 tests/elfstrmerge.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/elfstrmerge.c b/tests/elfstrmerge.c
index 56350bb9..4eb58bbb 100644
--- a/tests/elfstrmerge.c
+++ b/tests/elfstrmerge.c
@@ -447,7 +447,7 @@ main (int argc, char **argv)
 }
 
   newshnums = shdrnum - 1;
-  newscnbufs = calloc (sizeof (void *), newshnums);
+  newscnbufs = calloc (newshnums, sizeof (void *));
   if (newscnbufs == NULL)
 fail_errno ("Couldn't allocate memory for new section buffers", NULL);
 
-- 
2.42.0

[PATCH] diagnostics: fix gcc-urlifier.cc bootstrap failure [PR112379]

[Sergei Trofimovich]

From: Sergei Trofimovich 

Without the change `./configure --enable-checking=release` bootstrap
fails as:

gcc/gcc-urlifier.cc:100:1: error: 'get_url_suffix_for_quoted_text()'
defined but not used [-Werror=unused-function]

This happens because the helper is used only in `ASSERT` macros which
don't always get expanded to executable code.

The fix marks helper function with `ATTRIBUTE_UNUSED`.

gcc/
PR bootstrap/112379
* gcc-urlifier.cc (get_url_suffix_for_quoted_text): Mark as
ATTRIBUTE_UNUSED.
---
 gcc/gcc-urlifier.cc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gcc/gcc-urlifier.cc b/gcc/gcc-urlifier.cc
index 269246bc703..0dbff985313 100644
--- a/gcc/gcc-urlifier.cc
+++ b/gcc/gcc-urlifier.cc
@@ -37,7 +37,8 @@ public:
   char *get_url_for_quoted_text (const char *p, size_t sz) const final 
override;
 
   const char *get_url_suffix_for_quoted_text (const char *p, size_t sz) const;
-  const char *get_url_suffix_for_quoted_text (const char *p) const;
+  /* We use ATTRIBUTE_UNUSED as this helper is called only from ASSERTs.  */
+  const char *get_url_suffix_for_quoted_text (const char *p) const 
ATTRIBUTE_UNUSED;
 
 private:
   static char *
-- 
2.42.0

[PATCH] libelf/elf_newscn.c: fix build failure against gcc-14 (-Walloc-size)

[Sergei Trofimovich]

`gcc-14` adde a new -Walloc-size warning that makes sure that size of an
individual element matches size of a pointed type:

https://gcc.gnu.org/PR71219

`elfutils` triggers is on `calloc()` call where member size is sued as
`1`.

elf_newscn.c: In function `elf_newscn`:
elf_newscn.c:97:12: error: allocation of insufficient size «1» for type 
«Elf_ScnList» with size «16» [-Werror=alloc-size]
   97 |   newp = calloc (sizeof (Elf_ScnList)
  |^

The change swaps arguments to pass larger value as a member size.

Signed-off-by: Sergei Trofimovich 
---
 libelf/elf_newscn.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libelf/elf_newscn.c b/libelf/elf_newscn.c
index d6bdf153..ec731f75 100644
--- a/libelf/elf_newscn.c
+++ b/libelf/elf_newscn.c
@@ -94,9 +94,9 @@ elf_newscn (Elf *elf)
  1
 #endif
  )
-  newp = calloc (sizeof (Elf_ScnList)
-+ ((elf->state.elf.scnincr *= 2)
-   * sizeof (Elf_Scn)), 1);
+  newp = calloc (1, sizeof (Elf_ScnList)
+   + ((elf->state.elf.scnincr *= 2)
+  * sizeof (Elf_Scn)));
   if (newp == NULL)
{
  __libelf_seterrno (ELF_E_NOMEM);
-- 
2.42.0

Re: [PATCH] libgcc: make heap-based trampolines conditional on libc presence

[Sergei Trofimovich]

On Mon, 23 Oct 2023 13:54:01 +0100
Iain Sandoe  wrote:

> hi Sergei,
> 
> > On 23 Oct 2023, at 13:43, Sergei Trofimovich  wrote:
> > 
> > From: Sergei Trofimovich 
> > 
> > To build `libc` for a target one needs to build `gcc` without `libc`
> > support first. Commit r14-4823-g8abddb187b3348 "libgcc: support
> > heap-based trampolines" added unconditional `libc` dependency and broke
> > libc-less `gcc` builds.
> > 
> > An example failure on `x86_64-unknown-linux-gnu`:
> > 
> >$ mkdir -p /tmp/empty
> >$ ../gcc/configure \
> >--disable-multilib \
> >--without-headers \
> >--with-newlib \
> >--enable-languages=c \
> >--disable-bootstrap \
> >--disable-gcov \
> >--disable-threads \
> >--disable-shared \
> >--disable-libssp \
> >--disable-libquadmath \
> >--disable-libgomp \
> >--disable-libatomic \
> >--with-build-sysroot=/tmp/empty
> >$ make
> >...
> >/tmp/gb/./gcc/xgcc -B/tmp/gb/./gcc/ 
> > -B/usr/local/x86_64-pc-linux-gnu/bin/ -B/usr/local/x86_64-pc-linux-gnu/lib/ 
> > -isystem /usr/local/x86_64-pc-linux-gnu/include -isystem 
> > /usr/local/x86_64-pc-linux-gnu/sys-include --sysroot=/tmp/empty   -g -O2 
> > -O2  -g -O2 -DIN_GCC   -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual 
> > -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition  -isystem 
> > ./include  -fpic -mlong-double-80 -DUSE_ELF_SYMVER -fcf-protection -mshstk 
> > -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector -Dinhibit_libc -fpic 
> > -mlong-double-80 -DUSE_ELF_SYMVER -fcf-protection -mshstk -I. -I. 
> > -I../.././gcc -I/home/slyfox/dev/git/gcc/libgcc 
> > -I/home/slyfox/dev/git/gcc/libgcc/. 
> > -I/home/slyfox/dev/git/gcc/libgcc/../gcc 
> > -I/home/slyfox/dev/git/gcc/libgcc/../include  -DHAVE_CC_TLS  -DUSE_TLS  -o 
> > heap-trampoline.o -MT heap-trampoline.o -MD -MP -MF heap-trampoline.dep  -c 
> > .../gcc/libgcc/config/i386/heap-trampoline.c -fvisibility=hidden 
> > -DHIDE_EXPORTS
> >../gcc/libgcc/config/i386/heap-trampoline.c:3:10: fatal error: unistd.h: 
> > No such file or directory
> >3 | #include 
> >  |  ^~
> >compilation terminated.
> >make[2]: *** [.../gcc/libgcc/static-object.mk:17: heap-trampoline.o] 
> > Error 1
> >make[2]: Leaving directory '/tmp/gb/x86_64-pc-linux-gnu/libgcc'
> >make[1]: *** [Makefile:13307: all-target-libgcc] Error 2
> > 
> > The change inhibits any heap-based trampoline code.  
> 
> That looks reasonable to me (I was considering using __has_include(), but the 
> inhibit_libc is neater).
> 
> The fact that this first compiler is buit without heap-trampoline support, 
> would become relevant, I guess if libc wanted to use them, it would need 
> another iteration.
> 
> so, it looks fine, but I cannot actually approve it.

Sounds good. Let's wait for others to chime in. Maybe Richard? :)

AFAIU libcs (like `glibc`) try hard not to use link tests and uses
mainly preprocessor and code generator to specifically accommodate this
case. Maybe there is a way to pass the support flag to libc without the
reliance on code presence in libgcc.

Otherwise we could use __builtin_trap() as an implementation for exposed
symbols.

> 
> > 
> > libgcc/
> > 
> > * libgcc/config/aarch64/heap-trampoline.c: Disable when libc is
> >   not present.
> > ---
> > libgcc/config/aarch64/heap-trampoline.c | 5 +
> > libgcc/config/i386/heap-trampoline.c| 5 +
> > 2 files changed, 10 insertions(+)
> > 
> > diff --git a/libgcc/config/aarch64/heap-trampoline.c 
> > b/libgcc/config/aarch64/heap-trampoline.c
> > index c8b83681ed7..f22233987ca 100644
> > --- a/libgcc/config/aarch64/heap-trampoline.c
> > +++ b/libgcc/config/aarch64/heap-trampoline.c
> > @@ -1,5 +1,8 @@
> > /* Copyright The GNU Toolchain Authors. */
> > 
> > +/* libc is required to allocate trampolines.  */
> > +#ifndef inhibit_libc
> > +
> > #include 
> > #include 
> > #include 
> > @@ -170,3 +173,5 @@ __builtin_nested_func_ptr_deleted (void)
> >   tramp_ctrl_curr = prev;
> > }
> > }
> > +
> > +#endif /* !inhibit_libc */
> > diff --git a/libgcc/config/i386/heap-trampoline.c 
> > b/libgcc/config/i386/heap-trampoline.c
> > index 96e13bf828e..4b9f4365868 100644
> > --- a/libgcc/config/i386/heap-trampoline.c
> > +++ b/libgcc/config/i386/heap-trampoline.c
> > @@ -1,5 +1,8 @@
> > /* Copyright The GNU Toolchain Authors. */
> > 
> > +/* libc is required to allocate trampolines.  */
> > +#ifndef inhibit_libc
> > +
> > #include 
> > #include 
> > #include 
> > @@ -170,3 +173,5 @@ __builtin_nested_func_ptr_deleted (void)
> >   tramp_ctrl_curr = prev;
> > }
> > }
> > +
> > +#endif /* !inhibit_libc */
> > -- 
> > 2.42.0
> >   
> 


-- 

  Sergei

[PATCH] libgcc: make heap-based trampolines conditional on libc presence

[Sergei Trofimovich]

From: Sergei Trofimovich 

To build `libc` for a target one needs to build `gcc` without `libc`
support first. Commit r14-4823-g8abddb187b3348 "libgcc: support
heap-based trampolines" added unconditional `libc` dependency and broke
libc-less `gcc` builds.

An example failure on `x86_64-unknown-linux-gnu`:

$ mkdir -p /tmp/empty
$ ../gcc/configure \
--disable-multilib \
--without-headers \
--with-newlib \
--enable-languages=c \
--disable-bootstrap \
--disable-gcov \
--disable-threads \
--disable-shared \
--disable-libssp \
--disable-libquadmath \
--disable-libgomp \
--disable-libatomic \
--with-build-sysroot=/tmp/empty
$ make
...
/tmp/gb/./gcc/xgcc -B/tmp/gb/./gcc/ -B/usr/local/x86_64-pc-linux-gnu/bin/ 
-B/usr/local/x86_64-pc-linux-gnu/lib/ -isystem 
/usr/local/x86_64-pc-linux-gnu/include -isystem 
/usr/local/x86_64-pc-linux-gnu/sys-include --sysroot=/tmp/empty   -g -O2 -O2  
-g -O2 -DIN_GCC   -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual 
-Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition  -isystem 
./include  -fpic -mlong-double-80 -DUSE_ELF_SYMVER -fcf-protection -mshstk -g 
-DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector -Dinhibit_libc -fpic 
-mlong-double-80 -DUSE_ELF_SYMVER -fcf-protection -mshstk -I. -I. -I../.././gcc 
-I/home/slyfox/dev/git/gcc/libgcc -I/home/slyfox/dev/git/gcc/libgcc/. 
-I/home/slyfox/dev/git/gcc/libgcc/../gcc 
-I/home/slyfox/dev/git/gcc/libgcc/../include  -DHAVE_CC_TLS  -DUSE_TLS  -o 
heap-trampoline.o -MT heap-trampoline.o -MD -MP -MF heap-trampoline.dep  -c 
.../gcc/libgcc/config/i386/heap-trampoline.c -fvisibility=hidden -DHIDE_EXPORTS
../gcc/libgcc/config/i386/heap-trampoline.c:3:10: fatal error: unistd.h: No 
such file or directory
3 | #include 
  |  ^~
compilation terminated.
make[2]: *** [.../gcc/libgcc/static-object.mk:17: heap-trampoline.o] Error 1
make[2]: Leaving directory '/tmp/gb/x86_64-pc-linux-gnu/libgcc'
make[1]: *** [Makefile:13307: all-target-libgcc] Error 2

The change inhibits any heap-based trampoline code.

libgcc/

* libgcc/config/aarch64/heap-trampoline.c: Disable when libc is
  not present.
---
 libgcc/config/aarch64/heap-trampoline.c | 5 +
 libgcc/config/i386/heap-trampoline.c| 5 +
 2 files changed, 10 insertions(+)

diff --git a/libgcc/config/aarch64/heap-trampoline.c 
b/libgcc/config/aarch64/heap-trampoline.c
index c8b83681ed7..f22233987ca 100644
--- a/libgcc/config/aarch64/heap-trampoline.c
+++ b/libgcc/config/aarch64/heap-trampoline.c
@@ -1,5 +1,8 @@
 /* Copyright The GNU Toolchain Authors. */
 
+/* libc is required to allocate trampolines.  */
+#ifndef inhibit_libc
+
 #include 
 #include 
 #include 
@@ -170,3 +173,5 @@ __builtin_nested_func_ptr_deleted (void)
   tramp_ctrl_curr = prev;
 }
 }
+
+#endif /* !inhibit_libc */
diff --git a/libgcc/config/i386/heap-trampoline.c 
b/libgcc/config/i386/heap-trampoline.c
index 96e13bf828e..4b9f4365868 100644
--- a/libgcc/config/i386/heap-trampoline.c
+++ b/libgcc/config/i386/heap-trampoline.c
@@ -1,5 +1,8 @@
 /* Copyright The GNU Toolchain Authors. */
 
+/* libc is required to allocate trampolines.  */
+#ifndef inhibit_libc
+
 #include 
 #include 
 #include 
@@ -170,3 +173,5 @@ __builtin_nested_func_ptr_deleted (void)
   tramp_ctrl_curr = prev;
 }
 }
+
+#endif /* !inhibit_libc */
-- 
2.42.0

[PATCH v4] ipa-utils: avoid uninitialized probabilities on ICF [PR111559]

[Sergei Trofimovich]

On Thu, Oct 05, 2023 at 03:04:55PM +0200, Jan Hubicka wrote:
> > diff --git a/gcc/ipa-utils.cc b/gcc/ipa-utils.cc
> > index 956c6294fd7..1355ccac6f0 100644
> > --- a/gcc/ipa-utils.cc
> > +++ b/gcc/ipa-utils.cc
> > @@ -651,13 +651,16 @@ ipa_merge_profiles (struct cgraph_node *dst,
> > {
> >   edge srce = EDGE_SUCC (srcbb, i);
> >   edge dste = EDGE_SUCC (dstbb, i);
> > - dste->probability = 
> > -   dste->probability * dstbb->count.ipa ().probability_in
> > -(dstbb->count.ipa ()
> > - + srccount.ipa ())
> > -   + srce->probability * srcbb->count.ipa ().probability_in
> > -(dstbb->count.ipa ()
> > - + srccount.ipa ());
> > + profile_count sum =
> > +   dstbb->count.ipa () + srccount.ipa ();
> > + if (sum.nonzero_p ())
> > +   dste->probability =
> > + dste->probability * dstbb->count.ipa ().probability_in
> > +  (dstbb->count.ipa ()
> > +   + srccount.ipa ())
> > + + srce->probability * srcbb->count.ipa ().probability_in
> > +  (dstbb->count.ipa ()
> > +   + srccount.ipa ());
> 
> looks good.  You can use probability_in (sum) 
> in both of the places.

Oh, great point! Completely forgot about it. Attached v4.

If it still looks reasonable I'll check again if `python` and
`profiledbootstrap` still survives it and will push.

-- 

  Sergei
>From cb9852216b5b2524f72964b399c133557ec98df0 Mon Sep 17 00:00:00 2001
From: Sergei Trofimovich 
Date: Wed, 27 Sep 2023 14:29:12 +0100
Subject: [PATCH v4] ipa-utils: avoid uninitialized probabilities on ICF
 [PR111559]

r14-3459-g0c78240fd7d519 "Check that passes do not forget to define profile"
exposed check failures in cases when gcc produces uninitialized profile
probabilities. In case of PR/111559 uninitialized profile is generated
by edges executed 0 times reported by IPA profile:

$ gcc -O2 -fprofile-generate pr111559.c -o b -fopt-info
$ ./b
$ gcc -O2 -fprofile-use -fprofile-correction pr111559.c -o b -fopt-info

pr111559.c: In function 'rule1':
pr111559.c:6:13: error: probability of edge 3->4 not initialized
6 | static void rule1(void) { if (p) edge(); }
  | ^
during GIMPLE pass: fixup_cfg
pr111559.c:6:13: internal compiler error: verify_flow_info failed

The change conservatively ignores updates with zero execution counts and
uses initially assigned probabilities (`always` probability in case of
the example).

PR ipa/111283
PR gcov-profile/111559

gcc/
* ipa-utils.cc (ipa_merge_profiles): Avoid producing
uninitialized probabilities when merging counters with zero
denominators.

gcc/testsuite/
* gcc.dg/tree-prof/pr111559.c: New test.
---
 gcc/ipa-utils.cc  | 15 ---
 gcc/testsuite/gcc.dg/tree-prof/pr111559.c | 16 
 2 files changed, 24 insertions(+), 7 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/tree-prof/pr111559.c

diff --git a/gcc/ipa-utils.cc b/gcc/ipa-utils.cc
index 956c6294fd7..6024ac69cc2 100644
--- a/gcc/ipa-utils.cc
+++ b/gcc/ipa-utils.cc
@@ -651,13 +651,14 @@ ipa_merge_profiles (struct cgraph_node *dst,
{
  edge srce = EDGE_SUCC (srcbb, i);
  edge dste = EDGE_SUCC (dstbb, i);
- dste->probability = 
-   dste->probability * dstbb->count.ipa ().probability_in
-(dstbb->count.ipa ()
- + srccount.ipa ())
-   + srce->probability * srcbb->count.ipa ().probability_in
-(dstbb->count.ipa ()
- + srccount.ipa ());
+ profile_count sum =
+   dstbb->count.ipa () + srccount.ipa ();
+ if (sum.nonzero_p ())
+   dste->probability =
+ dste->probability * dstbb->count.ipa ().probability_in
+  (sum)
+ + srce->probability * srcbb->count.ipa ().probability_in
+  (sum);
}
  dstbb->count = dstbb->count.ipa () + srccount.ipa ();
}
diff --git

Re: [PATCH v2] ipa-utils: avoid uninitialized probabilities on ICF [PR111559]

[Sergei Trofimovich]

On Thu, Oct 05, 2023 at 01:52:30PM +0200, Jan Hubicka wrote:
> > From: Sergei Trofimovich 
> > 
> > r14-3459-g0c78240fd7d519 "Check that passes do not forget to define profile"
> > exposed check failures in cases when gcc produces uninitialized profile
> > probabilities. In case of PR/111559 uninitialized profile is generated
> > by edges executed 0 times reported by IPA profile:
> > 
> > $ gcc -O2 -fprofile-generate pr111559.c -o b -fopt-info
> > $ ./b
> > $ gcc -O2 -fprofile-use -fprofile-correction pr111559.c -o b -fopt-info
> > 
> > pr111559.c: In function 'rule1':
> > pr111559.c:6:13: error: probability of edge 3->4 not initialized
> > 6 | static void rule1(void) { if (p) edge(); }
> >   | ^
> > during GIMPLE pass: fixup_cfg
> > pr111559.c:6:13: internal compiler error: verify_flow_info failed
> > 
> > The change conservatively ignores updates with uninitialized values and
> > uses initially assigned probabilities (`always` probability in case of
> > the example).
> > 
> > PR ipa/111283
> > PR gcov-profile/111559
> > 
> > gcc/
> > * ipa-utils.cc (ipa_merge_profiles): Avoid producing
> > uninitialized probabilities when merging counters with zero
> > denominators.
> > 
> > gcc/testsuite/
> > * gcc.dg/tree-prof/pr111559.c: New test.
> > ---
> >  gcc/ipa-utils.cc  |  6 +-
> >  gcc/testsuite/gcc.dg/tree-prof/pr111559.c | 16 
> >  2 files changed, 21 insertions(+), 1 deletion(-)
> >  create mode 100644 gcc/testsuite/gcc.dg/tree-prof/pr111559.c
> > 
> > diff --git a/gcc/ipa-utils.cc b/gcc/ipa-utils.cc
> > index 956c6294fd7..7c53ae9dd45 100644
> > --- a/gcc/ipa-utils.cc
> > +++ b/gcc/ipa-utils.cc
> > @@ -651,13 +651,17 @@ ipa_merge_profiles (struct cgraph_node *dst,
> > {
> >   edge srce = EDGE_SUCC (srcbb, i);
> >   edge dste = EDGE_SUCC (dstbb, i);
> > - dste->probability = 
> > + profile_probability merged =
> > dste->probability * dstbb->count.ipa ().probability_in
> >  (dstbb->count.ipa ()
> >   + srccount.ipa ())
> > + srce->probability * srcbb->count.ipa ().probability_in
> >  (dstbb->count.ipa ()
> >   + srccount.ipa ());
> > + /* We produce uninitialized probabilities when
> > +denominator is zero: https://gcc.gnu.org/PR111559.  */
> > + if (merged.initialized_p ())
> > +   dste->probability = merged;
> 
> Thanks for the patch.
> We usually avoid the uninitialized value here by simply checking that
> parameter of probability_in satifies nonzero_p.  So I think it would be
> more consistent doing it here to:
> 
>   profile_probability sum = dstbb->count.ipa () + srccount.ipa ()
>   if (sum.nonzero_p ())
>   {
>  dste->probability = .
>   }

Aha, sounds good! I had to do `s/profile_probability/profile_count` as
it's a denominator value for probability.

Attached v3 just in case.

> OK with this change.
> Honza
> > }
> >   dstbb->count = dstbb->count.ipa () + srccount.ipa ();
> > }
> > diff --git a/gcc/testsuite/gcc.dg/tree-prof/pr111559.c 
> > b/gcc/testsuite/gcc.dg/tree-prof/pr111559.c
> > new file mode 100644
> > index 000..43202c6c888
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.dg/tree-prof/pr111559.c
> > @@ -0,0 +1,16 @@
> > +/* { dg-options "-O2" } */
> > +
> > +__attribute__((noipa)) static void edge(void) {}
> > +
> > +int p = 0;
> > +
> > +__attribute__((noinline))
> > +static void rule1(void) { if (p) edge(); }
> > +
> > +__attribute__((noinline))
> > +static void rule1_same(void) { if (p) edge(); }
> > +
> > +__attribute__((noipa)) int main(void) {
> > +rule1();
> > +rule1_same();
> > +}
> > -- 
> > 2.42.0
> > 

-- 

  Sergei
>From 97122ebae5a7ed43b6c31574c761a54bee3a96ec Mon Sep 17 00:00:00 2001
From: Sergei Trofimovich 
Date: Wed, 27 Sep 2023 14:29:12 +0100
Subject: [PATCH v3] ipa-utils: avoid uninitialized probabilities on ICF
 [PR111559]

r14-3459-g0c78240fd7d519 "Check that passes do not forget to define profile"
exposed check failures in cases when gcc produces uninit

Re: Scaling -fmacro-prefix-map= to thousands entries

[Sergei Trofimovich via Gcc]

On Thu, Oct 05, 2023 at 07:20:35AM -0400, Ben Boeckel wrote:
> On Wed, Oct 04, 2023 at 22:19:32 +0100, Sergei Trofimovich via Gcc wrote:
> > The prototype that creates equivalent of the following commands does
> > work for smaller packages:
> > 
> > 
> > -fmacro-prefix-map=/nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev=/nix/store/-glibc-2.37-8-dev
> > 
> > -fmacro-prefix-map=/nix/store/8n240jfdmsb3lnc2qa2vb9dwk638j1lp-gmp-with-cxx-6.3.0-dev=/nix/store/-gmp-with-cxx-6.3.0-dev
> > 
> > -fmacro-prefix-map=/nix/store/phjcmy025rd1ankw5y1b21xsdii83cyk-nlohmann_json-3.11.2=/nix/store/-nlohmann_json-3.11.2
> > ...
> > 
> > The above works for small amount of options (like, 100). But around 1000
> > options we start hitting linux limits on the single environment variable
> > or real-world packages like `qemu` with a ton of input depends.
> 
> Are you trying to pass this through via `CFLAGS` and friends?

Roughly via `CFLAGS`. `nixpkgs` uses it's own private
`NIX_CFLAGS_COMPILE` variable which gets extended in `gcc-wrapper` shel
wrapper as expcit list of arguments to real `gcc-binary`. It's almost
like `CFLAGS` but is expected to be transaparent to most build systems.

> > The command-line limitations are in various places:
> > - `gcc` limitation of lifting all command line options into a single
> >   environment variable: https://gcc.gnu.org/PR111527
> > - `linux` limitation of constraining single environ variable to a value
> >   way below than full available environment space:
> >   https://lkml.org/lkml/2023/9/24/381
> > 
> > `linux` fix would buy us 50x more budged (A Lot) but it will not help
> > much other operating systems like `Darwin` where absolute environment
> > limit is a lot lower than `linux`.
> > 
> > I already implemented [1.] in https://github.com/NixOS/nixpkgs/pull/255192
> > (also attached `mangle-NIX_STORE-in-__FILE__.patch` 3.5K patch against
> > `master` as a proof of concept).
> > 
> > What would be the best way to scale up `-fmacro-prefix-map=` up to NixOS
> > needs for `gcc`? I would like to implement something sensible I could
> > upstream.
> 
> How about `CFLAGS=@macro_prefix_map.args` and writing that file in the
> same codepath where you generate the flags today. It'll work with just
> about every compiler and tools like `ccache` will understand that it is
> an input that affects the build and properly take the file's contents
> into account.

That was my initial attempt. I'll duplicate my response from 
https://gcc.gnu.org/pipermail/gcc/2023-October/242639.html here as is:

"""
Yeah, in theory response files would extend the limit. In practice `gcc`
always extends response files internally into a single
`COLLECT_GCC_OPTIONS` option and hits the environment variable limit
very early:

https://gcc.gnu.org/PR111527

Example reproducer:

$ for i in `seq 1 1000`; do printf -- "-fmacro-prefix-map=%0*d=%0*d\n" 200 
1 200 2; done > a.rsp
$ touch a.c; gcc @a.rsp -c a.c
gcc: fatal error: cannot execute 'cc1': execv: Argument list too long
compilation terminated.

And if you want to look at the gory details:

$ strace -f -etrace=execve -s 100 -v -v -v  gcc @a.rsp -c a.c
...
[pid78] execve("cc1", ["cc1", "-quiet", "a.c", "-quiet", "-dumpbase", 
"a.c", "-dumpbase-ext", ".c", "-mtune=generic", "-march=x86-64",
"-fmacro-prefix-map=...=...",
"-fmacro-prefix-map=...=...",
...],
[...,
 "COLLECT_GCC=gcc",
 "COLLECT_GCC_OPTIONS='-fmacro-prefix-map=...=...' 
'-fmacro-prefix-map=...=...' ... '-c' '-mtune=generic' '-march=x86-64'"]) = -1 
E2BIG (Argument list too long)

Note how `gcc` not only expands response file into an argument list
(that is not too bad) but also duplicates the whole list as a single
`COLLECT_GCC_OPTIONS=...` environment variable with added quoting on
top.

Would be nice if `gcc` just passed response files around as is :)
"""

-- 

  Sergei

Re: Scaling -fmacro-prefix-map= to thousands entries

[Sergei Trofimovich via Gcc]

On Thu, Oct 05, 2023 at 09:19:15AM +0200, Richard Biener wrote:
> On Wed, Oct 4, 2023 at 11:20 PM Sergei Trofimovich via Gcc
>  wrote:
> >
> > Hi gcc developers!
> >
> > Tl;DR:
> >
> > I would like to implement a scalable way to pass `-fmacro-prefix-map=`
> > for `NixOS` distribution to avoid leaking build-time paths generated by
> > `__FILE__` macros used by various libraries.
> >
> > I need some guidance what path to take to be acceptable for `gcc`
> > upstream.
> >
> > I have a few possible solutions and wonder what I should try to upstream
> > to GCC. The options I see:
> >
> > 1. Hardcode NixOS-specific way to mangle paths.
> >
> >Pros: simplest to implement, can be easily configured away if needed
> >Cons: inflexible, `clang` might or might not accept the same hack
> >
> > 2. Extend `-fmacro-prefix-map=` (or add a new `-fmacro-prefix-map-file=`)
> >to allow passing a file
> >
> >Pros: still not too hard to implement, generic enough to be used in
> >  other contexts.
> >Cons: Will require client to construct the map file.
> >
> > 3. Have more flexible `-fmacro-prefix-map-regex=` option that allows
> >patterns. Something like:
> >
> >   
> > -fmacro-prefix-map-regex=/nix/store/[a-z0-9]{32}-=/nix/store/-
> >
> >   Pros: at least for NixOS one option will be enough to cover all
> > packages as they all share above template.
> >   Cons: pulls some form of regex with it's can of worms including escape
> > delimiters, might not be flexible enough for other use cases.
> >
> > 4. Something else?
> >
> > Which one(s) should I take to implement?
> >
> > More words:
> >
> > `NixOS` (and `nixpkgs` repository) install every software package into
> > an individual directory with unique prefix. Some examples:
> >
> > /nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev
> > /nix/store/rb3q4kcyfg77cmkiwywx2aqdd3x5ch93-libmpc-1.3.1
> > /nix/store/8n240jfdmsb3lnc2qa2vb9dwk638j1lp-gmp-with-cxx-6.3.0-dev
> > /nix/store/phjcmy025rd1ankw5y1b21xsdii83cyk-nlohmann_json-3.11.2
> > ...
> >
> > It's a fundamental design decision to allow parallel package installs.
> >
> > From dependency tracking standpoint it's highly undesirable to have
> > these absolute paths to be hardcoded into final executable binaries if
> > they are not used at runtime.
> >
> > Example redundant path we would like not to have in final binaries:
> >
> > $ strings result/bin/nix | grep phjcmy025rd1ankw5y1b21xsdii83cyk
> > 
> > /nix/store/phjcmy025rd1ankw5y1b21xsdii83cyk-nlohmann_json-3.11.2/include/nlohmann/json.hpp
> > 
> > /nix/store/phjcmy025rd1ankw5y1b21xsdii83cyk-nlohmann_json-3.11.2/include/nlohmann/detail/output/serializer.hpp
> > 
> > /nix/store/phjcmy025rd1ankw5y1b21xsdii83cyk-nlohmann_json-3.11.2/include/nlohmann/detail/conversions/to_chars.hpp
> > 
> > /nix/store/phjcmy025rd1ankw5y1b21xsdii83cyk-nlohmann_json-3.11.2/include/nlohmann/detail/input/lexer.hpp
> > 
> > /nix/store/phjcmy025rd1ankw5y1b21xsdii83cyk-nlohmann_json-3.11.2/include/nlohmann/detail/iterators/iter_impl.hpp
> > 
> > /nix/store/phjcmy025rd1ankw5y1b21xsdii83cyk-nlohmann_json-3.11.2/include/nlohmann/detail/input/json_sax.hpp
> > 
> > /nix/store/phjcmy025rd1ankw5y1b21xsdii83cyk-nlohmann_json-3.11.2/include/nlohmann/detail/iterators/iteration_proxy.hpp
> > 
> > /nix/store/phjcmy025rd1ankw5y1b21xsdii83cyk-nlohmann_json-3.11.2/include/nlohmann/detail/input/parser.hpp
> >
> > Those paths are inserted via glibc's assert() uses of `__FILE__`
> > directive and thus hardcode header file paths from various packages
> > (like lttng-ust or nlohmann/json) into compiled binaries. Sometimes
> > `__FILE__` usage is mire creating than assert().
> >
> > I would like to get rid of references to header files. I think
> > `-fmacro-prefix-map=` are ideal for this particular use case.
> >
> > The prototype that creates equivalent of the following commands does
> > work for smaller packages:
> >
> > 
> > -fmacro-prefix-map=/nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev=/nix/store/-glibc-2.37-8-dev
> > 
> > -fmacro-prefix-map=/nix/store/8n240jfdmsb3lnc2qa2vb9dwk638j1lp-gmp-with-cxx-6.3.0-dev=/nix/store/-gmp-with-cxx-6.3.0-dev
> > 
> > -fmacro-prefix-map=/nix/store/phjcmy025rd1ankw5y1b21xsdii83

Scaling -fmacro-prefix-map= to thousands entries

[Sergei Trofimovich via Gcc]

 for `gcc`? I would like to implement something sensible I could
upstream.

What do you think?

Thanks!

-- 

  Sergei
>From b10785c1be469319a09b10bc69db21159b0599ee Mon Sep 17 00:00:00 2001
From: Sergei Trofimovich 
Date: Fri, 22 Sep 2023 22:41:49 +0100
Subject: [PATCH] gcc/file-prefix-map.cc: always mangle __FILE__ into invalid
 store path

Without the change `__FILE__` used in static inline functions in headers
embed paths to header files into executable images. For local headers
it's not a problem, but for headers in `/nix/store` this causes `-dev`
inputs to be retained in runtime closure.

Typical examples are `nix` -> `nlonhamm_json` and `pipewire` ->
`lttng-ust.dev`.

Ideally we would like to use `-fmacro-prefix-map=` feature of `gcc` as:

  -fmacro-prefix-map=/nix/store/$hash1-nlohmann-json-ver=/nix/store/.eee-nlohmann-json-ver
  -fmacro-prefix-map=/nix/...

IN practice it quickly exhausts argument lengtth limit due to `gcc`
deficiency: https://gcc.gnu.org/PR111527

Until it;s fixed let's hardcode header mangling if $NIX_STORE variable
is present in the environment.

Tested as:

$ printf "# 0 \"/nix/store/01234567890123456789012345678901-pp-vvv\" \nconst char * f(void) { return __FILE__; }" | NIX_STORE=/nix/store ./gcc/xgcc -Bgcc -x c - -S -o -
...
.string "/nix/store/-pp-vvv"
...

Mangled successfully.
---
 gcc/file-prefix-map.cc | 27 ++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/gcc/file-prefix-map.cc b/gcc/file-prefix-map.cc
index 0e6db7c142a..da39404b9cd 100644
--- a/gcc/file-prefix-map.cc
+++ b/gcc/file-prefix-map.cc
@@ -69,6 +69,9 @@ add_prefix_map (file_prefix_map *, const char *arg, const char *opt)
   maps = map;
 }
 
+/* Forward declatration for a $NIX_STORE remap hack below. */
+static file_prefix_map *macro_prefix_maps; /* -fmacro-prefix-map  */
+
 /* Perform user-specified mapping of filename prefixes.  Return the
GC-allocated new name corresponding to FILENAME or FILENAME if no
remapping was performed.  */
@@ -102,6 +105,29 @@ remap_filename (file_prefix_map *maps, const char *filename)
   break;
   if (!map)
 {
+  if (maps == macro_prefix_maps)
+	{
+	  /* Remap all fo $NIX_STORE/.{32} paths to
+	   * equivalent $NIX_STORE/e{32}.
+	   *
+	   * That way we avoid argument parameters explosion
+	   * and still avoid embedding headers into runtime closure:
+	   *   https://gcc.gnu.org/PR111527
+	   */
+	   char * nix_store = getenv("NIX_STORE");
+	   size_t nix_store_len = nix_store ? strlen(nix_store) : 0;
+	   const char * name = realname ? realname : filename;
+	   size_t name_len = strlen(name);
+	   if (nix_store && name_len >= nix_store_len + 1 + 32 && memcmp(name, nix_store, nix_store_len) == 0)
+	 {
+		s = (char *) ggc_alloc_atomic (name_len + 1);
+		memcpy(s, name, name_len + 1);
+		memset(s + nix_store_len + 1, 'e', 32);
+		if (realname != filename)
+		  free (const_cast  (realname));
+		return s;
+	 }
+	}
   if (realname != filename)
 	free (const_cast  (realname));
   return filename;
@@ -124,7 +150,6 @@ remap_filename (file_prefix_map *maps, const char *filename)
ignore it in DW_AT_producer (gen_command_line_string in opts.cc).  */
 
 /* Linked lists of file_prefix_map structures.  */
-static file_prefix_map *macro_prefix_maps; /* -fmacro-prefix-map  */
 static file_prefix_map *debug_prefix_maps; /* -fdebug-prefix-map  */
 static file_prefix_map *profile_prefix_maps; /* -fprofile-prefix-map  */
 
-- 
2.42.0

[PATCH] Makefile.tpl: disable -Werror for feedback stage [PR111663]

[Sergei Trofimovich]

From: Sergei Trofimovich 

Without the change profiled bootstrap fails for various warnings on
master branch as:

$ ../gcc/configure
$ make profiledbootstrap
...
gcc/genmodes.cc: In function ‘int main(int, char**)’:
gcc/genmodes.cc:2152:1: error: ‘gcc/build/genmodes.gcda’ profile count data 
file not found [-Werror=missing-profile]
...
gcc/gengtype-parse.cc: In function ‘void parse_error(const char*, ...)’:
gcc/gengtype-parse.cc:142:21: error: ‘%s’ directive argument is null 
[-Werror=format-overflow=]

The change removes -Werror just like autofeedback does today.

/

PR bootstrap/111663
* Makefile.tpl (STAGEfeedback_CONFIGURE_FLAGS): Disable -Werror.
* Makefile.in: Regenerate.
---
 Makefile.in  | 4 
 Makefile.tpl | 4 
 2 files changed, 8 insertions(+)

diff --git a/Makefile.in b/Makefile.in
index 2f136839c35..e0e3c4c8fe8 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -638,6 +638,10 @@ STAGEtrain_TFLAGS = $(filter-out 
-fchecking=1,$(STAGE3_TFLAGS))
 
 STAGEfeedback_CFLAGS = $(STAGE4_CFLAGS) -fprofile-use 
-fprofile-reproducible=parallel-runs
 STAGEfeedback_TFLAGS = $(STAGE4_TFLAGS)
+# Disable warnings as errors for a few reasons:
+# - sources for gen* binaries do not have .gcda files available
+# - inlining decisions generate extra warnings
+STAGEfeedback_CONFIGURE_FLAGS = $(filter-out 
--enable-werror-always,$(STAGE_CONFIGURE_FLAGS))
 
 STAGEautoprofile_CFLAGS = $(filter-out -gtoggle,$(STAGE2_CFLAGS)) -g
 STAGEautoprofile_TFLAGS = $(STAGE2_TFLAGS)
diff --git a/Makefile.tpl b/Makefile.tpl
index 5872dd03f2c..8b7783bb4f1 100644
--- a/Makefile.tpl
+++ b/Makefile.tpl
@@ -561,6 +561,10 @@ STAGEtrain_TFLAGS = $(filter-out 
-fchecking=1,$(STAGE3_TFLAGS))
 
 STAGEfeedback_CFLAGS = $(STAGE4_CFLAGS) -fprofile-use 
-fprofile-reproducible=parallel-runs
 STAGEfeedback_TFLAGS = $(STAGE4_TFLAGS)
+# Disable warnings as errors for a few reasons:
+# - sources for gen* binaries do not have .gcda files available
+# - inlining decisions generate extra warnings
+STAGEfeedback_CONFIGURE_FLAGS = $(filter-out 
--enable-werror-always,$(STAGE_CONFIGURE_FLAGS))
 
 STAGEautoprofile_CFLAGS = $(filter-out -gtoggle,$(STAGE2_CFLAGS)) -g
 STAGEautoprofile_TFLAGS = $(STAGE2_TFLAGS)
-- 
2.42.0

[PATCH v2] ipa-utils: avoid uninitialized probabilities on ICF [PR111559]

[Sergei Trofimovich]

From: Sergei Trofimovich 

r14-3459-g0c78240fd7d519 "Check that passes do not forget to define profile"
exposed check failures in cases when gcc produces uninitialized profile
probabilities. In case of PR/111559 uninitialized profile is generated
by edges executed 0 times reported by IPA profile:

$ gcc -O2 -fprofile-generate pr111559.c -o b -fopt-info
$ ./b
$ gcc -O2 -fprofile-use -fprofile-correction pr111559.c -o b -fopt-info

pr111559.c: In function 'rule1':
pr111559.c:6:13: error: probability of edge 3->4 not initialized
6 | static void rule1(void) { if (p) edge(); }
  | ^
during GIMPLE pass: fixup_cfg
pr111559.c:6:13: internal compiler error: verify_flow_info failed

The change conservatively ignores updates with uninitialized values and
uses initially assigned probabilities (`always` probability in case of
the example).

PR ipa/111283
PR gcov-profile/111559

gcc/
* ipa-utils.cc (ipa_merge_profiles): Avoid producing
uninitialized probabilities when merging counters with zero
denominators.

gcc/testsuite/
* gcc.dg/tree-prof/pr111559.c: New test.
---
 gcc/ipa-utils.cc  |  6 +-
 gcc/testsuite/gcc.dg/tree-prof/pr111559.c | 16 
 2 files changed, 21 insertions(+), 1 deletion(-)
 create mode 100644 gcc/testsuite/gcc.dg/tree-prof/pr111559.c

diff --git a/gcc/ipa-utils.cc b/gcc/ipa-utils.cc
index 956c6294fd7..7c53ae9dd45 100644
--- a/gcc/ipa-utils.cc
+++ b/gcc/ipa-utils.cc
@@ -651,13 +651,17 @@ ipa_merge_profiles (struct cgraph_node *dst,
{
  edge srce = EDGE_SUCC (srcbb, i);
  edge dste = EDGE_SUCC (dstbb, i);
- dste->probability = 
+ profile_probability merged =
dste->probability * dstbb->count.ipa ().probability_in
 (dstbb->count.ipa ()
  + srccount.ipa ())
+ srce->probability * srcbb->count.ipa ().probability_in
 (dstbb->count.ipa ()
  + srccount.ipa ());
+ /* We produce uninitialized probabilities when
+denominator is zero: https://gcc.gnu.org/PR111559.  */
+ if (merged.initialized_p ())
+   dste->probability = merged;
}
  dstbb->count = dstbb->count.ipa () + srccount.ipa ();
}
diff --git a/gcc/testsuite/gcc.dg/tree-prof/pr111559.c 
b/gcc/testsuite/gcc.dg/tree-prof/pr111559.c
new file mode 100644
index 000..43202c6c888
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/tree-prof/pr111559.c
@@ -0,0 +1,16 @@
+/* { dg-options "-O2" } */
+
+__attribute__((noipa)) static void edge(void) {}
+
+int p = 0;
+
+__attribute__((noinline))
+static void rule1(void) { if (p) edge(); }
+
+__attribute__((noinline))
+static void rule1_same(void) { if (p) edge(); }
+
+__attribute__((noipa)) int main(void) {
+rule1();
+rule1_same();
+}
-- 
2.42.0

[PATCH v2] rtl: fix build failure on -fchecking=2 [PR111642]

[Sergei Trofimovich]

From: Sergei Trofimovich 

Before the change `make bootstrap4` (or `make profiledbootstrap`) failed
as:

gcc/rtl-tests.cc:249:25:   in ‘constexpr’ expansion of ‘poly_int<1, long 
int>(1, 1)’
gcc/poly-int.h:453:5: error: too many initializers for ‘long int [1]’

The failure happened only in stage4 due to a gcc bug which accepts
invalid C++ without `-fchecking=2`: https://gcc.gnu.org/PR111647#c1.

The change hides tests that rely on 2 and more polynomials under `#if`.

gcc/
PR bootstrap/111642
* rtl-tests.cc (const_poly_int_tests): Guard tests with
NUM_POLY_INT_COEFFS > 1.
* simplify-rtx.cc (simplify_const_poly_int_tests): Ditto.
---
 gcc/rtl-tests.cc| 23 +++
 gcc/simplify-rtx.cc | 21 ++---
 2 files changed, 13 insertions(+), 31 deletions(-)

diff --git a/gcc/rtl-tests.cc b/gcc/rtl-tests.cc
index ae8669419b6..d4130572501 100644
--- a/gcc/rtl-tests.cc
+++ b/gcc/rtl-tests.cc
@@ -228,24 +228,14 @@ test_uncond_jump ()
  jump_insn);
 }
 
-template
-struct const_poly_int_tests
-{
-  static void run ();
-};
-
-template<>
-struct const_poly_int_tests<1>
-{
-  static void run () {}
-};
-
 /* Test various CONST_POLY_INT properties.  */
 
-template
-void
-const_poly_int_tests::run ()
+static void
+const_poly_int_tests ()
 {
+  /* `poly_int64` call with two parameters requires target with at
+   least 2 COEFFs.  */
+#if NUM_POLY_INT_COEFFS > 1
   rtx x1 = gen_int_mode (poly_int64 (1, 1), QImode);
   rtx x255 = gen_int_mode (poly_int64 (1, 255), QImode);
 
@@ -282,6 +272,7 @@ const_poly_int_tests::run ()
 gen_int_mode (poly_int64 (5, -2), QImode));
   ASSERT_EQ (plus_constant (QImode, x1, poly_int64 (4, -2)),
 gen_int_mode (poly_int64 (5, -1), QImode));
+#endif
 }
 
 /* Check dumping of repeated RTL vectors.  */
@@ -317,7 +308,7 @@ rtl_tests_cc_tests ()
   test_dumping_rtx_reuse ();
   test_single_set ();
   test_uncond_jump ();
-  const_poly_int_tests::run ();
+  const_poly_int_tests ();
   test_dumping_repeat ();
 
   /* Purge state.  */
diff --git a/gcc/simplify-rtx.cc b/gcc/simplify-rtx.cc
index 170406aa28b..39361b11b1a 100644
--- a/gcc/simplify-rtx.cc
+++ b/gcc/simplify-rtx.cc
@@ -8671,24 +8671,14 @@ test_vector_ops ()
 }
 }
 
-template
-struct simplify_const_poly_int_tests
-{
-  static void run ();
-};
-
-template<>
-struct simplify_const_poly_int_tests<1>
-{
-  static void run () {}
-};
-
 /* Test various CONST_POLY_INT properties.  */
 
-template
 void
-simplify_const_poly_int_tests::run ()
+simplify_const_poly_int_tests ()
 {
+  /* `poly_int64` call with two parameters requires target with at
+   least 2 COEFFs.  */
+#if NUM_POLY_INT_COEFFS > 1
   rtx x1 = gen_int_mode (poly_int64 (1, 1), QImode);
   rtx x2 = gen_int_mode (poly_int64 (-80, 127), QImode);
   rtx x3 = gen_int_mode (poly_int64 (-79, -128), QImode);
@@ -8716,6 +8706,7 @@ simplify_const_poly_int_tests::run ()
   ASSERT_EQ (simplify_binary_operation (IOR, QImode, x4, two), x7);
   ASSERT_EQ (simplify_subreg (HImode, x5, QImode, 0), x8);
   ASSERT_EQ (simplify_subreg (QImode, x8, HImode, offset), x5);
+#endif
 }
 
 /* Run all of the selftests within this file.  */
@@ -8725,7 +8716,7 @@ simplify_rtx_cc_tests ()
 {
   test_scalar_ops ();
   test_vector_ops ();
-  simplify_const_poly_int_tests::run ();
+  simplify_const_poly_int_tests ();
 }
 
 } // namespace selftest
-- 
2.42.0

[PATCH] rtl: fix buidl failure on -fchecking=2 [PR111642]

[Sergei Trofimovich]

From: Sergei Trofimovich 

Before the change `make bootstrap4` (or `make profiledbootstrap`) failed
as:

gcc/rtl-tests.cc:249:25:   in ‘constexpr’ expansion of ‘poly_int<1, long 
int>(1, 1)’
gcc/poly-int.h:453:5: error: too many initializers for ‘long int [1]’

The failure happened only in stage4 due to a gcc bug which accepts
invalid C++ without `-fchecking=2`: https://gcc.gnu.org/PR111647#c1.

The change hides tests that rely on 2 and more polynomials under `#if`.

gcc/
PR bootstrap/111642
* rtl-tests.cc (const_poly_int_tests): Guard tests with
NUM_POLY_INT_COEFFS > 1.
* simplify-rtx.cc (simplify_const_poly_int_tests): Ditto.
---
 gcc/rtl-tests.cc| 25 -
 gcc/simplify-rtx.cc | 21 ++---
 2 files changed, 14 insertions(+), 32 deletions(-)

diff --git a/gcc/rtl-tests.cc b/gcc/rtl-tests.cc
index ae8669419b6..85c1e56048d 100644
--- a/gcc/rtl-tests.cc
+++ b/gcc/rtl-tests.cc
@@ -228,24 +228,14 @@ test_uncond_jump ()
  jump_insn);
 }
 
-template
-struct const_poly_int_tests
-{
-  static void run ();
-};
-
-template<>
-struct const_poly_int_tests<1>
-{
-  static void run () {}
-};
-
 /* Test various CONST_POLY_INT properties.  */
 
-template
-void
-const_poly_int_tests::run ()
+static void
+const_poly_int_tests ()
 {
+  /* `poly_int64` call with two parameters requires target with at
+   least 2 COEFFs.  */
+#if NUM_POLY_INT_COEFFS > 1
   rtx x1 = gen_int_mode (poly_int64 (1, 1), QImode);
   rtx x255 = gen_int_mode (poly_int64 (1, 255), QImode);
 
@@ -263,7 +253,7 @@ const_poly_int_tests::run ()
   ASSERT_KNOWN_EQ (rtx_to_poly_int64 (x255), poly_int64 (1, -1));
   ASSERT_MAYBE_NE (rtx_to_poly_int64 (x255), poly_int64 (1, 255));
 
-  /* Test plus_constant of a symbol.  */
+  /* Test plus_consctant of a symbol.  */
   rtx symbol = gen_rtx_SYMBOL_REF (Pmode, "foo");
   rtx offset1 = gen_int_mode (poly_int64 (9, 11), Pmode);
   rtx sum1 = gen_rtx_CONST (Pmode, gen_rtx_PLUS (Pmode, symbol, offset1));
@@ -282,6 +272,7 @@ const_poly_int_tests::run ()
 gen_int_mode (poly_int64 (5, -2), QImode));
   ASSERT_EQ (plus_constant (QImode, x1, poly_int64 (4, -2)),
 gen_int_mode (poly_int64 (5, -1), QImode));
+#endif
 }
 
 /* Check dumping of repeated RTL vectors.  */
@@ -317,7 +308,7 @@ rtl_tests_cc_tests ()
   test_dumping_rtx_reuse ();
   test_single_set ();
   test_uncond_jump ();
-  const_poly_int_tests::run ();
+  const_poly_int_tests ();
   test_dumping_repeat ();
 
   /* Purge state.  */
diff --git a/gcc/simplify-rtx.cc b/gcc/simplify-rtx.cc
index 170406aa28b..39361b11b1a 100644
--- a/gcc/simplify-rtx.cc
+++ b/gcc/simplify-rtx.cc
@@ -8671,24 +8671,14 @@ test_vector_ops ()
 }
 }
 
-template
-struct simplify_const_poly_int_tests
-{
-  static void run ();
-};
-
-template<>
-struct simplify_const_poly_int_tests<1>
-{
-  static void run () {}
-};
-
 /* Test various CONST_POLY_INT properties.  */
 
-template
 void
-simplify_const_poly_int_tests::run ()
+simplify_const_poly_int_tests ()
 {
+  /* `poly_int64` call with two parameters requires target with at
+   least 2 COEFFs.  */
+#if NUM_POLY_INT_COEFFS > 1
   rtx x1 = gen_int_mode (poly_int64 (1, 1), QImode);
   rtx x2 = gen_int_mode (poly_int64 (-80, 127), QImode);
   rtx x3 = gen_int_mode (poly_int64 (-79, -128), QImode);
@@ -8716,6 +8706,7 @@ simplify_const_poly_int_tests::run ()
   ASSERT_EQ (simplify_binary_operation (IOR, QImode, x4, two), x7);
   ASSERT_EQ (simplify_subreg (HImode, x5, QImode, 0), x8);
   ASSERT_EQ (simplify_subreg (QImode, x8, HImode, offset), x5);
+#endif
 }
 
 /* Run all of the selftests within this file.  */
@@ -8725,7 +8716,7 @@ simplify_rtx_cc_tests ()
 {
   test_scalar_ops ();
   test_vector_ops ();
-  simplify_const_poly_int_tests::run ();
+  simplify_const_poly_int_tests ();
 }
 
 } // namespace selftest
-- 
2.42.0

[PATCH] ggc: do not wipe out unrelated data via gt_ggc_rtab

[Sergei Trofimovich]

From: Sergei Trofimovich 

There are 3 GC root tables:

   gt_ggc_rtab
   gt_ggc_deletable_rtab
   gt_pch_scalar_rtab

`deletable` and `scalar` tables are both simple: each element always
contains a pointer to the beginning of the object and it's size is the
full object.

`rtab` is different: it's `base` is a pointer in the middle of the
struct and `stride` points to the next GC pointer in the array.

Before the change there were 2 problems:

1. We memset()ed not just pointers but data around them.
2. We wen out of bounds of the last object described by gt_ggc_rtab
   and triggered bootstrap failures in profile and asan bootstraps.

After the change we handle only pointers themselves like the rest of
ggc-common.cc code.

gcc/
PR/111505
* ggc-common.cc (ggc_zero_out_root_pointers): New helper.
* ggc-common.cc (ggc_common_finalize): Use helper instead of
memset() to wipe out pointers.
---
 gcc/ggc-common.cc | 15 +--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/gcc/ggc-common.cc b/gcc/ggc-common.cc
index 95803fa95a1..39e2581affd 100644
--- a/gcc/ggc-common.cc
+++ b/gcc/ggc-common.cc
@@ -75,6 +75,18 @@ ggc_mark_root_tab (const_ggc_root_tab_t rt)
   (*rt->cb) (*(void **) ((char *)rt->base + rt->stride * i));
 }
 
+/* Zero out all the roots in the table RT.  */
+
+static void
+ggc_zero_rtab_roots (const_ggc_root_tab_t rt)
+{
+  size_t i;
+
+  for ( ; rt->base != NULL; rt++)
+for (i = 0; i < rt->nelt; i++)
+  (*(void **) ((char *)rt->base + rt->stride * i)) = (void*)0;
+}
+
 /* Iterate through all registered roots and mark each element.  */
 
 void
@@ -1307,8 +1319,7 @@ ggc_common_finalize ()
   memset (rti->base, 0, rti->stride * rti->nelt);
 
   for (rt = gt_ggc_rtab; *rt; rt++)
-for (rti = *rt; rti->base != NULL; rti++)
-  memset (rti->base, 0, rti->stride * rti->nelt);
+ggc_zero_rtab_roots (*rt);
 
   for (rt = gt_pch_scalar_rtab; *rt; rt++)
 for (rti = *rt; rti->base != NULL; rti++)
-- 
2.42.0

[PATCH] ipa-utils: avoid generating uninitialized probabilities on merges.

[Sergei Trofimovich]

From: Sergei Trofimovich 

r14-3459-g0c78240fd7d519 "Check that passes do not forget to define profile"
exposed check failures in cases when gcc produces uninitialized profile
probabilities. In case of PR/111559 uninitialized profile is generated
by edges executed 0 times during profile:

__attribute__((noipa)) static void edge(void) {}

int p = 0;

__attribute__((noinline))
static void rule1(void) { if (p) edge(); }

__attribute__((noinline))
static void rule1_same(void) { if (p) edge(); }

__attribute__((noipa)) int main(void) {
rule1();
rule1_same();
}

$ gcc -O2 -fprofile-generate bug.c -o b -fopt-info
$ ./b
$ gcc -O2 -fprofile-use -fprofile-correction bug.c -o b -fopt-info

bug.c: In function 'rule1':
bug.c:6:13: error: probability of edge 3->4 not initialized
6 | static void rule1(void) { if (p) edge(); }
  | ^
during GIMPLE pass: fixup_cfg
bug.c:6:13: internal compiler error: verify_flow_info failed

The change conservatively ignores updates with uninitialized values and
uses initially assigned probabilities (`always` probability in case of
the example).

gcc/
PR/111283
PR/111559
* ipa-utils.cc (ipa_merge_profiles): Avoid producing
  uninitialized probabilities when merging counters with zero
  denominators.
---
 gcc/ipa-utils.cc | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/gcc/ipa-utils.cc b/gcc/ipa-utils.cc
index 956c6294fd7..7c53ae9dd45 100644
--- a/gcc/ipa-utils.cc
+++ b/gcc/ipa-utils.cc
@@ -651,13 +651,17 @@ ipa_merge_profiles (struct cgraph_node *dst,
{
  edge srce = EDGE_SUCC (srcbb, i);
  edge dste = EDGE_SUCC (dstbb, i);
- dste->probability = 
+ profile_probability merged =
dste->probability * dstbb->count.ipa ().probability_in
 (dstbb->count.ipa ()
  + srccount.ipa ())
+ srce->probability * srcbb->count.ipa ().probability_in
 (dstbb->count.ipa ()
  + srccount.ipa ());
+ /* We produce uninitialized probabilities when
+denominator is zero: https://gcc.gnu.org/PR111559.  */
+ if (merged.initialized_p ())
+   dste->probability = merged;
}
  dstbb->count = dstbb->count.ipa () + srccount.ipa ();
}
-- 
2.42.0

bug#65679: Request to cut new automake release to include new pythons

[Sergei Trofimovich]

Hi automake developers!

Projects using automake build system occasionally get bug reports that
python 3.11 and above do not get automatically detected by

AM_PATH_PYTHON([3.7])

Recent example: https://github.com/skvadrik/re2c/issues/456

It was fixed a while ago in 

 
https://git.savannah.gnu.org/cgit/automake.git/commit/?id=930a9a73f4bb776ec334eff4cf6e182802841daa

  python: add 3.10 - 3.15 to the version search list
Fixes automake bug https://bugs.gnu.org/53530.

Based on the cadence of Automake releases, add the current Python
release (3.10), the current Python development (3.11), and then 4
more versions on top of that.  It doesn't hurt to check for a few
extra versions here since this is the fallback logic when the main
`python` and `python3` programs aren't found.

* m4/python.m4: Add python3.10 - python3.15.
* NEWS: Mention new Python versions.

WDYT of cutting new `automake` that includes this fix? Then projects
could start cutting release tarballs that Just Work for such systems.

Thanks

-- 

  Sergei

pathological bash-malloc behaviour

[Sergei Trofimovich]

Hi bash developers!

I think I encountered an unusual pathological behaviour of `bash-malloc`.
I have a directory with ~500_000 files and directories in it. I tried
to iterate through it with a single `for` loop using `*` glob and
noticed that glob expansion takes more time that actual loop payload
(the payload is to match and print a few names).

The reproducer (best ran on `tmpfs`):

# prepare test directory with ~900_000 files of different name
# lengths:
$ mkdir dd; for i in {1..10}; do echo 
dd/$i.{,1,22,333,,5,66,777,,99}; done | xargs 
touch

Make sure you have started a new interactive shell after directory
creation.

# trigger bug:
$ time { echo ./* > /dev/null; }
real7m9,378s
user7m8,327s
sys 0m0,277s

# second run is faster
$ time { echo ./* > /dev/null; }
real0m1,915s
user0m1,723s
sys 0m0,190s

# until we do a minor change:
$ time { echo .//* > /dev/null; }
real0m34,689s
user0m34,470s
sys 0m0,199s

# second run is faster again:
$ time { echo .///* > /dev/null; }
real1m6,269s
user1m5,999s
sys 0m0,195s

# and so on:
$ time { echo .///* > /dev/null; }
real1m6,269s
user1m5,999s
sys 0m0,195s
$ time { echo .///* > /dev/null; }
real0m2,420s
user0m2,212s
sys 0m0,206s

Note that initial expansion always takes 100x of the time we do it
the second time. This happens on `x86_64-linux` system with `bash`
configured as `./configure`. I suspect it has to do with the total
amount of memory allocated (and freed) by `bash` by the time the test
is ran.

If I `./configure` `bash` as `--without-bash-malloc` I see faster
behaviour on `glibc` system:

$ time { echo * > /dev/null; }
real0m0,969s
user0m0,737s
sys 0m0,231s
$ time { echo ./* > /dev/null; }
real0m1,296s
user0m1,097s
sys 0m0,198s
$ time { echo .//* > /dev/null; }
real0m1,380s
user0m1,183s
sys 0m0,196s
$ time { echo .///* > /dev/null; }
real0m1,408s
user0m1,225s
sys 0m0,181s
$ time { echo .* > /dev/null; }
real0m2,520s
user0m2,309s
sys 0m0,203s

The performance analysis points at `internal_malloc()` as the main CPU
user:

  $ perf top -p $pid
  99,54%  bash  [.] internal_malloc.constprop.0
   0,03%  [kernel]  [k] read_tsc
   0,03%  [kernel]  [k] __update_load_avg_se
  ...

`INSTALL` file says that `--with-bash-malloc` is enabled by default and
hints that it's possibly a faster allocator that default system's one.

Should the allocator be tweaked to handle this pathological behaviour?
Or maybe it's a good time for `linux` target to switch to
`--without-bash-malloc` as a faster default?

Thanks!

-- 

  Sergei

[PATCH] Drop unused enum vrp_mode.

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

Follow removal of EVRP and clean up unused defines.

gcc/
* flag-types.h (vrp_mode): Remove unused.
---
 gcc/flag-types.h | 7 ---
 1 file changed, 7 deletions(-)

diff --git a/gcc/flag-types.h b/gcc/flag-types.h
index 36305de589e..7466c1106f2 100644
--- a/gcc/flag-types.h
+++ b/gcc/flag-types.h
@@ -478,13 +478,6 @@ enum threader_debug
   THREADER_DEBUG_ALL = 1
 };
 
-/* VRP modes.  */
-enum vrp_mode
-{
-  VRP_MODE_VRP,
-  VRP_MODE_RANGER
-};
-
 /* Modes of OpenACC 'kernels' constructs handling.  */
 enum openacc_kernels
 {
-- 
2.41.0

Re: [PATCH] Makefile.in: add missing intl dependency for unwind_prot.o

[Sergei Trofimovich]

On Tue, 25 Jul 2023 10:12:23 -0400
Chet Ramey  wrote:

> On 7/24/23 6:33 PM, Sergei Trofimovich wrote:
> > As reported by Robert Scott in https://github.com/NixOS/nixpkgs/pull/245066
> > without the change `make -j8` build of `make` occasionally fails to
> > buildin parallel.   
> 
> Thanks for the report. This was reported and fixed last week.
> 
> https://lists.gnu.org/archive/html/bug-bash/2023-07/msg00100.html

Aha, Thank you!

The first thing I checked was the master branch of 

https://git.savannah.gnu.org/cgit/bash.git/log/

I now see that the change is in devel branch (less than an hour ago):


https://git.savannah.gnu.org/cgit/bash.git/commit/?h=devel=8418224f32f4dda85019cae063370c18bbd2d036

"changes for quoting special and multibyte characters in glob
patterns; changes to filename unicode normalization on macOS for
globbing and filename completion; send SIGCONT unconditionally to
just-restarted job in fg/bg"

It looks like there is a more up to date tree I should have checked
instead. Can you point me to that one?

Thanks again!

-- 

  Sergei

[PATCH] Makefile.in: add missing intl dependency for unwind_prot.o

[Sergei Trofimovich]

As reported by Robert Scott in https://github.com/NixOS/nixpkgs/pull/245066
without the change `make -j8` build of `make` occasionally fails to
buildin parallel. The simplest reproducer is:

$$ ./configure
$$ make unwind_prot.o
...
In file included from unwind_prot.c:51:
In file included from ./bashintl.h:30:
./include/gettext.h:27:11: fatal error: 'libintl.h' file not found
# include 
  ^~~
1 error generated.
make: *** [Makefile:106: unwind_prot.o] Error 1

The change adds missing ttransitive `${LIBINTL_H}` dependency for
unwind_prot.o.
---
 Makefile.in | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile.in b/Makefile.in
index 0b4df73a..15edc389 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -1432,6 +1432,7 @@ siglist.o: bashintl.h ${LIBINTL_H} $(BASHINCDIR)/gettext.h
 subst.o: bashintl.h ${LIBINTL_H} $(BASHINCDIR)/gettext.h
 test.o: bashintl.h ${LIBINTL_H} $(BASHINCDIR)/gettext.h
 trap.o: bashintl.h ${LIBINTL_H} $(BASHINCDIR)/gettext.h
+unwind_prot.o: bashintl.h ${LIBINTL_H} $(BASHINCDIR)/gettext.h
 variables.o: bashintl.h ${LIBINTL_H} $(BASHINCDIR)/gettext.h
 version.o: bashintl.h ${LIBINTL_H} $(BASHINCDIR)/gettext.h
 xmalloc.o: bashintl.h ${LIBINTL_H} $(BASHINCDIR)/gettext.h
-- 
2.41.0

[PATCH] mh-mingw: drop unused BOOT_CXXFLAGS variable

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

gcc's build system has BOOT_CFLAGS and various STAGE_C{,XX}FLAGS
variables. BOOT_CXXFLAGS is not handled anywhere.

config/

* mh-mingw: Drop assignment of unused BOOT_CXXFLAGS variable.
---
 config/mh-mingw | 1 -
 1 file changed, 1 deletion(-)

diff --git a/config/mh-mingw b/config/mh-mingw
index e91367a7112..f5fb064813f 100644
--- a/config/mh-mingw
+++ b/config/mh-mingw
@@ -1,7 +1,6 @@
 # Add -D__USE_MINGW_ACCESS to enable the built compiler to work on Windows
 # Vista (see PR33281 for details).
 BOOT_CFLAGS += -D__USE_MINGW_ACCESS -Wno-pedantic-ms-format
-BOOT_CXXFLAGS += -D__USE_MINGW_ACCESS -Wno-pedantic-ms-format
 CFLAGS += -D__USE_MINGW_ACCESS
 CXXFLAGS += -D__USE_MINGW_ACCESS
 STAGE1_CXXFLAGS += -D__USE_MINGW_ACCESS
-- 
2.41.0

Re: 'exec -a' and $0 substitution

[Sergei Trofimovich]

On Thu, 26 Jan 2023 00:35:27 -0500
Lawrence Velázquez  wrote:

> On Wed, Jan 25, 2023, at 6:37 PM, Sergei Trofimovich wrote:
> > I fear it's a side-effect of the way 'bash' gets executed via shebang by
> > kernel. But maybe not? Somehow direct script execution still manages to
> > preserve script's name. Is it an intended behaviour that could not be
> > easily changed? Or it's a bug?  
> 
> It's not a bug.

Thank you! That will make generic wrapping a bit more involved for us.

> https://www.gnu.org/software/bash/manual/html_node/Special-Parameters.html#index-0
> 
>   0
>   ($0)  Expands to the name of the shell or shell
>   script.  This is set at shell initialization.  If
>   Bash is invoked with a file of commands (see Shell
>   Scripts), $0 is set to the name of that file.  If
>   Bash is started with the -c option (see Invoking
>   Bash), then $0 is set to the first argument after
>   the string to be executed, if one is present.
>   Otherwise, it is set to the filename used to invoke
>   Bash, as given by argument zero.
> 
> Whenever bash is executed with a script file, it sets $0 to the
> name of that file.  It only uses caller-provided values for $0 when
> a script file is not provided.
> 
>   $ (exec -a foo bash <<<'echo "$0"')
>   foo
>   $ (exec -a foo bash -s bar baz quux <<<'echo "$0"')
>   foo
>   $ (exec -a foo bash -c 'echo "$0"')
>   foo
>   $ (exec -a foo bash -c 'echo "$0"' bar baz quux)
>   bar
> 
> This behavior is standardized in POSIX [*] and is not unique to
> bash (I omitted ksh93, which actually does produce "foo" here).
> 
>   $ (exec -a foo zsh <(echo 'echo "$0"'))
>   /dev/fd/63
>   $ (exec -a foo dash <(echo 'echo "$0"'))
>   /dev/fd/63
>   $ (exec -a foo yash <(echo 'echo "$0"'))
>   /dev/fd/63
>   $ (exec -a foo mksh <(echo 'echo "$0"'))
>   /dev/fd/63
> 
> [*]: 
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html#tag_20_117_05
> 
> -- 
> vq


-- 

  Sergei

'exec -a' and $0 substitution

[Sergei Trofimovich]

Hello bash maintainers! nixpkgs package collection likes to wrap binaies
and shell scripts to pass extra environment variables via wrappers.

One example is dejagnu's runtest:

$ cat `which runtest` | unnix
#! /<>/bash-5.2-p15/bin/bash -e
PATH=...
export PATH
exec -a "$0" "/<>/dejagnu-1.6.3/bin/.runtest-wrapped"  "$@"

Script's idea is to call .runtest-wrapped as if it had a 'runtest' name.
It's especially important for `runtest` because it internally checks for
own $0:

mypath=${0-.}
...
if [ "$target" != runtest ] ; then
target="--target ${target}"
else
target=""
fi

To the bug: looks like 'exec -a' does not work for bash scripts, but
does work for other executables. An example:

  Wrap C executable:

$ cat b.bash
#!/usr/bin/env bash

exec -a "$0" $PWD/a "$@"

$ cat a.c
#include 

int main(int argc, char * argv[])
{
for (int i = 0; i < argc; ++i) {
printf ("A[%i]='%s'\n", i, argv[i]);
}
}
$ gcc a.c -o a

  Result:

$ bash -x ./b.bash 1 2 3
+ exec -a ./b.bash /home/slyfox/bugs/a 1 2 3
A[0]='./b.bash'
A[1]='1'
A[2]='2'
A[3]='3'

  Works as expected: './a' sees './b.bash' as an argv[0].

  Wrap 'bash' script:

$ cat a.bash
#!/usr/bin/env bash

exec -a "$0" $PWD/foo "$@"

$ cat foo
#!/usr/bin/env bash

echo "$0" "$@"

  Result:

$ bash -x ./a.bash 1 2 3
+ exec -a ./a.bash /home/slyfox/bugs/foo 1 2 3
/home/slyfox/bugs/foo 1 2 3

  For some reason '-a ./a.bash' did not take an effect.

Such a discrepancy makes these wrappers to break occasionally like in
https://sourceware.org/PR30052.

I fear it's a side-effect of the way 'bash' gets executed via shebang by
kernel. But maybe not? Somehow direct script execution still manages to
preserve script's name. Is it an intended behaviour that could not be
easily changed? Or it's a bug?

Thank you!

-- 

  Sergei

Re: gnulib's obstack_* symbols in dynamic exports on glibc. Intentional?

[Sergei Trofimovich]

On Fri, 30 Dec 2022 19:26:48 +0100
Bruno Haible  wrote:

> Sergei Trofimovich wrote:
> > to my surprise it did have one a set of dynamically exported symbols:
> > 
> > $ nm -D `which ls` | grep -v '^ '
> > 004c0d40 T _obstack_allocated_p
> > 00534808 D obstack_alloc_failed_handler
> > 004c0bd0 T _obstack_begin
> > 004c0bf0 T _obstack_begin_1
> > 004c0d80 T _obstack_free
> > 004c0e00 T _obstack_memory_used
> > 004c0c10 T _obstack_newchunk  
> 
> There's already a discussion on this topic, that started at
>   https://lists.gnu.org/archive/html/bug-gnulib/2022-12/msg7.html

Aha, thank you!

-- 

  Sergei

gnulib's obstack_* symbols in dynamic exports on glibc. Intentional?

[Sergei Trofimovich]

Hello bug-gnulib (and libc-alpha, CCed)!

The other day I was looking at simplest programs to make sure they don't
normally have dynamic exports. I picked coreutils-9.1 'ls'. And to my
surprise it did have one a set of dynamically exported symbols:

$ nm -D `which ls` | grep -v '^ '
004c0d40 T _obstack_allocated_p
00534808 D obstack_alloc_failed_handler
004c0bd0 T _obstack_begin
004c0bf0 T _obstack_begin_1
004c0d80 T _obstack_free
004c0e00 T _obstack_memory_used
004c0c10 T _obstack_newchunk

That was a surprise to me. I think they are exported because gnulib
defines them in obstack.c.

AFAIU gnulib assumes that local implementation is better that glibc's
as it supports 64-bit object sizes.

I think ld decides to put symbols into dynamic exports to make sure
local symbols intended to override glibc's need to propagate to glibc
and other libraries that might use it.

Questions:

1. Is it a safe override to mangle symbols ilke that?

   Or would glibc's versioned symbols be used just fine by existing
   libraries?

   $ nm -D /lib/libc.so.6 | fgrep obstack
   00201338 B _obstack@GLIBC_2.2.5
   0009aa70 T _obstack_allocated_p@@GLIBC_2.2.5
   001fb338 D obstack_alloc_failed_handler@@GLIBC_2.2.5
   0009a780 T _obstack_begin@@GLIBC_2.2.5
   0009a840 T _obstack_begin_1@@GLIBC_2.2.5
   001fa208 D obstack_exit_failure@@GLIBC_2.2.5
   0009aab0 T _obstack_free@@GLIBC_2.2.5
   0009aab0 T obstack_free@@GLIBC_2.2.5
   0009ab40 T _obstack_memory_used@@GLIBC_2.2.5
   0009a900 T _obstack_newchunk@@GLIBC_2.2.5
   0007e110 W obstack_printf@@GLIBC_2.2.5
   0011e9c0 T __obstack_printf_chk@@GLIBC_2.8
   0007e100 W obstack_vprintf@@GLIBC_2.2.5
   0011ea80 T __obstack_vprintf_chk@@GLIBC_2.8

2. Would it be feasible to avoid dynamic symbol export and maybe rename
   gnulib's symbols to avoid clash with glibc?

   I would expect loading to be a tiny bit faster for binaries without
   dynamic exports like that.

Thanks!

-- 

  Sergei

Re: [PATCH v2] tests: do not fail on zero sized DIEs (binutils-2.39 compatible)

[Sergei Trofimovich via Elfutils-devel]

On Thu, 13 Oct 2022 16:40:36 +0200
Mark Wielaard  wrote:

> Hi,
> 
> On Mon, 2022-08-08 at 01:17 +0200, Mark Wielaard wrote:
> > On Sun, Aug 07, 2022 at 07:31:38PM +0100, Sergei Trofimovich via
> > Elfutils-devel wrote:  
> > > binutils started producing 0-sized DIEs on functions interspersed
> > > by nested sections (".section ...; .previous). This led to
> > > run-low_high_pc.sh failure in form of:
> > > 
> > > FAIL: run-low_high_pc.sh
> > > 
> > > 
> > > [b] main.c
> > > [2d] main
> > > 
> > > [b] ../sysdeps/i386/start.S
> > > [26] _start
> > > [40] ../sysdeps/x86/abi-note.c
> > > [b52] init.c
> > > [b8e] static-reloc.c
> > > [2dba] _dl_relocate_static_pie
> > > [2dd8] ../sysdeps/i386/crti.S
> > > [2def] _init
> > > lowpc: 8049000, highpc: 8049000lx
> > > ../sysdeps/i386/crti.S: [2def] '_init' highpc <= lowpc
> > > FAIL run-low_high_pc.sh (exit status: 255)
> > > 
> > > To work it around let's allow lowpc == highpc special case.
> > > 
> > > https://sourceware.org/bugzilla/show_bug.cgi?id=29450  
> > 
> > Thanks for finding this and suggesting a workaround.  But lets first
> > try to fix binutils. This seems like a pretty bad bug, lets hope it
> > gets fixed soon. So we don't need these kind of workarounds.
> > 
> > I added a comment to the binutils bug:
> > https://sourceware.org/bugzilla/show_bug.cgi?id=29451#c2  
> 
> Since this binutils bug was fixed I assume this patch isn't needed
> anymore.

Agreed. https://sourceware.org/PR29450#c8 cosed the bug as RESOLVED/MOVED
to declare it a binutils deficiency. Downstream use binutils-2.39 with a
gas patch and are able to run unmodified elfutils testsuite as is.

Thank you!

-- 

  Sergei

PING [PATCH] gcc/config/t-i386: add build dependencies on i386-builtin-types.inc

[Sergei Trofimovich via Gcc-patches]

On Thu, 22 Sep 2022 22:07:52 +0100
Sergei Trofimovich  wrote:

> On Fri, 16 Sept 2022 at 19:49, Sergei Trofimovich  wrote:
> >
> > From: Sergei Trofimovich 
> >
> > i386-builtin-types.inc is included indirectly via i386-builtins.h
> > into 4 files: i386.cc i386-builtins.cc i386-expand.cc i386-features.cc
> >
> > Only i386.cc dependency was present in gcc/config/t-i386 makefile.
> >
> > As a result parallel builds occasionally fail as:
> >
> > g++ ... -o i386-builtins.o ... 
> > ../../gcc-13-20220911/gcc/config/i386/i386-builtins.cc
> > In file included from 
> > ../../gcc-13-20220911/gcc/config/i386/i386-builtins.cc:92:
> > ../../gcc-13-20220911/gcc/config/i386/i386-builtins.h:25:10:
> >  fatal error: i386-builtin-types.inc: No such file or directory
> >25 | #include "i386-builtin-types.inc"
> >   |  ^~~~
> > compilation terminated.
> > make[3]: *** [../../gcc-13-20220911/gcc/config/i386/t-i386:54: 
> > i386-builtins.o]
> >   Error 1 shuffle=1663349189
> >
> > gcc/
> > * config/i386/t-i386: Add build-time dependencies against
> > i386-builtin-types.inc to i386-builtins.o, i386-expand.o,
> > i386-features.o.
> > ---
> >  gcc/config/i386/t-i386 | 5 +
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/gcc/config/i386/t-i386 b/gcc/config/i386/t-i386
> > index 4e2a0efc615..ffdbbdfe8ce 100644
> > --- a/gcc/config/i386/t-i386
> > +++ b/gcc/config/i386/t-i386
> > @@ -62,7 +62,12 @@ i386-features.o: $(srcdir)/config/i386/i386-features.cc
> > $(COMPILE) $<
> > $(POSTCOMPILE)
> >
> > +# i386-builtin-types.inc is included into i386-builtins.h.
> > +# Below are direct users of i386-builtins.h:
> >  i386.o: i386-builtin-types.inc
> > +i386-builtins.o: i386-builtin-types.inc
> > +i386-expand.o: i386-builtin-types.inc
> > +i386-features.o: i386-builtin-types.inc
> >
> >  i386-builtin-types.inc: s-i386-bt ; @true
> >  s-i386-bt: $(srcdir)/config/i386/i386-builtin-types.awk \
> > --
> > 2.37.2
> >  
> 
> Is it a reasonable approach? Maybe gcc has an equivalent of automake's
> BUILT_SOURCES to avoid explicit tracking of such dependencies?
> 
> -- 
> Sergei


-- 

  Sergei

[bug #63098] make-4.3.90 regression of unexpected dependencies in pattern rules with multiple targets

[Sergei Trofimovich]

Follow-up Comment #3, bug #63098 (project make):

Aha, behavior change makes sense. Let's close it as intended then.

Looks like opensp's case is even more nuanced. `%.h %.cxx %.rc: %.msg`
produces `.cxx` files only for a subset of `.msg` files: `.cxx` generation
happens only where file contents has `!cxx` marking. Otherwise only `.h` and
`.rc` are produced. That explains why tarballs don't contain some of `.cxx`
files and cause the rules to attempt to regenerate them at each `make`
invocation.

I filed https://sourceforge.net/p/openjade/bugs/151/ against opensp upstream.
Looks like last activity was in 2007, might take a long while to get fixed.
I'll attach the path there if I manage to get something working.

Thank you!


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63098] make-4.3.90 regression of unexpected dependencies in pattern rules with multiple targets

[Sergei Trofimovich]

URL:
  <https://savannah.gnu.org/bugs/?63098>

 Summary: make-4.3.90 regression of unexpected dependencies in
pattern rules with multiple targets
 Project: make
   Submitter: slyfox
   Submitted: Fri 23 Sep 2022 07:05:25 PM UTC
Severity: 3 - Normal
  Item Group: None
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: None
Operating System: None
   Fixed Release: None
   Triage Status: None


___

Follow-up Comments:


---
Date: Fri 23 Sep 2022 07:05:25 PM UTC By: Sergei Trofimovich 
Initially noticed the problem in OpenSP-1.5.2 build failure. Here is a
minimised example:


# cat Makefile
all: foo.a foo.b

%.a %.b %.c: %.d
touch $*.a
touch $*.b
touch $*.c


To trigger it we need to have up-to-date foo.d, foo.a and foo.b:


$ touch foo.d; touch foo.a; touch foo.b

# good make-4.3:
$ make-4.3
make: Nothing to be done for 'all'.

# problematic make-4.3.90:
$ make-4.3.90
touch foo.a
touch foo.b
touch foo.c


Note how new make version decided it needs to do extra work.

Looks like a bug.

A bit of background on original bug context (in case I extracted it
incorrectly):

OpenSP-1.5.2 uses similar pattern in
https://sourceforge.net/p/openjade/code/HEAD/tree/trunk/sp/lib/Makefile.am


# ...
%.h %.cxx %.rc: %.msg
[ ! -f $(top_srcdir)/msggen.pl ] || $(PERL) -w $(top_srcdir)/msggen.pl
$(MSGGENFLAGS) $<


OpenSP's release tarball ships only .h and .rc files (but not .cxx):
lib/WinInetStorageMessages.h lib/WinInetStorageMessages.msg
lib/WinInetStorageMessages.rc. It looks like nothing requires .cxx files in
that directory.

I only noticed the failure because my sandbox environment did not have perl
and caused build failure of make upgrade to 4.3.90.







___

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63098>

___
Message sent via Savannah
https://savannah.gnu.org/

Re: [PATCH] gcc/config/t-i386: add build dependencies on i386-builtin-types.inc

[Sergei Trofimovich via Gcc-patches]

On Fri, 16 Sept 2022 at 19:49, Sergei Trofimovich  wrote:
>
> From: Sergei Trofimovich 
>
> i386-builtin-types.inc is included indirectly via i386-builtins.h
> into 4 files: i386.cc i386-builtins.cc i386-expand.cc i386-features.cc
>
> Only i386.cc dependency was present in gcc/config/t-i386 makefile.
>
> As a result parallel builds occasionally fail as:
>
> g++ ... -o i386-builtins.o ... 
> ../../gcc-13-20220911/gcc/config/i386/i386-builtins.cc
> In file included from 
> ../../gcc-13-20220911/gcc/config/i386/i386-builtins.cc:92:
> ../../gcc-13-20220911/gcc/config/i386/i386-builtins.h:25:10:
>  fatal error: i386-builtin-types.inc: No such file or directory
>25 | #include "i386-builtin-types.inc"
>   |  ^~~~
> compilation terminated.
> make[3]: *** [../../gcc-13-20220911/gcc/config/i386/t-i386:54: 
> i386-builtins.o]
>   Error 1 shuffle=1663349189
>
> gcc/
> * config/i386/t-i386: Add build-time dependencies against
> i386-builtin-types.inc to i386-builtins.o, i386-expand.o,
> i386-features.o.
> ---
>  gcc/config/i386/t-i386 | 5 +
>  1 file changed, 5 insertions(+)
>
> diff --git a/gcc/config/i386/t-i386 b/gcc/config/i386/t-i386
> index 4e2a0efc615..ffdbbdfe8ce 100644
> --- a/gcc/config/i386/t-i386
> +++ b/gcc/config/i386/t-i386
> @@ -62,7 +62,12 @@ i386-features.o: $(srcdir)/config/i386/i386-features.cc
> $(COMPILE) $<
> $(POSTCOMPILE)
>
> +# i386-builtin-types.inc is included into i386-builtins.h.
> +# Below are direct users of i386-builtins.h:
>  i386.o: i386-builtin-types.inc
> +i386-builtins.o: i386-builtin-types.inc
> +i386-expand.o: i386-builtin-types.inc
> +i386-features.o: i386-builtin-types.inc
>
>  i386-builtin-types.inc: s-i386-bt ; @true
>  s-i386-bt: $(srcdir)/config/i386/i386-builtin-types.awk \
> --
> 2.37.2
>

Is it a reasonable approach? Maybe gcc has an equivalent of automake's
BUILT_SOURCES to avoid explicit tracking of such dependencies?

-- 
Sergei

Re: [Mingw-w64-public] [PATCH 1/2] mingw-w64-crt: fix parallel build failures in dlltool calls

[Sergei Trofimovich]

On Thu, 22 Sep 2022 11:23:38 +0300 (EEST)
Martin Storsjö  wrote:

> On Sun, 18 Sep 2022, Sergei Trofimovich wrote:
> 
> > Before the change massively parallel builds failed when Makefile
> > attempted to build both lib64/libd3dx9_38.a and lib32/libd3dx9_38.a:
> >
> >x86_64-w64-mingw32-dlltool --as-flags=--64 -m i386:x86-64 ... \
> >  --output-lib lib64/libd3dx9_38.a \
> >  --temp-prefix $(basename lib64/libd3dx9_38.a .a) ...
> >...
> >x86_64-w64-mingw32-dlltool --as-flags=--32 -m i386 ... \
> >  --output-lib lib32/libd3dx9_38.a \
> >  --temp-prefix $(basename lib32/libd3dx9_38.a .a) ...
> >
> >x86_64-w64-mingw32-dlltool: bfd_open failed reopen stub file: \
> >  libd3dx9_38s00271.o: No such file or directory
> >make[3]: *** [Makefile:77947: lib32/libd3dx9_38.a] Error 1 
> > shuffle=1662967885
> >make[3]: *** Waiting for unfinished jobs  
> 
> Thanks for the patch, and I'm sorry you've run into this issue. However, 
> I'm pretty sure that this issue already is fixed in git master (where 
> you've made your patch).
> 
> I assume you've run into this issue in the tagged v10.0.0 release, then 
> analyzed the issue and fixed it there, and gone ahead and adapted the fix 
> on top of git master, without retesting that the issue still was present 
> on git master? (The fact that it is fixed is very subtle though.)
> 
> This issue was fixed in 953bcd32ae470c4647e94de8548dda5a8f07d82d in git 
> master. Before the fix, the commands looked like what you quoted above, 
> calling dlltool with "--temp-prefix $(basename lib64/libd3dx9_38.a .a)". 
> However, after the fix, the make output prints them as being called with 
> "--temp-prefix lib64/libd3dx9_38" (the difference between $(basename) and 
> $$(basename) in that fix commit is very subtle, but should fix the issue).
> 
> Can you retest your issue on latest git master?

Ah, yes. 953bcd32ae470c4647e94de8548dda5a8f07d82d does fix this build failure
for me! Re-tested on master with only e1b0c1420bbd52ef505c71737c57393ac1397b0a
applied and could not trigger any failures in 30 consecutive builds. Usually
4-5 builds are enough to step on the race condition.

> BTW, we generally don't include the updates to Makefile.in in the patches 
> posted to the list.

Got it.

Thank you!

-- 

  Sergei


___
Mingw-w64-public mailing list
Mingw-w64-public@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public

[Mingw-w64-public] [PATCH 1/2] mingw-w64-crt: fix parallel build failures in dlltool calls

[Sergei Trofimovich]

Before the change massively parallel builds failed when Makefile
attempted to build both lib64/libd3dx9_38.a and lib32/libd3dx9_38.a:

x86_64-w64-mingw32-dlltool --as-flags=--64 -m i386:x86-64 ... \
  --output-lib lib64/libd3dx9_38.a \
  --temp-prefix $(basename lib64/libd3dx9_38.a .a) ...
...
x86_64-w64-mingw32-dlltool --as-flags=--32 -m i386 ... \
  --output-lib lib32/libd3dx9_38.a \
  --temp-prefix $(basename lib32/libd3dx9_38.a .a) ...

x86_64-w64-mingw32-dlltool: bfd_open failed reopen stub file: \
  libd3dx9_38s00271.o: No such file or directory
make[3]: *** [Makefile:77947: lib32/libd3dx9_38.a] Error 1 
shuffle=1662967885
make[3]: *** Waiting for unfinished jobs

It's best reproducible in `make --shuffle` mode:
https://savannah.gnu.org/bugs/index.php?62100
---
 mingw-w64-crt/Makefile.am |   8 ++-
 mingw-w64-crt/Makefile.in | 137 --
 2 files changed, 79 insertions(+), 66 deletions(-)

diff --git a/mingw-w64-crt/Makefile.am b/mingw-w64-crt/Makefile.am
index 23b1ab9f0..f736cce3a 100644
--- a/mingw-w64-crt/Makefile.am
+++ b/mingw-w64-crt/Makefile.am
@@ -47,7 +47,13 @@ else
   DTDEF64=$(DLLTOOL) $(DLLTOOLFLAGS64) $(AM_DLLTOOLFLAGS) --input-def
 endif
 if DLLTOOL_HAS_TEMP_PREFIX
-  AM_DLLTOOLFLAGS += --temp-prefix $(basename $@)
+  # Use target-specific suffix to avoid collision on massively
+  # parallel builds when 32-bit build stomps on 64-bit build
+  # using the same temp directory
+  DLLTOOLFLAGSARM32 += --temp-prefix $(basename $@)-arm
+  DLLTOOLFLAGSARM64 += --temp-prefix $(basename $@)-arm64
+  DLLTOOLFLAGS32 += --temp-prefix $(basename $@)-i386
+  DLLTOOLFLAGS64 += --temp-prefix $(basename $@)-x86_64
 endif
 if DELAY_IMPORT_LIBS
   AM_DLLTOOLFLAGS += --output-delaylib $@.delayimp.a
diff --git a/mingw-w64-crt/Makefile.in b/mingw-w64-crt/Makefile.in
index ecdb71ba3..d07eb77bd 100644
--- a/mingw-w64-crt/Makefile.in
+++ b/mingw-w64-crt/Makefile.in
@@ -151,9 +151,12 @@ PRE_UNINSTALL = :
 POST_UNINSTALL = :
 build_triplet = @build@
 host_triplet = @host@
-@DLLTOOL_HAS_TEMP_PREFIX_TRUE@am__append_1 = --temp-prefix $(basename $@)
-@DELAY_IMPORT_LIBS_TRUE@am__append_2 = --output-delaylib $@.delayimp.a
-@ENABLE_SOFTMATH_FALSE@am__append_3 = \
+@DLLTOOL_HAS_TEMP_PREFIX_TRUE@am__append_1 = --temp-prefix $(basename $@)-arm
+@DLLTOOL_HAS_TEMP_PREFIX_TRUE@am__append_2 = --temp-prefix $(basename $@)-arm64
+@DLLTOOL_HAS_TEMP_PREFIX_TRUE@am__append_3 = --temp-prefix $(basename $@)-i386
+@DLLTOOL_HAS_TEMP_PREFIX_TRUE@am__append_4 = --temp-prefix $(basename 
$@)-x86_64
+@DELAY_IMPORT_LIBS_TRUE@am__append_5 = --output-delaylib $@.delayimp.a
+@ENABLE_SOFTMATH_FALSE@am__append_6 = \
 @ENABLE_SOFTMATH_FALSE@  math/arm/exp2.S \
 @ENABLE_SOFTMATH_FALSE@  math/arm/exp2f.S \
 @ENABLE_SOFTMATH_FALSE@  math/arm/nearbyint.S \
@@ -195,7 +198,7 @@ host_triplet = @host@
 @ENABLE_SOFTMATH_FALSE@  math/arm-common/s_remquof.c \
 @ENABLE_SOFTMATH_FALSE@  math/arm-common/scalbn.c
 
-@ENABLE_SOFTMATH_TRUE@am__append_4 = \
+@ENABLE_SOFTMATH_TRUE@am__append_7 = \
 @ENABLE_SOFTMATH_TRUE@  math/softmath/e_fmod.cmath/softmath/e_fmodf.c   
math/softmath/e_powf.cmath/softmath/e_pow.c \
 @ENABLE_SOFTMATH_TRUE@  math/softmath/acosf.c math/softmath/acosh.c 
math/softmath/acosl.c math/softmath/acoshf.cmath/softmath/acoshl.c\
 @ENABLE_SOFTMATH_TRUE@  math/softmath/asinf.c math/softmath/asinh.c 
math/softmath/asinl.c math/softmath/asinhf.cmath/softmath/asinhl.c\
@@ -213,24 +216,24 @@ host_triplet = @host@
 @ENABLE_SOFTMATH_TRUE@  math/softmath/scalbnl.c   math/softmath/sin.c   
math/softmath/sincos.cmath/softmath/sincosf.c   math/softmath/sincosl.c   \
 @ENABLE_SOFTMATH_TRUE@  math/softmath/sinf.c  math/softmath/sinl.c  
math/softmath/tanf.c  math/softmath/tanl.c
 
-@ENABLE_SOFTMATH_FALSE@am__append_5 = \
+@ENABLE_SOFTMATH_FALSE@am__append_8 = \
 @ENABLE_SOFTMATH_FALSE@  math/arm-common/ldexpl.c  math/arm/sincos.S 
math/arm/sincosf.S
 
-@LIB32_TRUE@@W32API_FALSE@am__append_6 = lib32/libmsvcrt.a \
+@LIB32_TRUE@@W32API_FALSE@am__append_9 = lib32/libmsvcrt.a \
 @LIB32_TRUE@@W32API_FALSE@ lib32/libmsvcrt-os.a
-@LIB32_TRUE@@W32API_FALSE@am__append_7 = lib32/libdelayimp.a \
+@LIB32_TRUE@@W32API_FALSE@am__append_10 = lib32/libdelayimp.a \
 @LIB32_TRUE@@W32API_FALSE@ lib32/libm.a lib32/libgmon.a \
 @LIB32_TRUE@@W32API_FALSE@ lib32/liblargeint.a \
 @LIB32_TRUE@@W32API_FALSE@ lib32/libmingw32.a \
 @LIB32_TRUE@@W32API_FALSE@ lib32/libmingwex.a \
 @LIB32_TRUE@@W32API_FALSE@ lib32/libmoldname.a \
 @LIB32_TRUE@@W32API_FALSE@ lib32/libmingwthrd.a
-@LIB32_TRUE@@W32API_FALSE@am__append_8 = lib32/msvcrt.def lib32/msvcr80.def 
lib32/msvcr90.def lib32/msvcr90d.def lib32/msvcr100.def lib32/msvcr110.def 
lib32/msvcr120.def lib32/msvcr120d.def \
+@LIB32_TRUE@@W32API_FALSE@am__append_11 = lib32/msvcrt.def lib32/msvcr80.def 
lib32/msvcr90.def lib32/msvcr90d.def lib32/msvcr100.def 

[Mingw-w64-public] [PATCH 2/2] Makefile.am: add missing msvcr${V} dependency on msvcr${V}_extra

[Sergei Trofimovich]

Without the change build fails with missing dependency on _extra
library similar to the following:

cd lib32/ && x86_64-w64-mingw32-ar -M < 
/build/mingw-w64/mingw-w64-crt/lib32/msvcr71.mri
x86_64-w64-mingw32-ar: libmsvcr71_extra.a: No such file or directory
make[3]: *** [Makefile:78262: lib32/libmsvcr71.a] Error 9 shuffle=1663492408

The build failures are best reproducible with make --shuffle:
https://savannah.gnu.org/bugs/index.php?62100

The dependency is required by msvcr${V}.mri script.

It's best reproducible in `make --shuffle` mode:
https://savannah.gnu.org/bugs/index.php?62100
---
 mingw-w64-crt/Makefile.am | 10 +-
 mingw-w64-crt/Makefile.in | 10 +-
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/mingw-w64-crt/Makefile.am b/mingw-w64-crt/Makefile.am
index f736cce3a..83ed8 100644
--- a/mingw-w64-crt/Makefile.am
+++ b/mingw-w64-crt/Makefile.am
@@ -935,23 +935,23 @@ lib32/libcrtdll.a: lib32/crtdll.mri lib32/libcrtdll_def.a 
lib32/libmsvcrt_common
cd $(dir $@) && $(AR) -M < $(abspath $<)
 
 lib32_DATA += lib32/libmsvcrt10.a
-lib32/libmsvcrt10.a: lib32/msvcrt10.mri lib32/libmsvcrt10_def.a 
lib32/libmsvcrt_common.a
+lib32/libmsvcrt10.a: lib32/msvcrt10.mri lib32/libmsvcrt10_def.a 
lib32/libmsvcrt_common.a lib32/libmsvcrt10_extra.a
cd $(dir $@) && $(AR) -M < $(abspath $<)
 
 lib32_DATA += lib32/libmsvcrt20.a
-lib32/libmsvcrt20.a: lib32/msvcrt20.mri lib32/libmsvcrt20_def.a 
lib32/libmsvcrt_common.a
+lib32/libmsvcrt20.a: lib32/msvcrt20.mri lib32/libmsvcrt20_def.a 
lib32/libmsvcrt_common.a lib32/libmsvcrt20_extra.a
cd $(dir $@) && $(AR) -M < $(abspath $<)
 
 lib32_DATA += lib32/libmsvcrt40.a
-lib32/libmsvcrt40.a: lib32/msvcrt40.mri lib32/libmsvcrt40_def.a 
lib32/libmsvcrt_common.a
+lib32/libmsvcrt40.a: lib32/msvcrt40.mri lib32/libmsvcrt40_def.a 
lib32/libmsvcrt_common.a lib32/libmsvcrt40_extra.a
cd $(dir $@) && $(AR) -M < $(abspath $<)
 
 lib32_DATA += lib32/libmsvcr70.a
-lib32/libmsvcr70.a: lib32/msvcr70.mri lib32/libmsvcr70_def.a 
lib32/libmsvcrt_common.a
+lib32/libmsvcr70.a: lib32/msvcr70.mri lib32/libmsvcr70_def.a 
lib32/libmsvcrt_common.a lib32/libmsvcr70_extra.a
cd $(dir $@) && $(AR) -M < $(abspath $<)
 
 lib32_DATA += lib32/libmsvcr71.a
-lib32/libmsvcr71.a: lib32/msvcr71.mri lib32/libmsvcr71_def.a 
lib32/libmsvcrt_common.a
+lib32/libmsvcr71.a: lib32/msvcr71.mri lib32/libmsvcr71_def.a 
lib32/libmsvcrt_common.a lib32/libmsvcr71_extra.a
cd $(dir $@) && $(AR) -M < $(abspath $<)
 
 lib32_DATA += lib32/libmsvcr80.a
diff --git a/mingw-w64-crt/Makefile.in b/mingw-w64-crt/Makefile.in
index d07eb77bd..d00181051 100644
--- a/mingw-w64-crt/Makefile.in
+++ b/mingw-w64-crt/Makefile.in
@@ -78251,15 +78251,15 @@ _libm_dummy.c:
 @LIB32_TRUE@@W32API_FALSE@ cd $(dir $@) && $(AR) -M < $(abspath $<)
 @LIB32_TRUE@@W32API_FALSE@lib32/libcrtdll.a: lib32/crtdll.mri 
lib32/libcrtdll_def.a lib32/libmsvcrt_common.a lib32/libcrtdll_extra.a
 @LIB32_TRUE@@W32API_FALSE@ cd $(dir $@) && $(AR) -M < $(abspath $<)
-@LIB32_TRUE@@W32API_FALSE@lib32/libmsvcrt10.a: lib32/msvcrt10.mri 
lib32/libmsvcrt10_def.a lib32/libmsvcrt_common.a
+@LIB32_TRUE@@W32API_FALSE@lib32/libmsvcrt10.a: lib32/msvcrt10.mri 
lib32/libmsvcrt10_def.a lib32/libmsvcrt_common.a lib32/libmsvcrt10_extra.a
 @LIB32_TRUE@@W32API_FALSE@ cd $(dir $@) && $(AR) -M < $(abspath $<)
-@LIB32_TRUE@@W32API_FALSE@lib32/libmsvcrt20.a: lib32/msvcrt20.mri 
lib32/libmsvcrt20_def.a lib32/libmsvcrt_common.a
+@LIB32_TRUE@@W32API_FALSE@lib32/libmsvcrt20.a: lib32/msvcrt20.mri 
lib32/libmsvcrt20_def.a lib32/libmsvcrt_common.a lib32/libmsvcrt20_extra.a
 @LIB32_TRUE@@W32API_FALSE@ cd $(dir $@) && $(AR) -M < $(abspath $<)
-@LIB32_TRUE@@W32API_FALSE@lib32/libmsvcrt40.a: lib32/msvcrt40.mri 
lib32/libmsvcrt40_def.a lib32/libmsvcrt_common.a
+@LIB32_TRUE@@W32API_FALSE@lib32/libmsvcrt40.a: lib32/msvcrt40.mri 
lib32/libmsvcrt40_def.a lib32/libmsvcrt_common.a lib32/libmsvcrt40_extra.a
 @LIB32_TRUE@@W32API_FALSE@ cd $(dir $@) && $(AR) -M < $(abspath $<)
-@LIB32_TRUE@@W32API_FALSE@lib32/libmsvcr70.a: lib32/msvcr70.mri 
lib32/libmsvcr70_def.a lib32/libmsvcrt_common.a
+@LIB32_TRUE@@W32API_FALSE@lib32/libmsvcr70.a: lib32/msvcr70.mri 
lib32/libmsvcr70_def.a lib32/libmsvcrt_common.a lib32/libmsvcr70_extra.a
 @LIB32_TRUE@@W32API_FALSE@ cd $(dir $@) && $(AR) -M < $(abspath $<)
-@LIB32_TRUE@@W32API_FALSE@lib32/libmsvcr71.a: lib32/msvcr71.mri 
lib32/libmsvcr71_def.a lib32/libmsvcrt_common.a
+@LIB32_TRUE@@W32API_FALSE@lib32/libmsvcr71.a: lib32/msvcr71.mri 
lib32/libmsvcr71_def.a lib32/libmsvcrt_common.a lib32/libmsvcr71_extra.a
 @LIB32_TRUE@@W32API_FALSE@ cd $(dir $@) && $(AR) -M < $(abspath $<)
 @LIB32_TRUE@@W32API_FALSE@lib32/libmsvcr80.a: lib-common/msvcr80.mri 
lib32/libmsvcr80_def.a lib32/libmsvcrt_common.a
 @LIB32_TRUE@@W32API_FALSE@ cd $(dir $@) && $(AR) -M < $(abspath $<)
-- 
2.37.2



___
Mingw-w64-public mailing list

[PATCH] gcc/config/t-i386: add build dependencies on i386-builtin-types.inc

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

i386-builtin-types.inc is included indirectly via i386-builtins.h
into 4 files: i386.cc i386-builtins.cc i386-expand.cc i386-features.cc

Only i386.cc dependency was present in gcc/config/t-i386 makefile.

As a result parallel builds occasionally fail as:

g++ ... -o i386-builtins.o ... 
../../gcc-13-20220911/gcc/config/i386/i386-builtins.cc
In file included from 
../../gcc-13-20220911/gcc/config/i386/i386-builtins.cc:92:
../../gcc-13-20220911/gcc/config/i386/i386-builtins.h:25:10:
 fatal error: i386-builtin-types.inc: No such file or directory
   25 | #include "i386-builtin-types.inc"
  |  ^~~~
compilation terminated.
make[3]: *** [../../gcc-13-20220911/gcc/config/i386/t-i386:54: 
i386-builtins.o]
  Error 1 shuffle=1663349189

gcc/
* config/i386/t-i386: Add build-time dependencies against
i386-builtin-types.inc to i386-builtins.o, i386-expand.o,
i386-features.o.
---
 gcc/config/i386/t-i386 | 5 +
 1 file changed, 5 insertions(+)

diff --git a/gcc/config/i386/t-i386 b/gcc/config/i386/t-i386
index 4e2a0efc615..ffdbbdfe8ce 100644
--- a/gcc/config/i386/t-i386
+++ b/gcc/config/i386/t-i386
@@ -62,7 +62,12 @@ i386-features.o: $(srcdir)/config/i386/i386-features.cc
$(COMPILE) $<
$(POSTCOMPILE)
 
+# i386-builtin-types.inc is included into i386-builtins.h.
+# Below are direct users of i386-builtins.h:
 i386.o: i386-builtin-types.inc
+i386-builtins.o: i386-builtin-types.inc
+i386-expand.o: i386-builtin-types.inc
+i386-features.o: i386-builtin-types.inc
 
 i386-builtin-types.inc: s-i386-bt ; @true
 s-i386-bt: $(srcdir)/config/i386/i386-builtin-types.awk \
-- 
2.37.2

[bug #63047] --shuffle mode does not work on .SECONDEXPANSION targets

[Sergei Trofimovich]

Follow-up Comment #1, bug #63047 (project make):

Posted the patch for review as
https://lists.gnu.org/archive/html/bug-make/2022-09/msg00069.html


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63047] --shuffle mode does not work on .SECONDEXPANSION targets

[Sergei Trofimovich]

Additional Item Attachment, bug #63047 (project make):

File name: 0001-Fix-shuffle-for-SECONDEXPANSION-prerequisites.patch Size:3 KB
   




___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[PATCH] Fix --shuffle for SECONDEXPANSION prerequisites

[Sergei Trofimovich]

From: Sergei Trofimovich 

Since commit 07eea3aa4 ("[SV 62706] Only second-expand targets that
might be built") `make --shuffle` stopped shuffling prerequisites that
use .SECONDEXPANSION feature for Makefiles like:

.SECONDEXPANSION:
all: $$(var)
%_: ; @echo $@
var = a_ b_ c_

Shuffle does not happen anymore because second expansion now happens
after shuffle phase. This has two problems:
1. No shuffling happens for such prerequisites.
2. Already freed data is accessed when when oudated '->shuf' links
   are used.

The fix inserts reshuffle into expansion phase right after dependency
changes to fix both problems.

* src/file.c (expand_deps): Add reshuffling phase if dependency updates.
* src/shuffle.c (identity_shuffle_array): Fix comment typo.
* tests/scripts/options/shuffle: Add new SECONDEXPANSION test.
---
 src/file.c| 10 ++
 src/shuffle.c |  2 +-
 tests/scripts/options/shuffle |  9 +
 3 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/src/file.c b/src/file.c
index 7596ff10..83aa593c 100644
--- a/src/file.c
+++ b/src/file.c
@@ -25,6 +25,7 @@ this program.  If not, see <http://www.gnu.org/licenses/>.  */
 #include "variable.h"
 #include "debug.h"
 #include "hash.h"
+#include "shuffle.h"
 
 
 /* Remember whether snap_deps has been invoked: we need this to be sure we
@@ -576,6 +577,7 @@ expand_deps (struct file *f)
   struct dep **dp;
   const char *fstem;
   int initialized = 0;
+  int changed_dep = 0;
 
   if (f->snapped)
 return;
@@ -664,6 +666,7 @@ expand_deps (struct file *f)
   if (new == 0)
 {
   *dp = d->next;
+  changed_dep = 1;
   free_dep (d);
   d = *dp;
   continue;
@@ -672,6 +675,7 @@ expand_deps (struct file *f)
   /* Add newly parsed prerequisites.  */
   fstem = d->stem;
   next = d->next;
+  changed_dep = 1;
   free_dep (d);
   *dp = new;
   for (dp = , d = new; d != 0; dp = >next, d = d->next)
@@ -688,6 +692,12 @@ expand_deps (struct file *f)
   *dp = next;
   d = *dp;
 }
+
+/* --shuffle mode assumes '->next' and '->shuf' links both traverse
+   the same dependencies (in different sequences).  Here we
+   regenerate '->shuf'.  Otherwise we risk referring stale data.  */
+if (changed_dep)
+  shuffle_deps_recursive (f->deps);
 }
 
 /* Add extra prereqs to the file in question.  */
diff --git a/src/shuffle.c b/src/shuffle.c
index ea6e836d..23c9c224 100644
--- a/src/shuffle.c
+++ b/src/shuffle.c
@@ -154,7 +154,7 @@ identity_shuffle_array (void **a UNUSED, size_t len UNUSED)
   /* No-op!  */
 }
 
-/* Shuffle list of dependencies by populating '->next'
+/* Shuffle list of dependencies by populating '->shuf'
field in each 'struct dep'.  */
 static void
 shuffle_deps (struct dep *deps)
diff --git a/tests/scripts/options/shuffle b/tests/scripts/options/shuffle
index e037ed1a..5661683c 100644
--- a/tests/scripts/options/shuffle
+++ b/tests/scripts/options/shuffle
@@ -116,4 +116,13 @@ run_make_test('
 ',
   '--shuffle=reverse a_ b_ c_', "a_\nb_\nc_");
 
+# Check if SECONDEXPANSION targets also get reshuffled.
+run_make_test('
+.SECONDEXPANSION:
+all: $$(var)
+%_: ; @echo $@
+var = a_ b_ c_
+',
+  '--shuffle=reverse', "c_\nb_\na_");
+
 1;
-- 
2.37.2

[bug #63047] --shuffle mode does not work on .SECONDEXPANSION targets

[Sergei Trofimovich]

URL:
  <https://savannah.gnu.org/bugs/?63047>

 Summary: --shuffle mode does not work on .SECONDEXPANSION
targets
 Project: make
   Submitter: slyfox
   Submitted: Sun 11 Sep 2022 08:13:12 PM UTC
Severity: 3 - Normal
  Item Group: None
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: None
Operating System: None
   Fixed Release: None
   Triage Status: None


___

Follow-up Comments:


---
Date: Sun 11 Sep 2022 08:13:12 PM UTC By: Sergei Trofimovich 
Since commit 07eea3aa496184bb763b7306e9de6a40a94605c9 ("[SV 62706] Only
second-expand targets that might be built") `make --shuffle` stopped shuffling
.SECONDEXPANSION. Example `Makefile`:

.SECONDEXPANSION:
all: $$(var)
%_: ; @echo $@
var = a_ b_ c_

Before (good):

$ make --shuffle=reverse
c_
b_
a_

After (bad):

$ ./make --shuffle=reverse
a_
b_
c_


I'll send a possible fix today.







___

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63047>

___
Message sent via Savannah
https://savannah.gnu.org/

Re: [PATCH] autoconf: fix build on upcoming GNU make-4.4

[Sergei Trofimovich]

On Fri, 9 Sep 2022 16:59:21 -0500
Paul Eggert  wrote:

> Thanks for the bug report. I installed the attached, which isn't the 
> same as what you suggested, but which should fix the problem.

You version works for me as well. Thank you!

-- 

  Sergei

[bug #63040] autoconf-2.69 and 2.71 fail to build after $(shell ...) environment handlign change

[Sergei Trofimovich]

Follow-up Comment #1, bug #63040 (project make):

Proposed possible fix for autoconf as
https://lists.gnu.org/archive/html/autoconf-patches/2022-09/msg7.html


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[PATCH] autoconf: fix build on upcoming GNU make-4.4

[Sergei Trofimovich]

>From https://savannah.gnu.org/bugs/?63040:
GNU make 4.4 will slightly change the rules it uses to export variables
to $(shell ...) calls. `autoconf` uses unportable variant of assignment:

export PATH = $(shell echo "`pwd`/tests:$$PATH")

This causes build failure on `make` from `git`:

bash: line 1: env: command not found
make  all-am
bash: line 1: make: command not found
make: *** [Makefile:928: all] Error 127

The change it to use probably intended for or eager assignment:

export PATH = $(shell echo "`pwd`/tests:$$PATH")

* cfg.mk (PATH): Assign path after $(shell) call.
---
 cfg.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cfg.mk b/cfg.mk
index 7e515130..fd34dfa0 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -18,7 +18,7 @@
 # This file is '-include'd into GNUmakefile.
 
 # Build with our own versions of these tools, when possible.
-export PATH = $(shell echo "`pwd`/tests:$$PATH")
+export PATH := $(shell echo "`pwd`/tests:$$PATH")
 
 # Remove the autoreconf-provided INSTALL, so that we regenerate it.
 _autoreconf = autoreconf -i -v && rm -f INSTALL
-- 
2.37.2

[RESEND PATCH] autoconf: store autom4te request keys in sorted order

[Sergei Trofimovich]

Notced unstable key order when debugging unrelated bug.
Data::Dumper's SortKeys() makes ordering stable and decreases
diffs from run to run. No functional change otherwise.

* lib/Autom4te/C4che.pm
* lib/Autom4te/Request.pm: Sort request keys in serialization.
---
 lib/Autom4te/C4che.pm   | 2 +-
 lib/Autom4te/Request.pm | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/Autom4te/C4che.pm b/lib/Autom4te/C4che.pm
index 8b7c3c11..e9561809 100644
--- a/lib/Autom4te/C4che.pm
+++ b/lib/Autom4te/C4che.pm
@@ -158,7 +158,7 @@ sub marshall ($)
   my $res = '';
 
   my $marshall = Data::Dumper->new ([\@request], [qw (*request)]);
-  $marshall->Indent(2)->Terse(0);
+  $marshall->Indent(2)->Terse(0)->Sortkeys(1);
   $res = $marshall->Dump . "\n";
 
   return $res;
diff --git a/lib/Autom4te/Request.pm b/lib/Autom4te/Request.pm
index c137ef21..1b922f7d 100644
--- a/lib/Autom4te/Request.pm
+++ b/lib/Autom4te/Request.pm
@@ -63,7 +63,7 @@ sub marshall($)
 
   # CALLER is an object: instance method.
   my $marshall = Data::Dumper->new ([$caller]);
-  $marshall->Indent(2)->Terse(0);
+  $marshall->Indent(2)->Terse(0)->Sortkeys(1);
   $res = $marshall->Dump . "\n";
 
   return $res;
-- 
2.37.2

[bug #63040] autoconf-2.69 and 2.71 fail to build after $(shell ...) environment handlign change

[Sergei Trofimovich]

URL:
  <https://savannah.gnu.org/bugs/?63040>

 Summary: autoconf-2.69 and 2.71 fail to build after $(shell
...) environment handlign change
 Project: make
   Submitter: slyfox
   Submitted: Fri 09 Sep 2022 10:35:02 AM UTC
Severity: 3 - Normal
  Item Group: None
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: None
Operating System: None
   Fixed Release: None
   Triage Status: None


___

Follow-up Comments:


---
Date: Fri 09 Sep 2022 10:35:02 AM UTC By: Sergei Trofimovich 
Tried to use `make` from today's git (commit
7d484017077089ac2642b89da8984ca46a07323d [SV 63016] Don't fail exporting to
$(shell ...)) and observed build failure on autoconf:


build flags: -j16 -l16 SHELL=bash
bash: line 1: env: command not found
make  all-am
bash: line 1: make: command not found
make: *** [Makefile:928: all] Error 127


The build failure happens because (I think) autoconf uses the following
trick:


cfg.mk:export PATH = $(shell echo "`pwd`/tests:$$PATH")


Comparing behaviour from `make` a few months ago:


$ make --version
GNU Make 4.3.90.20220619

$ printf 'export PATH = $(shell echo $$PWD:$$PATH)'"\n"'all:; echo $$PATH' |
make -f -
echo $PATH
/tmp:/run/current-system/sw/bin


Path is prepended to existing one, ok.

And the today's one:


$ ./make --version
GNU Make 4.3.90.20220909
$ printf 'export PATH = $(shell echo $$PWD:$$PATH)'"\n"'all:; echo $$PATH' |
./make -f -
echo $PATH
make: sh: No such file or directory
make: sh: No such file or directory
make: *** [/tmp/GmYdo0my:2: all] Error 127


If we use `:=` then it works as expected:


$ printf 'export PATH := $(shell echo $$PWD:$$PATH)'"\n"'all:; echo $$PATH' |
./make -f -
echo $PATH
/tmp:/run/current-system/sw/bin


In theory it should be easy to adapt `autoconf`s build system. Filing a bug to
make sure it's an intended change and not an unexpected regression.







___

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63040>

___
Message sent via Savannah
https://savannah.gnu.org/

[PATCH] Makefile.def: drop remnants of unused libelf

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

Use of libelf was removed from gcc in
misc/cutover-cvs2svn-32781-g48215350c24 ("re PR lto/46273 (Failed to
bootstrap)") around 2010, before gcc-4.6.0.

This change removes unused references to libelf from top-level condifure
and Makefile.

/
* Makefile.def: Drop libelf module and gcc-configure dependency
on it.
* Makefile.in: Regenerate with 'autogen Makefile.def'.
* Makefile.tpl (HOST_EXPORTS): Drop unused LIBELFLIBS and
LIBELFINC.
* configure: Regenrate.
* configure.ac (host_libs): Drop unused libelf.
---
 Makefile.def |4 -
 Makefile.in  | 1288 +-
 Makefile.tpl |6 -
 configure|2 +-
 configure.ac |2 +-
 5 files changed, 3 insertions(+), 1299 deletions(-)

diff --git a/Makefile.def b/Makefile.def
index 11e9f23dc42..3291b126b26 100644
--- a/Makefile.def
+++ b/Makefile.def
@@ -70,9 +70,6 @@ host_modules= { module= isl; lib_path=.libs; bootstrap=true;
extra_configure_flags='--disable-shared 
@extra_isl_gmp_configure_flags@';
extra_make_flags='V=1';
no_install= true; };
-host_modules= { module= libelf; lib_path=.libs; bootstrap=true;
-   extra_configure_flags='--disable-shared';
-   no_install= true; };
 host_modules= { module= gold; bootstrap=true; };
 host_modules= { module= gprof; };
 host_modules= { module= intl; bootstrap=true; };
@@ -353,7 +350,6 @@ dependencies = { module=configure-gcc; on=all-binutils; };
 dependencies = { module=configure-gcc; on=all-gas; };
 dependencies = { module=configure-gcc; on=all-ld; };
 dependencies = { module=configure-gcc; on=all-gold; };
-dependencies = { module=configure-gcc; on=all-libelf; };
 dependencies = { module=configure-gcc; on=all-libiconv; };
 dependencies = { module=all-gcc; on=all-libiberty; hard=true; };
 dependencies = { module=all-gcc; on=all-intl; };
diff --git a/Makefile.in b/Makefile.in
index 60077a6aa36..1919dfee829 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -236,8 +236,6 @@ HOST_EXPORTS = \
GMPINC="$(HOST_GMPINC)"; export GMPINC; \
ISLLIBS="$(HOST_ISLLIBS)"; export ISLLIBS; \
ISLINC="$(HOST_ISLINC)"; export ISLINC; \
-   LIBELFLIBS="$(HOST_LIBELFLIBS)"; export LIBELFLIBS; \
-   LIBELFINC="$(HOST_LIBELFINC)"; export LIBELFINC; \
XGCC_FLAGS_FOR_TARGET="$(XGCC_FLAGS_FOR_TARGET)"; export 
XGCC_FLAGS_FOR_TARGET; \
 @if gcc-bootstrap
$(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 
's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \
@@ -343,10 +341,6 @@ HOST_GMPINC = @gmpinc@
 HOST_ISLLIBS = @isllibs@
 HOST_ISLINC = @islinc@
 
-# Where to find libelf
-HOST_LIBELFLIBS = @libelflibs@
-HOST_LIBELFINC = @libelfinc@
-
 # --
 # Programs producing files for the BUILD machine
 # --
@@ -754,7 +748,7 @@ TARGET_LIB_PATH_libatomic = 
$$r/$(TARGET_SUBDIR)/libatomic/.libs:
 
 # This is the list of directories that may be needed in RPATH_ENVVAR
 # so that programs built for the host machine work.
-HOST_LIB_PATH = 
$(HOST_LIB_PATH_gmp)$(HOST_LIB_PATH_mpfr)$(HOST_LIB_PATH_mpc)$(HOST_LIB_PATH_isl)$(HOST_LIB_PATH_libelf)
+HOST_LIB_PATH = 
$(HOST_LIB_PATH_gmp)$(HOST_LIB_PATH_mpfr)$(HOST_LIB_PATH_mpc)$(HOST_LIB_PATH_isl)
 
 # Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch
 @if gcc
@@ -782,11 +776,6 @@ HOST_LIB_PATH_isl = \
   $$r/$(HOST_SUBDIR)/isl/.libs:$$r/$(HOST_SUBDIR)/prev-isl/.libs:
 @endif isl
 
-@if libelf
-HOST_LIB_PATH_libelf = \
-  $$r/$(HOST_SUBDIR)/libelf/.libs:$$r/$(HOST_SUBDIR)/prev-libelf/.libs:
-@endif libelf
-
 
 CXX_FOR_TARGET_FLAG_TO_PASS = \
"CXX_FOR_TARGET=$(CXX_FOR_TARGET)"
@@ -1078,7 +1067,6 @@ configure-host:  \
 maybe-configure-mpfr \
 maybe-configure-mpc \
 maybe-configure-isl \
-maybe-configure-libelf \
 maybe-configure-gold \
 maybe-configure-gprof \
 maybe-configure-intl \
@@ -1229,9 +1217,6 @@ all-host: maybe-all-mpc
 @if isl-no-bootstrap
 all-host: maybe-all-isl
 @endif isl-no-bootstrap
-@if libelf-no-bootstrap
-all-host: maybe-all-libelf
-@endif libelf-no-bootstrap
 @if gold-no-bootstrap
 all-host: maybe-all-gold
 @endif gold-no-bootstrap
@@ -1370,7 +1355,6 @@ info-host: maybe-info-gmp
 info-host: maybe-info-mpfr
 info-host: maybe-info-mpc
 info-host: maybe-info-isl
-info-host: maybe-info-libelf
 info-host: maybe-info-gold
 info-host: maybe-info-gprof
 info-host: maybe-info-intl
@@ -1460,7 +1444,6 @@ dvi-host: maybe-dvi-gmp
 dvi-host: maybe-dvi-mpfr
 dvi-host: maybe-dvi-mpc
 dvi-host: maybe-dvi-isl
-dvi-host: maybe-dvi-libelf
 dvi-host: maybe-dvi-gold
 dvi-host: maybe-dvi-gprof
 dvi-host: maybe-dvi-intl
@@ -1550,7 +1533,6 @@ pdf-host: maybe-pdf-gmp
 pdf-host: maybe-pdf-mpfr
 pdf-host: maybe-pdf-mpc
 pdf-host: maybe-pdf

[PATCH] Add libgo dependency on libbacktrace.

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

Noticed missing dependency when regenerated Makefile.in for unrelated
change with 'autoget Makefile.def'.

The change was lost in basepoints/gcc-12-6861-gaeac414923a
("Revert "Fix PR 67102: Add libstdc++ dependancy to libffi" [PR67102]").

/
Makefile.in: Regenerate.
---
 Makefile.in | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile.in b/Makefile.in
index 13ee95a2602..60077a6aa36 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -66523,6 +66523,7 @@ all-target-liboffloadmic: maybe-all-target-libgomp
 configure-target-newlib: maybe-all-binutils
 configure-target-newlib: maybe-all-ld
 configure-target-libgfortran: maybe-all-target-libbacktrace
+configure-target-libgo: maybe-all-target-libbacktrace
 @endunless gcc-bootstrap
 
 # Dependencies for target modules on other target modules are
-- 
2.37.1

[PATCH] driver: fix environ corruption after putenv() [PR106624]

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

The bug appeared afte r13-2010-g1270ccda70ca09 "Factor out
jobserver_active_p" slightly changed `putenv()` use from allocating
to non-allocating:

-xputenv (concat ("MAKEFLAGS=", dup, NULL));
+xputenv (jinfo.skipped_makeflags.c_str ());

`xputenv()` (and `putenv()`) don't copy strings and only store the
pointer in the `environ` global table. As a result `environ` got
corrupted as soon as `jinfo.skipped_makeflags` store got deallocated.

This started causing bootstrap crashes in `execv()` calls:

xgcc: fatal error: cannot execute '/build/build/./prev-gcc/collect2': 
execv: Bad address

The change restores memory allocation for `xputenv()` argument.

gcc/

PR driver/106624
* gcc (driver::detect_jobserver): Allocate storage xputenv()
argument using xstrdup().
---
 gcc/gcc.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gcc/gcc.cc b/gcc/gcc.cc
index cac11c1a117..75ca0ece1a4 100644
--- a/gcc/gcc.cc
+++ b/gcc/gcc.cc
@@ -9182,7 +9182,7 @@ driver::detect_jobserver () const
 {
   jobserver_info jinfo;
   if (!jinfo.is_active && !jinfo.skipped_makeflags.empty ())
-xputenv (jinfo.skipped_makeflags.c_str ());
+xputenv (xstrdup (jinfo.skipped_makeflags.c_str ()));
 }
 
 /* Determine what the exit code of the driver should be.  */
-- 
2.37.1

[PATCH v2] tests: do not fail on zero sized DIEs (binutils-2.39 compatible)

[Sergei Trofimovich via Elfutils-devel]

binutils started producing 0-sized DIEs on functions interspersed
by nested sections (".section ...; .previous). This led to
run-low_high_pc.sh failure in form of:

FAIL: run-low_high_pc.sh


[b] main.c
[2d] main

[b] ../sysdeps/i386/start.S
[26] _start
[40] ../sysdeps/x86/abi-note.c
[b52] init.c
[b8e] static-reloc.c
[2dba] _dl_relocate_static_pie
[2dd8] ../sysdeps/i386/crti.S
[2def] _init
lowpc: 8049000, highpc: 8049000lx
../sysdeps/i386/crti.S: [2def] '_init' highpc <= lowpc
FAIL run-low_high_pc.sh (exit status: 255)

To work it around let's allow lowpc == highpc special case.

https://sourceware.org/bugzilla/show_bug.cgi?id=29450

Signed-off-by: Sergei Trofimovich 
---
 tests/ChangeLog | 4 
 tests/low_high_pc.c | 8 +---
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/tests/ChangeLog b/tests/ChangeLog
index 0c6f68ef..59b4252a 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,7 @@
+2022-08-07  Sergei Trofimovich  
+
+   * low_high_pc.c (handle_die): Allow zero sized DIEs for binutils-2.39.
+
 2022-08-01  Mark Wielaard  
 
* run-debuginfod-percent-escape.sh: Add initial scan wait_ready.
diff --git a/tests/low_high_pc.c b/tests/low_high_pc.c
index 78b6ad08..63268f4c 100644
--- a/tests/low_high_pc.c
+++ b/tests/low_high_pc.c
@@ -66,11 +66,13 @@ handle_die (Dwarf_Die *die, void *arg)
 fail (off, name, "has DW_AT_high_pc but dwarf_highpc fails");
 
   /* GCC < 4.7 had a bug where no code CUs got a highpc == lowpc.
- Allow that, because it is not the main purpose of this test.  */
+ Allow that, because it is not the main purpose of this test.
+ gas-2.39 produces zero sized DIEs for subprograms sometimes:
+ https://sourceware.org/PR29451.
+   */
   if (dwarf_hasattr (die, DW_AT_low_pc)
   && dwarf_hasattr (die, DW_AT_high_pc)
-  && highpc <= lowpc
-  && ! (dwarf_tag (die) == DW_TAG_compile_unit && highpc == lowpc))
+  && highpc < lowpc)
 {
   printf("lowpc: %" PRIx64 ", highpc: %" PRIx64 "lx\n", lowpc, highpc);
   fail (off, name, "highpc <= lowpc");
-- 
2.37.1

[PATCH] tests: run-low_high_pc.sh: drop redundant 'lx' suffix

[Sergei Trofimovich via Elfutils-devel]

Noticed when debugged test failure:

lowpc: 8049000, highpc: 8049000lx
../sysdeps/i386/crti.S: [2def] '_init' highpc <= lowpc

Signed-off-by: Sergei Trofimovich 
---
 tests/ChangeLog | 4 
 tests/low_high_pc.c | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/tests/ChangeLog b/tests/ChangeLog
index 0c6f68ef..d2952cc9 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,7 @@
+2022-08-04  Sergei Trofimovich  
+
+   * low_high_pc.c (handle_die): Drop redundant 'lx' suffix.
+
 2022-08-01  Mark Wielaard  
 
* run-debuginfod-percent-escape.sh: Add initial scan wait_ready.
diff --git a/tests/low_high_pc.c b/tests/low_high_pc.c
index 78b6ad08..cd022b1c 100644
--- a/tests/low_high_pc.c
+++ b/tests/low_high_pc.c
@@ -72,7 +72,7 @@ handle_die (Dwarf_Die *die, void *arg)
   && highpc <= lowpc
   && ! (dwarf_tag (die) == DW_TAG_compile_unit && highpc == lowpc))
 {
-  printf("lowpc: %" PRIx64 ", highpc: %" PRIx64 "lx\n", lowpc, highpc);
+  printf("lowpc: %" PRIx64 ", highpc: %" PRIx64 "\n", lowpc, highpc);
   fail (off, name, "highpc <= lowpc");
 }
 
-- 
2.37.1

[PATCH] tests: do not fail on zero sized DIEs (binutils-2.39 compatible)

[Sergei Trofimovich via Elfutils-devel]

binutils started producing 0-sized DIEs on functions interspersed
by nested sections (".section ...; .previous). This led to
run-low_high_pc.sh failure in form of:

FAIL: run-low_high_pc.sh


[b] main.c
[2d] main

[b] ../sysdeps/i386/start.S
[26] _start
[40] ../sysdeps/x86/abi-note.c
[b52] init.c
[b8e] static-reloc.c
[2dba] _dl_relocate_static_pie
[2dd8] ../sysdeps/i386/crti.S
[2def] _init
lowpc: 8049000, highpc: 8049000lx
../sysdeps/i386/crti.S: [2def] '_init' highpc <= lowpc
FAIL run-low_high_pc.sh (exit status: 255)

To work it around let's allow lowpc == highpc special case.

https://sourceware.org/bugzilla/show_bug.cgi?id=29450

Signed-off-by: Sergei Trofimovich 
---
 tests/ChangeLog | 4 
 tests/low_high_pc.c | 3 +++
 2 files changed, 7 insertions(+)

diff --git a/tests/ChangeLog b/tests/ChangeLog
index 0c6f68ef..8296e0b6 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,7 @@
+2022-08-04  Sergei Trofimovich  
+
+   * low_high_pc.c (handle_die): Allow zero sized DIEs for binutils-2.39.
+
 2022-08-01  Mark Wielaard  
 
* run-debuginfod-percent-escape.sh: Add initial scan wait_ready.
diff --git a/tests/low_high_pc.c b/tests/low_high_pc.c
index cd022b1c..80c83b6d 100644
--- a/tests/low_high_pc.c
+++ b/tests/low_high_pc.c
@@ -70,6 +70,9 @@ handle_die (Dwarf_Die *die, void *arg)
   if (dwarf_hasattr (die, DW_AT_low_pc)
   && dwarf_hasattr (die, DW_AT_high_pc)
   && highpc <= lowpc
+  /* gas-2.39 produces zero sized DIEs sometimes:
+ https://sourceware.org/PR29451.  */
+  && highpc != lowpc
   && ! (dwarf_tag (die) == DW_TAG_compile_unit && highpc == lowpc))
 {
   printf("lowpc: %" PRIx64 ", highpc: %" PRIx64 "\n", lowpc, highpc);
-- 
2.37.1

Re: 'make -j install' fails for missing depends

[Sergei Trofimovich]

On Wed, 29 Jun 2022 09:16:17 +0200
Daniel Sahlberg  wrote:

> Den lör 25 juni 2022 kl 00:41 skrev Sergei Trofimovich :
> 
> > On Fri, 24 Jun 2022 15:14:23 -0400
> > Nathan Hartman  wrote:
> >  
> > > On Thu, Jun 23, 2022 at 5:58 AM Sergei Trofimovich   
> > wrote:  
> > > >
> > > > Hi subversion developers!
> > > >
> > > > I'm working on fixing parallel build (or install) faliures in upstream
> > > > packages used in NixOS linux distribution. Mostly to speed package  
> > builds  
> > > > up on continuous build farm.
> > > >
> > > > That usually means that
> > > > $ ./configure && make && make install
> > > > is changed to
> > > > $ ./configure && make -j$(nproc) && make -j$(nproc) install
> > > >
> > > > To make the error more obvioud GNU make recently added --shuffle option
> > > > to better expose missing dependencies across Makefile targets:
> > > > https://savannah.gnu.org/bugs/index.php?62100
> > > >
> > > > 'subversion-1.14.2' came as one of the packages that sometimes fails on
> > > > 'make install'
> > > >
> > > > There is a build log snippet that exposed missing install dependencies:
> > > >
> > > > $ ./configure --disable-static --prefix=/<>/subversion-1.14.2  
> > --bindir=/<>/subversion-1.14.2/bin
> > --sbindir=/<>/subversion-1.14.2/sbin
> > --includedir=/<>/subversion-1.14.2-dev/include
> > --oldincludedir=/<>/subversion-1.14.2-dev/include
> > --mandir=/<>/subversion-1.14.2-man/share/man
> > --infodir=/<>/subversion-1.14.2/share/info
> > --docdir=/<>/subversion-1.14.2/share/doc/subversion
> > --libdir=/<>/subversion-1.14.2/lib
> > --libexecdir=/<>/subversion-1.14.2/libexec
> > --localedir=/<>/subversion-1.14.2/share/locale --with-berkeley-db
> > --without-apxs --without-swig --without-sasl
> > --with-serf=/<>/serf-1.3.9 --with-zlib=/<>/zlib-1.2.12-dev
> > --with-sqlite=/<>/sqlite-3.38.5-dev --build=x86_64-unknown-linux-gnu
> > --host=x86_64-unknown-linux-gnu  
> > > > ...
> > > > $ make -j16 -l16 SHELL=/<>/bash-5.1-p16/bin/bash  
> > APACHE_LIBEXECDIR=/<>/subversion-1.14.2/modules  
> > > > ...
> > > > $ make --shuffle  
> > APACHE_LIBEXECDIR=/<>/subversion-1.14.2/modules
> > pkgconfigdir=/<>/subversion-1.14.2-dev/lib/pkgconfig
> > m4datadir=/<>/subversion-1.14.2-dev/share/aclocal
> > aclocaldir=/<>/subversion-1.14.2-dev/share/aclocal install  
> > > > test -d /<>/subversion-1.14.2-dev/include/subversion-1 || \
> > > >   /<>/coreutils-9.1/bin/install -c -d  
> > /<>/subversion-1.14.2-dev/include/subversion-1  
> > > > (subversion/svnversion/svnversion . 2> /dev/null ||  \
> > > >  svnversion . 2> /dev/null ||\
> > > >  echo "unknown"; \
> > > > ) >  
> > /<>/subversion-1.14.2-dev/include/subversion-1/svn-revision.txt  
> > > > /<>/coreutils-9.1/bin/install -c -d  
> > /<>/subversion-1.14.2/lib /<>/subversion-1.14.2/share/pkgconfig  
> > > > cd subversion/libsvn_fs_base ; /<>/bash-5.1-p16/bin/bash  
> > "/build/subversion/libtool" --mode=install
> > /<>/coreutils-9.1/bin/install -c libsvn_fs_base-1.la
> > /<>/subversion-1.14.2/lib/libsvn_fs_base-1.la  
> > > > libtool: warning: relinking 'libsvn_fs_base-1.la'
> > > > libtool: install: (cd /build/subversion/subversion/libsvn_fs_base;  
> > /<>/bash-5.1-p16/bin/bash "/build/subversion/libtool"  --tag CC
> > --silent --mode=relink gcc -shared -g -O2 -g -O2
> > -L/<>/openssl-1.1.1o/lib -L/<>/db-5.3.28/lib
> > -L/<>/expat-2.4.8/lib -L/<>/serf-1.3.9/lib -rpath
> > /<>/subversion-1.14.2/lib -version-info 0 -Wl,--no-undefined -o
> > libsvn_fs_base-1.la bdb/bdb-err.lo bdb/bdb_compat.lo bdb/changes-table.lo
> > bdb/checksum-reps-table.lo bdb/copies-table.lo bdb/dbt.lo bdb/env.lo
> > bdb/lock-tokens-table.lo bdb/locks-table.lo bdb/miscellaneous-table.lo
> > bdb/node-origins-table.lo bdb/nodes-table.lo bdb/reps-table.lo
> > bdb/rev-table.lo bdb/strings-table.lo bdb/txn-table.lo bdb/uuids-table.lo
> > dag.lo err.lo fs.lo id.lo key-gen.lo lock.lo nod

[PATCH v2] jit: avoid calloc() poisoning on musl [PR106102]

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

On musl  uses calloc() (via ). jit/ includes
it directly and exposes use of poisoned calloc():

/build/build/./prev-gcc/xg++ ... 
../../gcc-13-20220626/gcc/jit/jit-playback.cc
make[3]: *** [Makefile:1143: jit/libgccjit.o] Error 1
make[3]: *** Waiting for unfinished jobs
In file included from /<>/musl-1.2.3-dev/include/pthread.h:30,
 from ../../gcc-13-20220626/gcc/jit/jit-playback.cc:44:
/<>/musl-1.2.3-dev/include/sched.h:84:7: error: attempt to use 
poisoned "calloc"
   84 | void *calloc(size_t, size_t);
  |   ^
/<>/musl-1.2.3-dev/include/sched.h:124:36: error: attempt to use 
poisoned "calloc"
  124 | #define CPU_ALLOC(n) ((cpu_set_t *)calloc(1,CPU_ALLOC_SIZE(n)))
  |^

The change moves  inclusion to "system.h" under new
INCLUDE_PTHREAD_H guard and uses this mechanism in libgccjit.

gcc/

PR c++/106102
* system.h: Introduce INCLUDE_PTHREAD_H macros to include .

gcc/jit/

PR c++/106102
* jit-playback.cc: Include  via "system.h" to avoid calloc()
poisoning.
* jit-recording.cc: Ditto.
* libgccjit.cc: Ditto.
---
 gcc/jit/jit-playback.cc  | 3 +--
 gcc/jit/jit-recording.cc | 2 +-
 gcc/jit/libgccjit.cc | 2 +-
 gcc/system.h | 4 
 4 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/gcc/jit/jit-playback.cc b/gcc/jit/jit-playback.cc
index 6be6bdf8dea..79714132b91 100644
--- a/gcc/jit/jit-playback.cc
+++ b/gcc/jit/jit-playback.cc
@@ -19,6 +19,7 @@ along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */
 
 #include "config.h"
+#define INCLUDE_PTHREAD_H
 #include "system.h"
 #include "coretypes.h"
 #include "target.h"
@@ -41,8 +42,6 @@ along with GCC; see the file COPYING3.  If not see
 #include "diagnostic.h"
 #include "stmt.h"
 
-#include 
-
 #include "jit-playback.h"
 #include "jit-result.h"
 #include "jit-builtins.h"
diff --git a/gcc/jit/jit-recording.cc b/gcc/jit/jit-recording.cc
index 697dee66e73..f78daed2d71 100644
--- a/gcc/jit/jit-recording.cc
+++ b/gcc/jit/jit-recording.cc
@@ -19,13 +19,13 @@ along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */
 
 #include "config.h"
+#define INCLUDE_PTHREAD_H
 #include "system.h"
 #include "coretypes.h"
 #include "tm.h"
 #include "pretty-print.h"
 #include "toplev.h"
 
-#include 
 
 #include "jit-builtins.h"
 #include "jit-recording.h"
diff --git a/gcc/jit/libgccjit.cc b/gcc/jit/libgccjit.cc
index 0e76097b4ba..ca862662777 100644
--- a/gcc/jit/libgccjit.cc
+++ b/gcc/jit/libgccjit.cc
@@ -19,12 +19,12 @@ along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */
 
 #include "config.h"
+#define INCLUDE_PTHREAD_H
 #include "system.h"
 #include "coretypes.h"
 #include "timevar.h"
 #include "typed-splay-tree.h"
 #include "cppbuiltin.h"
-#include 
 
 #include "libgccjit.h"
 #include "jit-recording.h"
diff --git a/gcc/system.h b/gcc/system.h
index 67158b70c78..f8d42ff6815 100644
--- a/gcc/system.h
+++ b/gcc/system.h
@@ -753,6 +753,10 @@ extern int vsnprintf (char *, size_t, const char *, 
va_list);
 #endif
 #endif
 
+#ifdef INCLUDE_PTHREAD_H
+#include 
+#endif
+
 #ifdef INCLUDE_ISL
 #ifdef HAVE_isl
 #include 
-- 
2.36.1

[PATCH] jit: avoid calloc() poisoning on musl [PR106102]

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

On musl  uses calloc() (via ). jit/ includes
it directly and exposes use of poisoned calloc():

/build/build/./prev-gcc/xg++ ... 
../../gcc-13-20220626/gcc/jit/jit-playback.cc
make[3]: *** [Makefile:1143: jit/libgccjit.o] Error 1
make[3]: *** Waiting for unfinished jobs
In file included from /<>/musl-1.2.3-dev/include/pthread.h:30,
 from ../../gcc-13-20220626/gcc/jit/jit-playback.cc:44:
/<>/musl-1.2.3-dev/include/sched.h:84:7: error: attempt to use 
poisoned "calloc"
   84 | void *calloc(size_t, size_t);
  |   ^
/<>/musl-1.2.3-dev/include/sched.h:124:36: error: attempt to use 
poisoned "calloc"
  124 | #define CPU_ALLOC(n) ((cpu_set_t *)calloc(1,CPU_ALLOC_SIZE(n)))
  |^

The change moves sys-specific header before "system.h" inclusion
to evade poisoning.

gcc/jit/

PR c++/106102
* jit-playback.cc: Include  before "system.h" to avoid 
calloc()
poisoning.
* jit-recording.cc: Ditto.
* libgccjit.cc: Ditto.
---
 gcc/jit/jit-playback.cc  | 5 +++--
 gcc/jit/jit-recording.cc | 4 +++-
 gcc/jit/libgccjit.cc | 4 +++-
 3 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/gcc/jit/jit-playback.cc b/gcc/jit/jit-playback.cc
index 6be6bdf8dea..4ab7b59cab9 100644
--- a/gcc/jit/jit-playback.cc
+++ b/gcc/jit/jit-playback.cc
@@ -19,6 +19,9 @@ along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */
 
 #include "config.h"
+/*  has to go before "system.h" as it pull in to-be-poisoned
+   symbols on musl like calloc().  */
+#include 
 #include "system.h"
 #include "coretypes.h"
 #include "target.h"
@@ -41,8 +44,6 @@ along with GCC; see the file COPYING3.  If not see
 #include "diagnostic.h"
 #include "stmt.h"
 
-#include 
-
 #include "jit-playback.h"
 #include "jit-result.h"
 #include "jit-builtins.h"
diff --git a/gcc/jit/jit-recording.cc b/gcc/jit/jit-recording.cc
index 697dee66e73..9505ed3b279 100644
--- a/gcc/jit/jit-recording.cc
+++ b/gcc/jit/jit-recording.cc
@@ -19,13 +19,15 @@ along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */
 
 #include "config.h"
+/*  has to go before "system.h" as it pull in to-be-poisoned
+   symbols on musl like calloc().  */
+#include 
 #include "system.h"
 #include "coretypes.h"
 #include "tm.h"
 #include "pretty-print.h"
 #include "toplev.h"
 
-#include 
 
 #include "jit-builtins.h"
 #include "jit-recording.h"
diff --git a/gcc/jit/libgccjit.cc b/gcc/jit/libgccjit.cc
index 0e76097b4ba..6a0ac8aaba2 100644
--- a/gcc/jit/libgccjit.cc
+++ b/gcc/jit/libgccjit.cc
@@ -19,12 +19,14 @@ along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */
 
 #include "config.h"
+/*  has to go before "system.h" as it pull in to-be-poisoned
+   symbols on musl like calloc().  */
+#include 
 #include "system.h"
 #include "coretypes.h"
 #include "timevar.h"
 #include "typed-splay-tree.h"
 #include "cppbuiltin.h"
-#include 
 
 #include "libgccjit.h"
 #include "jit-recording.h"
-- 
2.36.1

[PATCH] c++: avoid poisoning on musl [PR106102]

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

On musl  uses calloc() (via ).  includes
it indirectly and exposes use of poisoned calloc() when module code
is built:

/build/build/./prev-gcc/xg++ ... 
../../gcc-13-20220626/gcc/cp/mapper-resolver.cc
In file included from /<>/musl-1.2.3-dev/include/pthread.h:30,
 from 
/build/build/prev-x86_64-unknown-linux-musl/libstdc++-v3/include/x86_64-unknown-linux-musl/bits/gthr-default.h:35,
 
 from 
/build/build/prev-x86_64-unknown-linux-musl/libstdc++-v3/include/memory:77,
 from ../../gcc-13-20220626/gcc/../libcody/cody.hh:24,
 from ../../gcc-13-20220626/gcc/cp/../../c++tools/resolver.h:25,
 from 
../../gcc-13-20220626/gcc/cp/../../c++tools/resolver.cc:23,
 from ../../gcc-13-20220626/gcc/cp/mapper-resolver.cc:32:
/<>/musl-1.2.3-dev/include/sched.h:84:7: error: attempt to use 
poisoned "calloc"
   84 | void *calloc(size_t, size_t);
  |   ^
/<>/musl-1.2.3-dev/include/sched.h:124:36: error: attempt to use 
poisoned "calloc"
  124 | #define CPU_ALLOC(n) ((cpu_set_t *)calloc(1,CPU_ALLOC_SIZE(n)))
  |^

gcc/cp/
* mapper-client.cc: Include  via "system.h".
* mapper-resolver.cc: Ditto.
* module.cc: Ditto.

libcc1/
* libcc1plugin.cc: Ditto.
* libcp1plugin.cc: Ditto.
---
 gcc/cp/mapper-client.cc   | 1 +
 gcc/cp/mapper-resolver.cc | 1 +
 gcc/cp/module.cc  | 1 +
 libcc1/libcc1plugin.cc| 1 +
 libcc1/libcp1plugin.cc| 1 +
 5 files changed, 5 insertions(+)

diff --git a/gcc/cp/mapper-client.cc b/gcc/cp/mapper-client.cc
index 8603a886a09..fe9544b5ba4 100644
--- a/gcc/cp/mapper-client.cc
+++ b/gcc/cp/mapper-client.cc
@@ -27,6 +27,7 @@ along with GCC; see the file COPYING3.  If not see
 #define INCLUDE_STRING
 #define INCLUDE_VECTOR
 #define INCLUDE_MAP
+#define INCLUDE_MEMORY
 #include "system.h"
 
 #include "line-map.h"
diff --git a/gcc/cp/mapper-resolver.cc b/gcc/cp/mapper-resolver.cc
index e3d29fb5ada..e70d1b4ae2c 100644
--- a/gcc/cp/mapper-resolver.cc
+++ b/gcc/cp/mapper-resolver.cc
@@ -25,6 +25,7 @@ along with GCC; see the file COPYING3.  If not see
 #define INCLUDE_VECTOR
 #define INCLUDE_ALGORITHM
 #define INCLUDE_MAP
+#define INCLUDE_MEMORY
 #include "system.h"
 
 // We don't want or need to be aware of networking
diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc
index e7ce40ef464..99f10733d4b 100644
--- a/gcc/cp/module.cc
+++ b/gcc/cp/module.cc
@@ -206,6 +206,7 @@ Classes used:
 
 #define _DEFAULT_SOURCE 1 /* To get TZ field of struct tm, if available.  */
 #include "config.h"
+#define INCLUDE_MEMORY
 #define INCLUDE_STRING
 #define INCLUDE_VECTOR
 #include "system.h"
diff --git a/libcc1/libcc1plugin.cc b/libcc1/libcc1plugin.cc
index 12ab5a57c8d..bdd0bdabe77 100644
--- a/libcc1/libcc1plugin.cc
+++ b/libcc1/libcc1plugin.cc
@@ -31,6 +31,7 @@
 #undef PACKAGE_TARNAME
 #undef PACKAGE_VERSION
 
+#define INCLUDE_MEMORY
 #include "gcc-plugin.h"
 #include "system.h"
 #include "coretypes.h"
diff --git a/libcc1/libcp1plugin.cc b/libcc1/libcp1plugin.cc
index 83dab7f58b1..e2d5039a0a1 100644
--- a/libcc1/libcp1plugin.cc
+++ b/libcc1/libcp1plugin.cc
@@ -32,6 +32,7 @@
 #undef PACKAGE_TARNAME
 #undef PACKAGE_VERSION
 
+#define INCLUDE_MEMORY
 #include "gcc-plugin.h"
 #include "system.h"
 #include "coretypes.h"
-- 
2.36.1

Re: 'make -j install' fails for missing depends

[Sergei Trofimovich]

On Fri, 24 Jun 2022 15:14:23 -0400
Nathan Hartman  wrote:

> On Thu, Jun 23, 2022 at 5:58 AM Sergei Trofimovich  wrote:
> >
> > Hi subversion developers!
> >
> > I'm working on fixing parallel build (or install) faliures in upstream
> > packages used in NixOS linux distribution. Mostly to speed package builds
> > up on continuous build farm.
> >
> > That usually means that
> > $ ./configure && make && make install
> > is changed to
> > $ ./configure && make -j$(nproc) && make -j$(nproc) install
> >
> > To make the error more obvioud GNU make recently added --shuffle option
> > to better expose missing dependencies across Makefile targets:
> > https://savannah.gnu.org/bugs/index.php?62100
> >
> > 'subversion-1.14.2' came as one of the packages that sometimes fails on
> > 'make install'
> >
> > There is a build log snippet that exposed missing install dependencies:
> >
> > $ ./configure --disable-static --prefix=/<>/subversion-1.14.2 
> > --bindir=/<>/subversion-1.14.2/bin 
> > --sbindir=/<>/subversion-1.14.2/sbin 
> > --includedir=/<>/subversion-1.14.2-dev/include 
> > --oldincludedir=/<>/subversion-1.14.2-dev/include 
> > --mandir=/<>/subversion-1.14.2-man/share/man 
> > --infodir=/<>/subversion-1.14.2/share/info 
> > --docdir=/<>/subversion-1.14.2/share/doc/subversion 
> > --libdir=/<>/subversion-1.14.2/lib 
> > --libexecdir=/<>/subversion-1.14.2/libexec 
> > --localedir=/<>/subversion-1.14.2/share/locale --with-berkeley-db 
> > --without-apxs --without-swig --without-sasl 
> > --with-serf=/<>/serf-1.3.9 --with-zlib=/<>/zlib-1.2.12-dev 
> > --with-sqlite=/<>/sqlite-3.38.5-dev --build=x86_64-unknown-linux-gnu 
> > --host=x86_64-unknown-linux-gnu
> > ...
> > $ make -j16 -l16 SHELL=/<>/bash-5.1-p16/bin/bash 
> > APACHE_LIBEXECDIR=/<>/subversion-1.14.2/modules
> > ...
> > $ make --shuffle APACHE_LIBEXECDIR=/<>/subversion-1.14.2/modules 
> > pkgconfigdir=/<>/subversion-1.14.2-dev/lib/pkgconfig 
> > m4datadir=/<>/subversion-1.14.2-dev/share/aclocal 
> > aclocaldir=/<>/subversion-1.14.2-dev/share/aclocal install
> > test -d /<>/subversion-1.14.2-dev/include/subversion-1 || \
> >   /<>/coreutils-9.1/bin/install -c -d 
> > /<>/subversion-1.14.2-dev/include/subversion-1
> > (subversion/svnversion/svnversion . 2> /dev/null ||  \
> >  svnversion . 2> /dev/null ||\
> >  echo "unknown"; \
> > ) > /<>/subversion-1.14.2-dev/include/subversion-1/svn-revision.txt
> > /<>/coreutils-9.1/bin/install -c -d /<>/subversion-1.14.2/lib 
> > /<>/subversion-1.14.2/share/pkgconfig
> > cd subversion/libsvn_fs_base ; /<>/bash-5.1-p16/bin/bash 
> > "/build/subversion/libtool" --mode=install 
> > /<>/coreutils-9.1/bin/install -c libsvn_fs_base-1.la 
> > /<>/subversion-1.14.2/lib/libsvn_fs_base-1.la
> > libtool: warning: relinking 'libsvn_fs_base-1.la'
> > libtool: install: (cd /build/subversion/subversion/libsvn_fs_base; 
> > /<>/bash-5.1-p16/bin/bash "/build/subversion/libtool"  --tag CC 
> > --silent --mode=relink gcc -shared -g -O2 -g -O2 
> > -L/<>/openssl-1.1.1o/lib -L/<>/db-5.3.28/lib 
> > -L/<>/expat-2.4.8/lib -L/<>/serf-1.3.9/lib -rpath 
> > /<>/subversion-1.14.2/lib -version-info 0 -Wl,--no-undefined -o 
> > libsvn_fs_base-1.la bdb/bdb-err.lo bdb/bdb_compat.lo bdb/changes-table.lo 
> > bdb/checksum-reps-table.lo bdb/copies-table.lo bdb/dbt.lo bdb/env.lo 
> > bdb/lock-tokens-table.lo bdb/locks-table.lo bdb/miscellaneous-table.lo 
> > bdb/node-origins-table.lo bdb/nodes-table.lo bdb/reps-table.lo 
> > bdb/rev-table.lo bdb/strings-table.lo bdb/txn-table.lo bdb/uuids-table.lo 
> > dag.lo err.lo fs.lo id.lo key-gen.lo lock.lo node-rev.lo reps-strings.lo 
> > revs-txns.lo trail.lo tree.lo util/fs_skels.lo uuid.lo 
> > ../../subversion/libsvn_delta/libsvn_delta-1.la 
> > ../../subversion/libsvn_subr/libsvn_subr-1.la -L/<>/apr-util-1.6.1/lib 
> > -laprutil-1 -L/<>/apr-1.7.0/lib -lapr-1 -ldb-5.3 
../../subversion/libsvn_fs_util/libsvn_fs_util-1.la )
> > /<>/binutils-2.38/bin/ld: cannot find -lsvn_delta-1: No such file 
> > or directory
> > /<>/binutils-2.38/bin/ld: cannot find -lsvn_fs_util-1: No such 
> &

'make -j install' fails for missing depends

[Sergei Trofimovich]

Hi subversion developers!

I'm working on fixing parallel build (or install) faliures in upstream
packages used in NixOS linux distribution. Mostly to speed package builds
up on continuous build farm.

That usually means that
$ ./configure && make && make install
is changed to
$ ./configure && make -j$(nproc) && make -j$(nproc) install

To make the error more obvioud GNU make recently added --shuffle option
to better expose missing dependencies across Makefile targets:
https://savannah.gnu.org/bugs/index.php?62100

'subversion-1.14.2' came as one of the packages that sometimes fails on
'make install'

There is a build log snippet that exposed missing install dependencies:

$ ./configure --disable-static --prefix=/<>/subversion-1.14.2 
--bindir=/<>/subversion-1.14.2/bin 
--sbindir=/<>/subversion-1.14.2/sbin 
--includedir=/<>/subversion-1.14.2-dev/include 
--oldincludedir=/<>/subversion-1.14.2-dev/include 
--mandir=/<>/subversion-1.14.2-man/share/man 
--infodir=/<>/subversion-1.14.2/share/info 
--docdir=/<>/subversion-1.14.2/share/doc/subversion 
--libdir=/<>/subversion-1.14.2/lib 
--libexecdir=/<>/subversion-1.14.2/libexec 
--localedir=/<>/subversion-1.14.2/share/locale --with-berkeley-db 
--without-apxs --without-swig --without-sasl --with-serf=/<>/serf-1.3.9 
--with-zlib=/<>/zlib-1.2.12-dev --with-sqlite=/<>/sqlite-3.38.5-dev 
--build=x86_64-unknown-linux-gnu --host=x86_64-unknown-linux-gnu
...
$ make -j16 -l16 SHELL=/<>/bash-5.1-p16/bin/bash 
APACHE_LIBEXECDIR=/<>/subversion-1.14.2/modules
...
$ make --shuffle APACHE_LIBEXECDIR=/<>/subversion-1.14.2/modules 
pkgconfigdir=/<>/subversion-1.14.2-dev/lib/pkgconfig 
m4datadir=/<>/subversion-1.14.2-dev/share/aclocal 
aclocaldir=/<>/subversion-1.14.2-dev/share/aclocal install
test -d /<>/subversion-1.14.2-dev/include/subversion-1 || \
  /<>/coreutils-9.1/bin/install -c -d 
/<>/subversion-1.14.2-dev/include/subversion-1
(subversion/svnversion/svnversion . 2> /dev/null ||  \
 svnversion . 2> /dev/null ||\
 echo "unknown"; \
) > /<>/subversion-1.14.2-dev/include/subversion-1/svn-revision.txt
/<>/coreutils-9.1/bin/install -c -d /<>/subversion-1.14.2/lib 
/<>/subversion-1.14.2/share/pkgconfig
cd subversion/libsvn_fs_base ; /<>/bash-5.1-p16/bin/bash 
"/build/subversion/libtool" --mode=install /<>/coreutils-9.1/bin/install 
-c libsvn_fs_base-1.la /<>/subversion-1.14.2/lib/libsvn_fs_base-1.la
libtool: warning: relinking 'libsvn_fs_base-1.la'
libtool: install: (cd /build/subversion/subversion/libsvn_fs_base; 
/<>/bash-5.1-p16/bin/bash "/build/subversion/libtool"  --tag CC --silent 
--mode=relink gcc -shared -g -O2 -g -O2 -L/<>/openssl-1.1.1o/lib 
-L/<>/db-5.3.28/lib -L/<>/expat-2.4.8/lib -L/<>/serf-1.3.9/lib 
-rpath /<>/subversion-1.14.2/lib -version-info 0 -Wl,--no-undefined -o 
libsvn_fs_base-1.la bdb/bdb-err.lo bdb/bdb_compat.lo bdb/changes-table.lo 
bdb/checksum-reps-table.lo bdb/copies-table.lo bdb/dbt.lo bdb/env.lo 
bdb/lock-tokens-table.lo bdb/locks-table.lo bdb/miscellaneous-table.lo 
bdb/node-origins-table.lo bdb/nodes-table.lo bdb/reps-table.lo bdb/rev-table.lo 
bdb/strings-table.lo bdb/txn-table.lo bdb/uuids-table.lo dag.lo err.lo fs.lo 
id.lo key-gen.lo lock.lo node-rev.lo reps-strings.lo revs-txns.lo trail.lo 
tree.lo util/fs_skels.lo uuid.lo 
../../subversion/libsvn_delta/libsvn_delta-1.la 
../../subversion/libsvn_subr/libsvn_subr-1.la -L/<>/apr-util-1.6.1/lib 
-laprutil-1 -L/<>/apr-1.7.0/lib -lapr-1 -ldb-5.3 
../../subversion/libsvn_fs_util/libsvn_fs_util-1.la )
/<>/binutils-2.38/bin/ld: cannot find -lsvn_delta-1: No such file or 
directory
/<>/binutils-2.38/bin/ld: cannot find -lsvn_fs_util-1: No such file or 
directory
collect2: error: ld returned 1 exit status
libtool:   error: error: relink 'libsvn_fs_base-1.la' with the above 
command before installing it
make: *** [build-outputs.mk:1422: install-bdb-lib] Error 1 
shuffle=1656015674

The failure as I understand it:

'install-bdb-lib' install target assumes that '-lsvn_delta-1' and 
'-lsvn_fs_util-1'
are already installed into 'DESTDIR'. But they are not. 'libtool' fails to 
relink
'libsvn_fs_base' against 'DESTDIR'.

I think the dependencies usually happen to be present if 'install-fsmod-lib' 
target
was lucky to be executed first. But nothing in 'Makefile.in' or 
'build-outputs.mk'
seems to have an equivalent of dependency like:

# hypothetical fix
install-bdb-lib : install-fsmod-lib

It looks like 'build.conf' does contain some build dependency information.

I also see special cases like fs-lib to handle some of install deps:

# manual
Makefile.in:SVN_FS_LIB_INSTALL_DEPS = @SVN_FS_LIB_INSTALL_DEPS@

# generated:
build-outputs.mk:install-ramod-lib: $(SVN_FS_LIB_INSTALL_DEPS)

Would it make sense to add 'install-bdb-lib : install-fsmod-lib' dependency 
explicitly?

Thank you!

-- 

  Sergei

[bug #62100] [PATCH] Add '--shuffle' argument support

[Sergei Trofimovich]

Additional Item Attachment, bug #62100 (project make):

File name: v7-0001-Add-shuffle-argument-support.patch Size:35 KB
   




___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #62100] [PATCH] Add '--shuffle' argument support

[Sergei Trofimovich]

Follow-up Comment #6, bug #62100 (project make):

Adding a v7-0001-Add-shuffle-argument-support.patch which 2 extra changes:
1. fixed 'if' indentation in src/main.c noticed by Fangrui Song
2. tweaked --help output to '--shuffle[={SEED|random|identity|reverse|none}]'
noticed by Fangrui Song


___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[gnugo-devel] [patch #10208] fix gcc-10 build failure (-fno-common)

[Sergei Trofimovich]

URL:
  <https://savannah.gnu.org/patch/?10208>

 Summary: fix gcc-10 build failure (-fno-common)
 Project: GNU Go
Submitted by: slyfox
Submitted on: Sat 21 May 2022 09:49:16 AM UTC
Category: None
Priority: 5 - Normal
  Status: None
 Privacy: Public
 Assigned to: None
Originator Email: 
 Open/Closed: Open
 Discussion Lock: Any


___

Follow-up Comments:


---
Date: Sat 21 May 2022 09:49:16 AM UTC By: Sergei Trofimovich 
gcc-10 changed the default from -fcommon to fno-common:
  https://gcc.gnu.org/PR85678

As a result build fails as:

ld: dfa.o:/build/gnugo-3.8/patterns/../engine/liberty.h:861: multiple
definition of
  `meaningless_white_moves';
mkpat.o:/build/gnugo-3.8/patterns/../engine/liberty.h:861: first defined here
ld: dfa.o:/build/gnugo-3.8/patterns/../engine/liberty.h:860: multiple
definition of
  `meaningless_black_moves';
mkpat.o:/build/gnugo-3.8/patterns/../engine/liberty.h:860: first defined here

The change moves variable definitions into a single .c file that uses them.







___

Reply to this item at:

  <https://savannah.gnu.org/patch/?10208>

___
  Message sent via Savannah
  https://savannah.gnu.org/


___
gnugo-devel mailing list
gnugo-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/gnugo-devel

[bug #62100] [PATCH] Add '--shuffle' argument support

[Sergei Trofimovich]

Follow-up Comment #4, bug #62100 (project make):

Caught bugs in a few upstream projects. Two nice simple examples:
- vim: https://github.com/vim/vim/pull/9978
- groff: https://savannah.gnu.org/bugs/?62084


___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #62084] Parallel build failure: fatal error: cannot load 'DESC' description file for device 'ps'

[Sergei Trofimovich]

Follow-up Comment #3, bug #62084 (project groff):

Survived 15 rebuilds for me on GNU make with --shuffle option
(https://savannah.gnu.org/bugs/index.php?62100). Thank you!


___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

Re: rx2660 + debian

[Sergei Trofimovich]

On Tue, 26 Apr 2022 08:18:25 +
Pedro Miguel Justo  wrote:

> > On 2022/Apr/26, at 01:01, Sergei Trofimovich  wrote:
> > 
> > On Tue, 26 Apr 2022 02:43:00 +
> > Pedro Miguel Justo  wrote:
> >   
> >>> On 2022/Apr/25, at 14:27, Pedro Miguel Justo  wrote:
> >>> 
> >>> 
> >>>   
> >>>> On 2022/Apr/25, at 14:09, Sergei Trofimovich  wrote:
> >>>> 
> >>>> On Mon, 25 Apr 2022 15:07:58 +
> >>>> Pedro Miguel Justo  wrote:
> >>>>   
> >>>>>> On 2022/Apr/25, at 01:22, Pedro Miguel Justo  wrote:
> >>>>>> 
> >>>>>> 
> >>>>>>   
> >>>>>>> On 2022/Apr/25, at 01:14, Frank Scheiner  
> >>>>>>> wrote:
> >>>>>>> 
> >>>>>>> Hi guys,
> >>>>>>> 
> >>>>>>> On 25.04.22 10:09, John Paul Adrian Glaubitz wrote:  
> >>>>>>>>> From what I can understand by the information in the bugcheck, this 
> >>>>>>>>> is somewhat related to a violation
> >>>>>>>>> in parameter copy from user to kernel during some boot-time, 
> >>>>>>>>> crypto, self-test. Does that sound right?
> >>>>>>>>> If that is the case, how would this be related to FW?  
> >>>>>>>> 
> >>>>>>>> I'm not claiming that it must be related to the firmware, I'm just 
> >>>>>>>> saying that I don't see this problem
> >>>>>>>> on my RX2660 at all and I have even reinstalled it recently with one 
> >>>>>>>> of the latest firmware images
> >>>>>>>> without having to pass any parameter to the command line.  
> >>>>>>> 
> >>>>>>> A difference between Adrian's rx2660 and Pedro's rx2660 is Montecito
> >>>>>>> left and Montvale right.
> >>>>>>> 
> >>>>>>> But could still be multiple other reasons we haven't looked at yet in
> >>>>>>> detail:
> >>>>>>> 
> >>>>>>> * amount of memory installed
> >>>>>>> * SMT enabled or not
> >>>>>>> * number of processor modules installed
> >>>>>>> 
> >>>>>>> It might be possible for me to check on my rx2660s (one with Montvale
> >>>>>>> and one with Montecito(s)) tomorrow. I will then also look at my other
> >>>>>>> Itanium gear and gather relevant information.
> >>>>>>>   
> >>>>>> 
> >>>>>> Yes, this sounds mode likely to me too.
> >>>>>> 
> >>>>>> The crypto self-tests seem to be an innocent bystander here. I tried 
> >>>>>> booting the most recent kernel with the option “cryptomgr.notests” and 
> >>>>>> it went much farther. Alas it still failed with another buffer copy 
> >>>>>> validation for a different caller altogether:
> >>>>>> 
> >>>>>> [3.836466]  [] usercopy_abort+0x120/0x130
> >>>>>> [3.836466] sp=e001000cfdf0 
> >>>>>> bsp=e001000c9388
> >>>>>> [3.836466]  [] __check_object_size+0x3c0/0x420
> >>>>>> [3.836466] sp=e001000cfe00 
> >>>>>> bsp=e001000c9350
> >>>>>> [3.836466]  [] sys_getcwd+0x250/0x420
> >>>>>> [3.836466] sp=e001000cfe00 
> >>>>>> bsp=e001000c92c8
> >>>>>> [3.836466]  [] ia64_ret_from_syscall+0x0/0x20
> >>>>>> [3.836466] sp=e001000cfe30 
> >>>>>> bsp=e001000c92c8
> >>>>>> [3.836466]  [] ia64_ivt+0x00040720/0x400
> >>>>>> [3.836466] sp=e001000d 
> >>>>>> bsp=e001000c92c8
> >>>>>> 
> >>>>>> This suggests the bug might be in the logic validating these buffers 
> >>>>>> against the allocations (heap, span, etc).
> >>>>>> 
> >>>>>> I don’t know w

Re: rx2660 + debian

[Sergei Trofimovich]

On Mon, 25 Apr 2022 15:07:58 +
Pedro Miguel Justo  wrote:

> > On 2022/Apr/25, at 01:22, Pedro Miguel Justo  wrote:
> > 
> > 
> >   
> >> On 2022/Apr/25, at 01:14, Frank Scheiner  wrote:
> >> 
> >> Hi guys,
> >> 
> >> On 25.04.22 10:09, John Paul Adrian Glaubitz wrote:  
>  From what I can understand by the information in the bugcheck, this is 
>  somewhat related to a violation
>  in parameter copy from user to kernel during some boot-time, crypto, 
>  self-test. Does that sound right?
>  If that is the case, how would this be related to FW?  
> >>> 
> >>> I'm not claiming that it must be related to the firmware, I'm just saying 
> >>> that I don't see this problem
> >>> on my RX2660 at all and I have even reinstalled it recently with one of 
> >>> the latest firmware images
> >>> without having to pass any parameter to the command line.  
> >> 
> >> A difference between Adrian's rx2660 and Pedro's rx2660 is Montecito
> >> left and Montvale right.
> >> 
> >> But could still be multiple other reasons we haven't looked at yet in
> >> detail:
> >> 
> >> * amount of memory installed
> >> * SMT enabled or not
> >> * number of processor modules installed
> >> 
> >> It might be possible for me to check on my rx2660s (one with Montvale
> >> and one with Montecito(s)) tomorrow. I will then also look at my other
> >> Itanium gear and gather relevant information.
> >>   
> > 
> > Yes, this sounds mode likely to me too.
> > 
> > The crypto self-tests seem to be an innocent bystander here. I tried 
> > booting the most recent kernel with the option “cryptomgr.notests” and it 
> > went much farther. Alas it still failed with another buffer copy validation 
> > for a different caller altogether:
> > 
> > [3.836466]  [] usercopy_abort+0x120/0x130
> > [3.836466] sp=e001000cfdf0 
> > bsp=e001000c9388
> > [3.836466]  [] __check_object_size+0x3c0/0x420
> > [3.836466] sp=e001000cfe00 
> > bsp=e001000c9350
> > [3.836466]  [] sys_getcwd+0x250/0x420
> > [3.836466] sp=e001000cfe00 
> > bsp=e001000c92c8
> > [3.836466]  [] ia64_ret_from_syscall+0x0/0x20
> > [3.836466] sp=e001000cfe30 
> > bsp=e001000c92c8
> > [3.836466]  [] ia64_ivt+0x00040720/0x400
> > [3.836466] sp=e001000d 
> > bsp=e001000c92c8
> > 
> > This suggests the bug might be in the logic validating these buffers 
> > against the allocations (heap, span, etc).
> > 
> > I don’t know why hardened_usercopy=off is not being observed by the kernel. 
> > As a work-around I am copying myself a new kernel with 
> > CONFIG_HARDENED_USERCOPY disabled at the source. 
> >   
> 
> Even with kernel "Linux debian 4.19.0-5-mckinley #1 SMP Debian 4.19.37-5 
> (2019-06-19) ia64 GNU/Linux"
> 
> Things are still not 100%. After a few hours into building the kernel it 
> started crashing also with usercopy validations but, this time, the other way 
> around. And because it was the other way around, it led to process 
> termination instead of full-blown bugcheck. This could be related or not. 
> Coule very well be a different bug that happens to manifest itself round the 
> same validation.
> 
>   CC [M]  drivers/net/wireless/realtek/rtw88/rtw8822be.o
>   LD [M]  drivers/net/wireless/realtek/rtw88/rtw88_8822be.o
>   CC [M]  drivers/net/wireless/realtek/rtw88/rtw8822c.o
> Segmentation fault
> make[5]: *** [scripts/Makefile.build:293: 
> drivers/net/wireless/realtek/rtw88/rtw8822c.o] Error 139
> make[5]: *** Deleting file 'drivers/net/wireless/realtek/rtw88/rtw8822c.o'
> make[4]: *** [scripts/Makefile.build:555: drivers/net/wireless/realtek/rtw88] 
> Error 2
> make[3]: *** [scripts/Makefile.build:555: drivers/net/wireless/realtek] Error 
> 2
> make[2]: *** [scripts/Makefile.build:555: drivers/net/wireless] Error 2
> make[1]: *** [scripts/Makefile.build:555: drivers/net] Error 2
> make: *** [Makefile:1855: drivers] Error 2
> pmsjt@debian:~/linux-source-5.17$ make
> 
> Message from syslogd@debian at Apr 25 07:58:08 ...
>  kernel:[23420.984012] usercopy: Kernel memory overwrite attempt detected to 
> linear kernel text (offset 1916912, size 8)!
> 
> Message from syslogd@debian at Apr 25 07:58:08 ...
>  kernel:[23421.268009] usercopy: Kernel memory overwrite attempt detected to 
> linear kernel text (offset 1818608, size 8)!
>   HOSTCC  scripts/sign-file
>   CALLscripts/checksyscalls.sh
> :1517:2: warning: #warning syscall clone3 not implemented [-Wcpp]
>   CALLscripts/atomic/check-atomics.sh
>   CHK include/generated/compile.h
> make[2]: *** [scripts/Makefile.build:294: arch/ia64/kernel/signal.o] 
> Segmentation fault
> 
> Message from syslogd@debian at Apr 25 07:58:11 ...
>  kernel:[23423.626254] usercopy: Kernel memory overwrite attempt detected to 
> linear kernel text (offset 1933296, size 8)!
> make[1]: *** 

[PATCH, gcc-11 backport] gcov-profile: Allow negative counts of indirect calls [PR105282]

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

TOPN metrics are histograms that contain overall count and per-bucket
count. Overall count can be negative when two profiles merge and some
of per-bucket metrics are disacarded.

Noticed as an ICE on python PGO build where gcc crashes as:

during IPA pass: modref
a.c:36:1: ICE: in stream_out_histogram_value, at value-prof.cc:340
   36 | }
  | ^
stream_out_histogram_value(output_block*, histogram_value_t*)
gcc/value-prof.cc:340

gcc/ChangeLog:

PR gcov-profile/105282
* value-prof.cc (stream_out_histogram_value): Allow negative counts
on HIST_TYPE_INDIR_CALL.

(cherry picked from commit 90a29845bfe7d6002e6c2fd49a97820b00fbc4a3)
---
 gcc/value-prof.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/gcc/value-prof.c b/gcc/value-prof.c
index 42748771192..688089b04d2 100644
--- a/gcc/value-prof.c
+++ b/gcc/value-prof.c
@@ -336,6 +336,10 @@ stream_out_histogram_value (struct output_block *ob, 
histogram_value hist)
/* Note that the IOR counter tracks pointer values and these can have
   sign bit set.  */
;
+  else if (hist->type == HIST_TYPE_INDIR_CALL && i == 0)
+   /* 'all' counter overflow is stored as a negative value. Individual
+  counters and values are expected to be non-negative.  */
+   ;
   else
gcc_assert (value >= 0);
 
-- 
2.35.1

[PATCH] gcov-profile: Allow negavive counts of indirect calls [PR105282]

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

TOPN metrics are histograms that contain overall count and per-bucket
count. Overall count can be nevative when two profiles merge and some
of per-bucket metrics are dropped.

Noticed as an ICE on python PGO build where gcc crashes as:

during IPA pass: modref
a.c:36:1: ICE: in stream_out_histogram_value, at value-prof.cc:340
   36 | }
  | ^
stream_out_histogram_value(output_block*, histogram_value_t*)
gcc/value-prof.cc:340

gcc/ChangeLog:

PR gcov-profile/105282
* value-prof.cc (stream_out_histogram_value): Allow negavive counts
on HIST_TYPE_INDIR_CALL.
---
 gcc/value-prof.cc | 4 
 1 file changed, 4 insertions(+)

diff --git a/gcc/value-prof.cc b/gcc/value-prof.cc
index 9785c7a03ea..4927d119aa0 100644
--- a/gcc/value-prof.cc
+++ b/gcc/value-prof.cc
@@ -319,40 +319,44 @@ stream_out_histogram_value (struct output_block *ob, 
histogram_value hist)
   streamer_write_bitpack ();
   switch (hist->type)
 {
 case HIST_TYPE_INTERVAL:
   streamer_write_hwi (ob, hist->hdata.intvl.int_start);
   streamer_write_uhwi (ob, hist->hdata.intvl.steps);
   break;
 default:
   break;
 }
   for (i = 0; i < hist->n_counters; i++)
 {
   /* When user uses an unsigned type with a big value, constant converted
 to gcov_type (a signed type) can be negative.  */
   gcov_type value = hist->hvalue.counters[i];
   if (hist->type == HIST_TYPE_TOPN_VALUES
  || hist->type == HIST_TYPE_IOR)
/* Note that the IOR counter tracks pointer values and these can have
   sign bit set.  */
;
+  else if (hist->type == HIST_TYPE_INDIR_CALL && i == 0)
+   /* 'all' counter overflow is stored as a negative value. Individual
+  counters and values are expected to be non-negative.  */
+   ;
   else
gcc_assert (value >= 0);
 
   streamer_write_gcov_count (ob, value);
 }
   if (hist->hvalue.next)
 stream_out_histogram_value (ob, hist->hvalue.next);
 }
 
 /* Dump information about HIST to DUMP_FILE.  */
 
 void
 stream_in_histogram_value (class lto_input_block *ib, gimple *stmt)
 {
   enum hist_type type;
   unsigned int ncounters = 0;
   struct bitpack_d bp;
   unsigned int i;
   histogram_value new_val;
   bool next;
-- 
2.35.1

Re: gcc inserts __builtin_popcount, causes 'modpost: "__popcountdi2" ... amdgpu.ko] undefined'

[Sergei Trofimovich]

On Mon, Apr 11, 2022 at 10:08:15PM +0100, Sergei Trofimovich wrote:
> Current linux-5.17.1 on fresh gcc-12 fails to build with errors like:
> 
> ERROR: modpost: "__popcountdi2" 
> [drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko] undefined!
> ERROR: modpost: "__popcountdi2" [drivers/gpu/drm/amd/amdgpu/amdgpu.ko] 
> undefined!
> 
> It is occasionally seen by others on previous gcc versions as well:
> 
> https://lkml.org/lkml/2021/7/11/261
> https://lkml.org/lkml/2018/10/24/403
> 
> '__popcountdi2' are inserted by gcc for code like the following
> from 'drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_execution.c':
> 
> static inline enum mod_hdcp_status validate_bksv(struct mod_hdcp *hdcp)
> {
> uint64_t n = 0;
> uint8_t count = 0;
> u8 bksv[sizeof(n)] = { };
> 
> memcpy(bksv, hdcp->auth.msg.hdcp1.bksv, 
> sizeof(hdcp->auth.msg.hdcp1.bksv));
> n = *(uint64_t *)bksv;
> 
> /* Here gcc inserts 'count = __builtin_popcount(n);' */
> while (n) {
> count++;
> n &= (n - 1);
> }
> 
> return (count == 20) ? MOD_HDCP_STATUS_SUCCESS :
>MOD_HDCP_STATUS_HDCP1_INVALID_BKSV;
> }
> 
> Note that gcc can insert it regardless of -mno-* options.
> 
> How should linux.git handle it? A few options come to mind:
> 
> - Perhaps use libgcc.a directly.
> - Just implement '__popcountdi2'. Example definition from libgcc:
>   https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=libgcc/libgcc2.c;hb=HEAD#l846
> - Or workaround it with -fno-builtin-popcount in Makefiles.
> 
> CCing people who can help routing it and/or deciding on the fix:
> amd-gfx@lists.freedesktop.org, Joe Perches, linux-kbu...@vger.kernel.org,
> Jakub Jelinek, Segher Boessenkool, Thomas Gleixner,Peter Zijlstra, Andy
> Lutomirski.

There is now a discussion in gcc bugzilla:

https://gcc.gnu.org/PR105253

-- 

  Sergei

gcc inserts __builtin_popcount, causes 'modpost: "__popcountdi2" ... amdgpu.ko] undefined'

[Sergei Trofimovich]

Current linux-5.17.1 on fresh gcc-12 fails to build with errors like:

ERROR: modpost: "__popcountdi2" 
[drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko] undefined!
ERROR: modpost: "__popcountdi2" [drivers/gpu/drm/amd/amdgpu/amdgpu.ko] 
undefined!

It is occasionally seen by others on previous gcc versions as well:

https://lkml.org/lkml/2021/7/11/261
https://lkml.org/lkml/2018/10/24/403

'__popcountdi2' are inserted by gcc for code like the following
from 'drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_execution.c':

static inline enum mod_hdcp_status validate_bksv(struct mod_hdcp *hdcp)
{
uint64_t n = 0;
uint8_t count = 0;
u8 bksv[sizeof(n)] = { };

memcpy(bksv, hdcp->auth.msg.hdcp1.bksv, 
sizeof(hdcp->auth.msg.hdcp1.bksv));
n = *(uint64_t *)bksv;

/* Here gcc inserts 'count = __builtin_popcount(n);' */
while (n) {
count++;
n &= (n - 1);
}

return (count == 20) ? MOD_HDCP_STATUS_SUCCESS :
   MOD_HDCP_STATUS_HDCP1_INVALID_BKSV;
}

Note that gcc can insert it regardless of -mno-* options.

How should linux.git handle it? A few options come to mind:

- Perhaps use libgcc.a directly.
- Just implement '__popcountdi2'. Example definition from libgcc:
  https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=libgcc/libgcc2.c;hb=HEAD#l846
- Or workaround it with -fno-builtin-popcount in Makefiles.

CCing people who can help routing it and/or deciding on the fix:
amd-gfx@lists.freedesktop.org, Joe Perches, linux-kbu...@vger.kernel.org,
Jakub Jelinek, Segher Boessenkool, Thomas Gleixner,Peter Zijlstra, Andy
Lutomirski.

WDYT?

Thanks!

-- 

  Sergei

[PATCH] libgcc: IA64: don't compile glibc-based unwinder without libc headers

[Sergei Trofimovich via Gcc-patches]

From: Sergei Trofimovich 

In --without-headers mode gcc fails to bootstrap on libgcc as:

/build/build/./gcc/xgcc -B/build/build/./gcc/ ... -Dinhibit_libc -c 
fde-glibc.c
../../../gcc-12-20220403/libgcc/config/ia64/fde-glibc.c:33:10:
fatal error: stdlib.h: No such file or directory

Most other linux targets are able to build the --without-headers
compiler without additional effort. This change adds IA64 to the fold.

The change drops part of the code that relies on DYNAMIC glibc
section traversal for backtraces.

Tested bootstrap of ia64-unknown-linux-gnu with and without libc
headers present.

libgcc/
* config/ia64/fde-glibc.c: Make a no-op in inhibit_libc mode.
---
 libgcc/config/ia64/fde-glibc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libgcc/config/ia64/fde-glibc.c b/libgcc/config/ia64/fde-glibc.c
index 9caac2bca24..bd74847fa85 100644
--- a/libgcc/config/ia64/fde-glibc.c
+++ b/libgcc/config/ia64/fde-glibc.c
@@ -22,20 +22,21 @@
see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
<http://www.gnu.org/licenses/>.  */
 
 /* Locate the FDE entry for a given address, using glibc ld.so routines
to avoid register/deregister calls at DSO load/unload.  */
 
 #ifndef _GNU_SOURCE
 #define _GNU_SOURCE 1
 #endif
 #include "config.h"
+#ifndef inhibit_libc
 #include 
 #include 
 #include 
 #include "unwind-ia64.h"
 
 #if __GLIBC__ < 2 || (__GLIBC__ == 2 && __GLIBC_MINOR__ < 2) \
 || (__GLIBC__ == 2 && __GLIBC_MINOR__ == 2 && !defined(DT_CONFIG))
 # error You need GLIBC 2.2.4 or later on IA-64 Linux
 #endif
 
@@ -152,10 +153,11 @@ _Unwind_FindTableEntry (void *pc, unw_word *segment_base, 
unw_word *gp,
   data.pc = (Elf64_Addr) pc;
   data.segment_base = segment_base;
   data.gp = gp;
   data.ret = NULL;
 
   if (dl_iterate_phdr (_Unwind_IteratePhdrCallback, ) < 0)
 return NULL;
 
   return data.ret;
 }
+#endif
-- 
2.35.1

[bug #62100] [PATCH] Add '--shuffle' argument support

[Sergei Trofimovich]

Follow-up Comment #3, bug #62100 (project make):

Uploaded v6-0001-Add-shuffle-argument-support.patch. It's the rebased v5
against today's make.git where a small merge conflict is fixed around added
option. Otherwise identical to v5.

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #62100] [PATCH] Add '--shuffle' argument support

[Sergei Trofimovich]

Additional Item Attachment, bug #62100 (project make):

File name: v6-0001-Add-shuffle-argument-support.patch Size:35 KB
   




___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #62100] [PATCH] Add '--shuffle' argument support

[Sergei Trofimovich]

Follow-up Comment #1, bug #62100 (project make):

Changes from v4: presence of `.NOTPARALLEL:` target disables shuffling as
makefile is already known to be broken in target reorder caused by -j
parallelism.

Example of real package is `netpbm`. Upstream does not plan to fix it.

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #62100] [PATCH] Add '--shuffle' argument support

[Sergei Trofimovich]

Additional Item Attachment, bug #62100 (project make):

File name: v5-0001-Add-shuffle-argument-support.patch Size:35 KB
   




___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #62100] [PATCH] Add '--shuffle' argument support

[Sergei Trofimovich]

URL:
  

 Summary: [PATCH] Add '--shuffle' argument support
 Project: make
Submitted by: slyfox
Submitted on: Wed 23 Feb 2022 07:51:47 AM UTC
Severity: 3 - Normal
  Item Group: None
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: None
Operating System: None
   Fixed Release: None
   Triage Status: None

___

Details:

The idea of the change is to introduce non-deterministic ordering
into goal and prerequisite traversal to imitate non-deterministic
ordering of 'make -j g1 g2 g3' mode.

Filing the bug to make sure it does not get lost.

The patch is against current git tree:


$ git describe
4.3-138-gfcc23a4d






___

File Attachments:


---
Date: Wed 23 Feb 2022 07:51:47 AM UTC  Name:
v4-0001-Add-shuffle-argument-support.patch  Size: 35KiB   By: slyfox



___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

Re: [PATCH] Fix src/function.c build failure on gcc-12.

[Sergei Trofimovich]

On Mon, 21 Feb 2022, 15:29 Paul Smith,  wrote:
>
> On Mon, 2022-02-21 at 08:59 +, Edward Welbourne wrote:
> > and that's assuming a 32-bit int; the signed range is from -
> > 2,147,483,647 to 2,147,483,648.  However, may I suggest the following
> > (which I know included in the GPL'd cfengine sources at some point):
>
> This computation is already included in GNU make (in Git) as the
> constant INTSTR_LENGTH (which uses sizeof(uintmax_t) as the basis),
> from a previous, similar suggestion you made a few months ago :).
>
> I modified this and a few other buffers that used static lengths, to
> use the INTSTR_LENGTH macro.

Sounds good. Which repository I should use to pull changes like that?
I looked at https://git.savannah.gnu.org/cgit/make.git and did not find it.

[PATCH] Test -flto on both compiler and linker

[Sergei Trofimovich]

Before the change ./configure incorrectly enabled -flto on toolchains
that support -flto on compiler side but don't support -flto on linker
side. This caused incorrect type size detection on nixpkgs' Darwin:

 configure:54594: checking size of size_t
 configure:54600: clang -std=gnu11 -o conftest -g -O2 -flto   conftest.c  >&5
 ld: warning: ignoring file 
/private/tmp/nix-build-guile-3.0.8.drv-0/conftest-00e93d.o,
   building for macOS-x86_64 but attempting to link with file built
   for unknown-unsupported file format ( 0xDE 0xC0 0x17 0x0B 0x00 0x00 0x00 
0x00 0x14 0x00 0x00 0x00 0x80 0x1A 0x00 0x00 )
 Undefined symbols for architecture x86_64:
   "_main", referenced from:
  implicit entry/start for main executable
 ld: symbol(s) not found for architecture x86_64
 clang-11: error: linker command failed with exit code 1 (use -v to see 
invocation)

Taken from https://github.com/NixOS/nixpkgs/pull/160051#issuecomment-1046105041

The change makes sure -flto support tests basic support of just for
object file generation but for linker as well.

* configure.ac: use AC_LINK_IFELSE instead of AC_COMPILE_IFELSE.
---
 configure.ac | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 827e1c09d..4064533b3 100644
--- a/configure.ac
+++ b/configure.ac
@@ -69,7 +69,7 @@ AC_MSG_CHECKING([whether the compiler supports -flto])
 old_CFLAGS="$CFLAGS"
 LTO_CFLAGS="-flto"
 CFLAGS="$CFLAGS $LTO_CFLAGS"
-AC_COMPILE_IFELSE([AC_LANG_SOURCE([int foo;])],, [LTO_CFLAGS=])
+AC_LINK_IFELSE([AC_LANG_PROGRAM([int foo;], [])],, [LTO_CFLAGS=])
 CFLAGS="$old_CFLAGS"
 if test -n "$LTO_CFLAGS"; then
   AC_MSG_RESULT([yes])
-- 
2.35.1

[PATCH v4] Add '--shuffle' argument support

[Sergei Trofimovich]

On Sun, Feb 20, 2022 at 10:30:10AM +, Sergei Trofimovich wrote:
> On Sat, Feb 19, 2022 at 09:57:13AM +0200, Eli Zaretskii wrote:

> > I think NEWS should also call out the new feature.

Forgot the actual NEWS entry. Attached v4.

-- 

  Sergei
>From 6eb8c759f6e917ff82326769e509505f8ff0e310 Mon Sep 17 00:00:00 2001
From: Sergei Trofimovich 
Date: Fri, 4 Feb 2022 22:43:28 +
Subject: [PATCH v4] Add '--shuffle' argument support

The idea of the change is to introduce non-deterministic ordering
into goal and prerequisite traversal to imitate non-deterministic
ordering of 'make -j g1 g2 g3' mode.

The implementation is to introduce second order into each dependency chain:
1. existing order is syntactic order reachable via 'dep->next'
2. new order is shuffled order stored as 'dep->shuf' in each 'dep'

When goals or prerequisites are updated we use shuffled order when
'--shuffle' is passed to make. When recipes are evaluated we use
syntactic order of parameters.

Sample execution looks like:

$ while echo === && rm -f a b && make --shuffle; do sleep 1; done
===
touch a
cp a b
===
cp a b
cp: cannot stat 'a': No such file or directory
make: *** [Makefile:5: b] Error 1 --shuffle=1644097687

Note: here first run succeeds while second run fails and exposes the bug in
Makefile. --shuffle=1644097687 allows us to rerun the same build sequence
again.

There are a few shuffle modes available:
- --shuffle (or --shuffle=random): use random seed and produce random order
- --shuffle=identity: use original order, but store and use it explicitly
  (most useful for GNU make testing)
- --shuffle=reverse: use inverse order for every goal list and prerequisite
  list. It is frequently enough to trigger simplest bugs.
- --shuffle=none: disable shuffle infrastructure completely. Useful to negate
  effect of previous --shuffle options.

* Makefile.am (make_SRCS): Add src/shuf.h and src/shuf.c file references.
* NEWS: Mention --shuffle option in news.
* builddos.bat: Add shuf.o into build script.
* build_w32.bat: Add shuf.c into build script.
* doc/make.1: Add '--shuffle' option description.
* doc/make.texi: Add '--shuffle' option description.
* makefile.com: Add shuf.c into build script.
* src/dep.h (DEP): Add 'shuf' field to store shuffled order.
* src/filedef.h (struct file): Add 'was_shuffled' bit flag to guard against
  circular dependencies.
* src/implicit.c (pattern_search): Reshuffle dependencies for targets
  dynamically extended with dependencies from implicit rules.
* src/job.c (child_error): Print shuffle parameter used for dependency
  shuffling.
* src/main.c: Add '--shuffle' option handling.
* src/remake.c (update_goal_chain): Change goal traversal order to shuffled.
* src/shuf.c: New file with shuffle infrastructure.
* src/shuf.h: New file with shuffle infrastructure declarations.
* tests/scripts/misc/rand-shuf: New file with randomness tests.
* tests/scripts/options/shuffle-reverse: New file with basic tests.
---
 Makefile.am   |   5 +-
 NEWS  |   6 +
 build_w32.bat |   1 +
 builddos.bat  |   3 +-
 doc/make.1|  34 
 doc/make.texi |  51 ++
 makefile.com  |   2 +-
 src/dep.h |   1 +
 src/filedef.h |   2 +
 src/implicit.c|   7 +
 src/job.c |  21 ++-
 src/main.c|  29 
 src/remake.c  | 138 ---
 src/shuf.c| 241 ++
 src/shuf.h|  23 +++
 tests/scripts/misc/rand-shuf  |  40 +
 tests/scripts/options/shuffle-reverse | 112 
 17 files changed, 644 insertions(+), 72 deletions(-)
 create mode 100644 src/shuf.c
 create mode 100644 src/shuf.h
 create mode 100644 tests/scripts/misc/rand-shuf
 create mode 100644 tests/scripts/options/shuffle-reverse

diff --git a/Makefile.am b/Makefile.am
index 9be60fec..3ad19e0c 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -35,8 +35,9 @@ make_SRCS =   src/ar.c src/arscan.c src/commands.c 
src/commands.h \
src/hash.c src/hash.h src/implicit.c src/job.c src/job.h \
src/load.c src/loadapi.c src/main.c src/makeint.h src/misc.c \
src/os.h src/output.c src/output.h src/read.c src/remake.c \
-   src/rule.c src/rule.h src/signame.c src/strcache.c \
-   src/variable.c src/variable.h src/version.c src/vpath.c
+   src/rule.c src/rule.h src/shuf.h src/shuf.c src/signame.c \
+   src/strcache.c src/variable.c src/variable.h src/version.c \
+   src/vpath.c
 
 w32_SRCS = src/w32/pathstuff.c src/w32/w32os.c src/w32/compat/dirent.c \
src/w32/compat/posixfcn

Re: [PATCH v2] Add '--shuffle' argument support

[Sergei Trofimovich]

On Sat, Feb 19, 2022 at 09:57:13AM +0200, Eli Zaretskii wrote:

> > * Makefile.am (make_SRCS): Add src/shuf.h and src/shuf.c file references.
> > * builddos.bat: Add shuf.o into build script.

> This should also update build_w32.bat, which is used to build Make on
> MS-Windows.  I think NEWS should also call out the new feature.

Sounds good! Added there and to makefile.com.

> > +Note that @code{make --shuffle clean all install} does reorder goals
> > +similar to how @code{make -j clean all install} runs all targets in
> 
> These should use @kbd, not @code, since you are describing commands
> the user will type.  I also recommend to wrap each one in @w{..}, so
> that these (long) command isn't broken between 2 lines.

Changed to @w{@kbd{..}}.

> > +@samp{--shuffle=} accepts an optional value:
> 
> I think nowadays it's customary to use @option as markup for
> command-line options (unless Make wants to keep its manual compatible
> to very old versions of Texinfo -- this is up to Paul to decide).

I left it as is as I see on @option{} being used in make.texi.
Can do as a separate patch if others agree.

> > +  /* TODO: could we make this shuffle more deterministic and less
> > + dependent on current filesystem state?  */
> > +  if (! file->was_shuffled)
> > +shuffle_file_deps_recursive (file);
> 
> Should this TODO be resolved before installing the feature?

Sounds good. Added re-seeding for each implicit dependency to get
more deterministic shuffling.

> > +  /* Handle shuffle mode argument.  */
> > +  if (shuffle_mode_arg)
> > +  {
> > +if (streq (shuffle_mode_arg, "none"))
> > +  sm = sm_none;
> > +else if (streq (shuffle_mode_arg, "identity"))
> > +  sm = sm_identity;
> > +else if (streq (shuffle_mode_arg, "reverse"))
> > +  sm = sm_reverse;
> > +else if (streq (shuffle_mode_arg, "random"))
> > +  sm = sm_random;
> > +/* Assume explicit seed if starts from a digit.  */
> > +else if (ISDIGIT (shuffle_mode_arg[0]))
> > +  {
> > +sm = sm_random;
> > +shuffle_seed = atoi (shuffle_mode_arg);
> > +  }
> > +else
> > +  {
> > +O (error, NILF, _("error: unsupported --shuffle= option."));
> > +die (MAKE_FAILURE);
> > +  }
> > +set_shuffle_mode (sm, shuffle_seed);
> > +
> > +/* Write fixed seed back to argument list to propagate fixed seed
> > +   to child $(MAKE) runs.  */
> > +free (shuffle_mode_arg);
> > +shuffle_mode_arg = xstrdup (shuffle_get_str_value ());
> > +  }
> 
> Should this be factored out and put on shuf.c?

Sounds goo.d Moved out to shuf.c. Only left small option rewrite bit in main.c.

> > +  switch (mode)
> > +{
> > +case sm_random:
> > +  if (seed == 0)
> > +seed = (int)time(NULL);
> > +  srand (seed);
> > +  shuffler = random_shuffle_array;
> > +  sprintf(shuffle_str_value, "%d", seed);
>^^
> Stylistic comment: I believe our style is to leave one space between
> the function name and the opening parenthesis (here and elsewhere in
> the patch).

Good catch! Added whitespace everywhere in the new code.

> > +random_shuffle_array (void ** a, size_t len)
> > +{
> > +  for (unsigned int i = 0; i < len; i++)
>  ^^^
> This requires some minimal level of ANSI C support that I'm not sure
> Make already requires.  Older compilers will error out here.

i think it does. Moved out to a separate declaration and switched to
size_t while at it to match function argument type.

> > +  /* TODO: below is almost identical to goaldeps shuffle.  The only 
> > difference
> > + is a type change.  Worth deduplicating?  */
> 
> Another TODO.

Deduplicated by coercing 'strct goaldep*' traversal into 'struct dep *'.
Similar to other dep.h primitives.

> > +  /* Shuffle dependencies. */
> > +  /* TODO: this is a naive recursion.  Is it good enough?  Or better 
> > change it
> > + to queue based implementation?  */
> 
> And another one.

Dropped a TODO as it's no worse than naive recursion in update_file_1.

> > --- /dev/null
> > +++ b/tests/scripts/options/shuffle-reverse
> 
> This test doesn't seem to test the random option.  I understand why
> this is not easy, but shouldn't it still be tested, if only to make
> sure the option works, and also that different seeds produce different
> outcomes?

Added tests/scripts/misc/rand-shuf test to test for random order and
for unchanged order for a fixed seed.

Attaching v3-0001-Add-shuffle

[bug #62084] Parallel build failure: fatal error: cannot load 'DESC' description file for device 'ps'

[Sergei Trofimovich]

URL:
  

 Summary: Parallel build failure: fatal error: cannot load
'DESC' description file for device 'ps'
 Project: GNU troff
Submitted by: slyfox
Submitted on: Sat 19 Feb 2022 10:13:26 AM UTC
Category: None
Severity: 3 - Normal
  Item Group: Build/Installation
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
 Planned Release: None

___

Details:

Incomplete dependencies in Makefile causes occasional parallel build failures
(make -j).

Here is seemingly deterministic way to make `groff` build fail in sequential
mode for me:


$ ./bootstrap
$ ./configure
$ make contrib/hdtbl/examples/col_rowspan_colors.ps
...
make contrib/hdtbl/examples/col_rowspan_colors.ps
  GROFFcontrib/hdtbl/examples/col_rowspan_colors.ps
/home/slyfox/dev/git/groff/groff: fatal error: cannot load 'DESC' description
file for device 'ps'
make: *** [Makefile:12392: contrib/hdtbl/examples/col_rowspan_colors.ps] Error
3
make: *** Deleting file 'contrib/hdtbl/examples/col_rowspan_colors.ps'






___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

make re-exec regression in 'make -sf -' mode

[Sergei Trofimovich]

Noticed the regression on lowdown-0.10.0 upstream package.

Here is a complete trigger:

$ printf 'all:\n\techo $(CC)' | make -sf -

[good] GNU make 4.3 works as expected:

$ printf 'all:\n\techo $(CC)' | make -sf -
cc

[bad] GNU make from git loops indefinitely in re-execution:

$ printf 'all:\n\techo $(CC)' | ./make -sf -


Curiously space separation workaround is enough to
get the result:

$ printf 'all:\n\techo $(CC)' | ./make -s -f -
cc

Bitsect points at commit 7c4e6b0299 "[SV 60595] Restart
whenever any makefile is rebuilt".

-- 

  Sergei

Re: [PATCH] RFC: add --shuffle[=seed] argument support

[Sergei Trofimovich]

On Sun, 6 Feb 2022 09:34:22 +
Sergei Trofimovich  wrote:

> On Sat, 05 Feb 2022 18:39:41 -0500
> Paul Smith  wrote:
> 
> > Nice work!
> > 
> > On Sat, 2022-02-05 at 22:04 +, Sergei Trofimovich wrote:  
> > > Some high level TODOs:
> > 
> > For this amount of change it's likely that you'll need to provide
> > copyright assignment paperwork.  Let me know if you'd like more details
> > about this.

Somehow managed to completely miss this part last time. Google (my
employer) has the assignment agreement with FSF. Do I also need to sign
personal agreement? I am UK citizen if that afects the decision.

Sent v2 of the patch with @google.com address as an author and attempted
to address almost everything you mentioned. The only thing I left in place
is goal reordering.

> > I recommend you try your version of GNU make on a bunch of different
> > real codebases and make sure it still works (and of course, create the
> > above-mentioned regression tests).  
> 
> Will do. I tried on 100 simple packages, but most of them are resursive
> makefiles (which I missed): 'toenv = 0' effectively disabled the option
> very early. Will try a few crafted inputs and handpick projects with
> handwritten build systems.

Tested '--shuffle=random' on ~2000 packages as part of NixOS distribution
bootstrap. It builds all dependencies from binutils, gcc and kernel up to
xorg. Build found a few real bugs on: ion-3, cramfsswap, dev86, libf2c,
groff, source-highlight, aspell, bind, pth, slang, gpm and subversion.

Having inspected failures manually most of the failures are real bugs of
missing dependencies and they came up before when 'make -j' was tested on
them.

cramfsswap is the shortest example:
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996964

-- 

  Sergei

[PATCH v2] Add '--shuffle' argument support

[Sergei Trofimovich]

From: Sergei Trofimovich 

The idea of the change is to introduce non-deterministic ordering
into goal and prerequisite traversal to imitate non-deterministic
ordering of 'make -j g1 g2 g3' mode.

The implementation is to introduce second order into each dependency chain:
1. existing order is syntactic order reachable via 'dep->next'
2. new order is shuffled order stored as 'dep->shuf' in each 'dep'

When goals or prerequisites are updated we use shuffled order when
'--shuffle' is passed to make. When recipes are evaluated we use
syntactic order of parameters.

Sample execution looks like:

$ while echo === && rm -f a b && make --shuffle; do sleep 1; done
===
touch a
cp a b
===
cp a b
cp: cannot stat 'a': No such file or directory
make: *** [Makefile:5: b] Error 1 --shuffle=1644097687

Note: here first run succeeds while second run fails and exposes the bug in
Makefile. --shuffle=1644097687 allows us to rerun the same build sequence
again.

* Makefile.am (make_SRCS): Add src/shuf.h and src/shuf.c file references.
* builddos.bat: Add shuf.o into build script.
* doc/make.1: Add '--shuffle' option description.
* doc/make.texi: Add '--shuffle' option description.
* src/dep.h (DEP): Add 'shuf' field to store shuffled order.
* src/filedef.h (struct file): Add 'was_shuffled' bit flag to guard against
  circular dependencies.
* src/implicit.c (pattern_search): Reshuffle dependencies for targets
  dynamically extended with dependencies from implicit rules.
* src/job.c (child_error): Print shuffle parameter used for dependency
  shuffling.
* src/main.c: Add '--shuffle' option handling.
* src/remake.c (update_goal_chain): Change goal traversal order to shuffled.
* src/shuf.c: New file with shuffle infrastructure.
* src/shuf.h: New file with shuffle infrastructure declarations.
* tests/scripts/options/shuffle-reverse: New file with basic tests.
---
 Makefile.am   |   5 +-
 builddos.bat  |   3 +-
 doc/make.1|  34 +
 doc/make.texi |  51 +++
 src/dep.h |   1 +
 src/filedef.h |   2 +
 src/implicit.c|   9 ++
 src/job.c |  19 ++-
 src/main.c|  45 ++
 src/remake.c  |  54 ---
 src/shuf.c| 202 ++
 src/shuf.h|  34 +
 tests/scripts/options/shuffle-reverse | 112 ++
 13 files changed, 542 insertions(+), 29 deletions(-)
 create mode 100644 src/shuf.c
 create mode 100644 src/shuf.h
 create mode 100644 tests/scripts/options/shuffle-reverse

diff --git a/Makefile.am b/Makefile.am
index 9be60fec..3ad19e0c 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -35,8 +35,9 @@ make_SRCS =   src/ar.c src/arscan.c src/commands.c 
src/commands.h \
src/hash.c src/hash.h src/implicit.c src/job.c src/job.h \
src/load.c src/loadapi.c src/main.c src/makeint.h src/misc.c \
src/os.h src/output.c src/output.h src/read.c src/remake.c \
-   src/rule.c src/rule.h src/signame.c src/strcache.c \
-   src/variable.c src/variable.h src/version.c src/vpath.c
+   src/rule.c src/rule.h src/shuf.h src/shuf.c src/signame.c \
+   src/strcache.c src/variable.c src/variable.h src/version.c \
+   src/vpath.c
 
 w32_SRCS = src/w32/pathstuff.c src/w32/w32os.c src/w32/compat/dirent.c \
src/w32/compat/posixfcn.c src/w32/include/dirent.h \
diff --git a/builddos.bat b/builddos.bat
index 9cecabec..cfb224c2 100644
--- a/builddos.bat
+++ b/builddos.bat
@@ -68,12 +68,13 @@ gcc -c -I./src -I%XSRC%/src -I./lib -I%XSRC%/lib 
-DHAVE_CONFIG_H -O2 -g %XSRC%/s
 gcc -c -I./src -I%XSRC%/src -I./lib -I%XSRC%/lib -DHAVE_CONFIG_H -O2 -g 
%XSRC%/src/remote-stub.c -o remote-stub.o
 gcc -c -I./src -I%XSRC%/src -I./lib -I%XSRC%/lib -DHAVE_CONFIG_H -O2 -g 
%XSRC%/src/getopt.c -o getopt.o
 gcc -c -I./src -I%XSRC%/src -I./lib -I%XSRC%/lib -DHAVE_CONFIG_H -O2 -g 
%XSRC%/src/getopt1.c -o getopt1.o
+gcc -c -I./src -I%XSRC%/src -I./lib -I%XSRC%/lib -DHAVE_CONFIG_H -O2 -g 
%XSRC%/src/shuf.c -o shuf.o
 gcc -c -I./src -I%XSRC%/src -I./lib -I%XSRC%/lib -DHAVE_CONFIG_H -O2 -g 
%XSRC%/lib/glob.c -o lib/glob.o
 gcc -c -I./src -I%XSRC%/src -I./lib -I%XSRC%/lib -DHAVE_CONFIG_H -O2 -g 
%XSRC%/lib/fnmatch.c -o lib/fnmatch.o
 @echo off
 echo commands.o > respf.$$$
 for %%f in (job output dir file misc main read remake rule implicit default 
variable) do echo %%f.o >> respf.$$$
-for %%f in (expand function vpath hash strcache version ar arscan signame 
remote-stub getopt getopt1) do echo %%f.o >> respf.$$$
+for %%f in (expand function vpath hash strcache version ar arscan signame 
remote-stub getopt getopt1 shuf) do echo %%f.o >> respf.$$$
 for 

[PATCH] Fix src/function.c build failure on gcc-12.

[Sergei Trofimovich]

From: Sergei Trofimovich 

Upcoming gcc-12 detects possible buffer overflow for 1 byte:

src/function.c: In function 'func_call':
src/function.c:2781:24: error: '__builtin___sprintf_chk' may write a 
terminating nul past the end of the destination [-Werror=format-overflow=8]
 2781 |   sprintf (num, "%d", i);
  |^
In file included from glibc-2.35-dev/include/stdio.h:894,
 from src/makeint.h:89,
 from src/function.c:17:
In function 'sprintf',
inlined from 'func_call' at src/function.c:2781:7:
glibc-2.35-dev/include/bits/stdio2.h:38:10: note: '__builtin___sprintf_chk' 
output between 2 and 12 bytes into a destination of size 11

Unlikely numbers like '-1234567890' including null terminator take 12
bytes of storage.

* src/function.c: Allocate enough storage for num
---
 src/function.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/function.c b/src/function.c
index 9add8f65..af2095da 100644
--- a/src/function.c
+++ b/src/function.c
@@ -2776,7 +2776,7 @@ func_call (char *o, char **argv, const char *funcname 
UNUSED)
 
   for (; i < max_args; ++i)
 {
-  char num[11];
+  char num[12];
 
   sprintf (num, "%d", i);
   define_variable (num, strlen (num), "", o_automatic, 0);
-- 
2.35.1

Re: [PATCH] RFC: add --shuffle[=seed] argument support

[Sergei Trofimovich]

On Sat, 05 Feb 2022 18:39:41 -0500
Paul Smith  wrote:

> Nice work!
> 
> On Sat, 2022-02-05 at 22:04 +0000, Sergei Trofimovich wrote:
> > Some high level TODOs:  
> 
> For this amount of change it's likely that you'll need to provide
> copyright assignment paperwork.  Let me know if you'd like more details
> about this.
> 
> > - No documentation for the optin yet.  
> 
> This feature would also need a set of regression tests.

Sounds good. Will start adding the tests.

> > - No(?) environment passing for recursive make.  I would prefer to
> > share the same see across all calls to get easier reproducers.  
> 
> You explicitly disabled this, though... was there a reason?

Just a bug. I misinterpreted the 'toenv' meaning.

> > - The dependency traversal is naive and uses potentially unbounded
> > stack.
> > - srand() / rand() is used from system libc.  This might make it
> > harder to reproduce a failure on another machine.  But maybe it's
> > fine.  
> 
> There are a few issues here:
> 
> I recommend you try your version of GNU make on a bunch of different
> real codebases and make sure it still works (and of course, create the
> above-mentioned regression tests).

Will do. I tried on 100 simple packages, but most of them are resursive
makefiles (which I missed): 'toenv = 0' effectively disabled the option
very early. Will try a few crafted inputs and handpick projects with
handwritten build systems.

> First, I think it's not correct to shuffle the goaldeps.  The goals
> that are specified on the command line should always be invoked in the
> order that the user requested.  It would be bad to convert:
> 
>   make clean all install
> 
> to:
> 
>   make install clean all

Just to clarify for myself: I agree changing the order of MAKECMDGOALS is
unexpected and harmful as it's observable in rules definitions (like
'echo $MAKECMDGOALS'). But I also think reordering rule execution should
be fine as it would be close to effect of -j in:

make -j clean all install

Does it sound about right? Or there is some subtler difference between goal
order and prerequisite order treatment?

> Second, you cannot rearrange the first prerequisite.  The first
> prerequisite always must be placed in $<.  Consider the simple pattern
> rule:
> 
>   %.o : %.c
>   $(CC) $(CFLAGS) -c -o $@ $<
> 
>   foo.o: foo.c foo.h bar.h baz.h
> 
> If you rearrange the prerequisites to "bar.h foo.h foo.c baz.h" it will
> not work well :).
> 
> Lastly, I'm not entirely sure that this is the best way to do the
> shuffle.  Similar to my concern above about the first prerequisite,
> this change will break makefiles that rely on the order of
> prerequisites in perfectly legitimate ways; for example:
> 
>   foo%: arg%1 arg%2 arg%3 arg%4
>   bld $< $(word 3,$^) $(word 2,$^) $(word 4,$^)
> 
> The concept you want to implement is not the shuffling of the actual
> prerequisites, it's the shuffling of the BUILDING of the prerequisites.
> The list of deps should not be modified but instead when make goes to
> build the deps it should build them in a different order than they
> appear in the prerequisites list.
> 
> Put another way, you don't want to change the structure of make's
> dependency graph; you just want to change the order in which it's
> walked.
> 
> If you do it this way you don't have to worry about any of the
> reordering issues I raise above, because the values of $<, $^, etc.
> won't change, and also you won't have to worry about deep recursions
> etc. because you'll just be rearranging the targets that are being
> built.
> 
> But, I think making it work this way will be trickier to code.

Oh, I did not realize prerequisite reordering completely breaks rule
definitions! That makes sense.

I'll spend some time to understand where I can plug in to reshuffle
schedulable queue instead.

Ideally I would like to preserve the order of execution across
incremental runs of:

make --shuffle=$seed foo
make --shuffle=$seed foo

but was afraid of the change of already satisfied prerequisites. Might
have to abandon the ideal for simplest first implementation.

Thank you for the detailed comment!

-- 

  Sergei

[PATCH] RFC: add --shuffle[=seed] argument support

[Sergei Trofimovich]

The idea of the change is to introduce non-deterministic ordering
into goal and prerequisite traversal to imitate non-deterministic
ordering of 'make -j' mode.

The implementation is to reorder lists of goals to build and list
of dependencies attached to files reachable via goals.

tested on the following problematic Makefile:

all: a b
a:
touch a
b:
cp a b

Sample execution looks like:

$ while echo === && rm -f a b && make --shuffle; do sleep 1; done
===
random seed: --shuffle=1644097687
touch a
cp a b
===
random seed: --shuffle=1644097688
cp a b
cp: cannot stat 'a': No such file or directory
make: *** [Makefile:5: b] Error 1

Note: here first run succeeds while second run fails exposing the failure.

Some high level TODOs:
- No documentation for the optin yet.
- No(?) environment passing for recursive make.  I would prefer to share
  the same see across all calls to get easier reproducers.
- The dependency traversal is naive and uses potentially unbounded stack.
- srand() / rand() is used from system libc.  This might make it harder
  to reproduce a failure on another machine.  But maybe it's fine.

If the general shape is reasonable I can try to get it up to standard.

What do you think?

* src/dep.h: Add 'shuffle_goaldeps_recursive' helper.
* src/filedef.h (struct file): Add 'was_shuffled' bit to track visited files.
* src/main.c: Add '--shuffle' argument and it's handling.
* src/misc.c: Implement 'shuffle_goaldeps_recursive()' helper.
---
 src/dep.h |   1 +
 src/filedef.h |   2 +
 src/main.c|  23 +++
 src/misc.c| 109 ++
 4 files changed, 135 insertions(+)

diff --git a/src/dep.h b/src/dep.h
index e492a0b3..2b098650 100644
--- a/src/dep.h
+++ b/src/dep.h
@@ -135,3 +135,4 @@ struct dep *copy_dep_chain (const struct dep *d);
 struct goaldep *read_all_makefiles (const char **makefiles);
 void eval_buffer (char *buffer, const floc *floc);
 enum update_status update_goal_chain (struct goaldep *goals);
+struct goaldep * shuffle_goaldeps_recursive(struct goaldep * g);
diff --git a/src/filedef.h b/src/filedef.h
index 0b9f6e17..d36a6011 100644
--- a/src/filedef.h
+++ b/src/filedef.h
@@ -108,6 +108,8 @@ struct file
pattern-specific variables.  */
 unsigned int no_diag:1; /* True if the file failed to update and no
diagnostics has been issued (dontcare). */
+unsigned int was_shuffled:1; /* Did we already shuffle 'deps'? used when
+--shuffle passes through the graph.  */
   };
 
 
diff --git a/src/main.c b/src/main.c
index 2d98a64b..9731f3d3 100644
--- a/src/main.c
+++ b/src/main.c
@@ -233,6 +233,12 @@ static const int inf_jobs = 0;
 
 static char *jobserver_auth = NULL;
 
+#define NO_SHUFFLE (-1)
+#define RANDOM_SHUFFLE (0)
+static int arg_shuffle_seed = NO_SHUFFLE;
+static const int no_shuffle = NO_SHUFFLE;
+static const int random_shuffle = RANDOM_SHUFFLE;
+
 /* Handle for the mutex used on Windows to synchronize output of our
children under -O.  */
 
@@ -358,6 +364,8 @@ static const char *const usage[] =
 N_("\
   -R, --no-builtin-variables  Disable the built-in variable settings.\n"),
 N_("\
+  --shuffle[=seed]Perform random shuffle on prerequisites.\n"),
+N_("\
   -s, --silent, --quiet   Don't echo recipes.\n"),
 N_("\
   --no-silent Echo recipes (disable --silent mode).\n"),
@@ -472,6 +480,8 @@ static const struct command_switch switches[] =
 { CHAR_MAX+8, flag_off, _flag, 1, 1, 0, 0, _silent_flag,
   "no-silent" },
 { CHAR_MAX+9, string, _auth, 1, 0, 0, 0, 0, "jobserver-fds" },
+{ CHAR_MAX+10, positive_int, _shuffle_seed, 1, 0, 0, _shuffle,
+  _shuffle, "shuffle" },
 { 0, 0, 0, 0, 0, 0, 0, 0, 0 }
   };
 
@@ -1498,6 +1508,11 @@ main (int argc, char **argv, char **envp)
   arg_job_slots = env_slots;
   }
 
+  if (arg_shuffle_seed == RANDOM_SHUFFLE)
+  arg_shuffle_seed = (int)time(NULL);
+  if (arg_shuffle_seed != NO_SHUFFLE)
+  printf("random seed: --shuffle=%i\n", arg_shuffle_seed);
+
   /* Set a variable specifying whether stdout/stdin is hooked to a TTY.  */
 #ifdef HAVE_ISATTY
   if (isatty (fileno (stdout)))
@@ -2645,6 +2660,14 @@ main (int argc, char **argv, char **envp)
   O (fatal, NILF, _("No targets specified and no makefile found"));
 }
 
+  /* Shuffle prerequisites to catch makefiles with incomplete depends. */
+  if (arg_shuffle_seed != NO_SHUFFLE)
+{
+  /* TODO: provide portable deterministic rand/srand.  */
+  srand(arg_shuffle_seed);
+  goals = shuffle_goaldeps_recursive (goals);
+}
+
   /* Update the goals.  */
 
   DB (DB_BASIC, (_("Updating goal targets\n")));
diff --git a/src/misc.c b/src/misc.c
index f400135b..b46c6d07 100644
--- a/src/misc.c
+++ b/src/misc.c
@@ -854,3 +854,112 @@ mempcpy (void *dest, 

Re: Idea of triggering bugs in users' Makefiles

[Sergei Trofimovich]

On Fri, 04 Feb 2022 10:27:01 -0500
Paul Smith  wrote:

> On Fri, 2022-02-04 at 09:13 +0000, Sergei Trofimovich wrote:
> > 1. Enable parallel builds by GNU make by default
> > 
> > 2. Do not run dependencies in deterministic order by default:  
> 
> GNU make (unlike Ninja) is bound by the POSIX standard in terms of its
> behaviors.  The POSIX standard makes very clear the way in which make
> operates on prerequisites and neither of these changes are conforming
> to the standard.
> 
> So, it's not possible to modify the default behavior of GNU make in
> these  ways.  Not to mention that GNU make is also a foundational
> component of hundreds of thousands of build systems and changes that
> would cause a large number of them to break isn't acceptable, even in
> pursuit of a good cause.

Sounds good.

> The idea of having a non-deterministic order of prerequisite builds is
> something others have suggested but no one has implemented.  Of course
> it would not be possible to move the first prerequisite as that one is
> special but randomly rearranging the rest of them could be done.  This
> would require an option to enable of course; it could not be the
> default behavior.

Thank you!

-- 

  Sergei

Idea of triggering bugs in users' Makefiles

[Sergei Trofimovich]

Individual developers more often than not build large projects
in parallel to speed the process up with 'make -j '.

OS distributions would also prefer to enable parallel builds
by default. But there is a problem of deciding if it's safe
to build a package in general.

In small projects developers frequently forget that their
Makefile might be used in a 'make -j' setting and don't test
the scenario.

Recently I attempted to enable 'make -j $(nproc)' instead of
'make' for the whole NixOS software collection.

I found quite a few build failures. Most of them have very easy
fixes. The good simple example is 'cramfsswap':

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996964

There one of the targets uses effect of another target, but does
not declare it. Contrived example:

# Makefile:
all: a b
a:
touch a
b:
cp a b

Here 'make all' happens to work, but 'make b' alone does not.

This made me thinking: what could GNU make do to make these
types of errors more obvious?

I cane up with two ideas:

1. Enable parallel builds by GNU make by default, so 'make'
   would mean 'make -j $(nproc)' unless asked otherwise.

   Some build systems (like ninja or waf) already do it.
   While sounding drastic I think it's a reasonable change
   longer term.

2. Do not run dependencies in deterministic order by default:

   Even when make is ran sequentially GNU make could do some
   effort to reorder targets to have a higher chance of executing
   steps in unexpected order.

   A possible implementation would be to pick a random seed at
   GNU make startup and perform target order permutation when
   allowed. If build were to fail random seed would be printed
   along with failure message. That way it would be trivial for
   others to reproduce the failure. Sometimes build failures
   are very hard to replicate.

I think both are worth implementing.

What do you think? Would it be reasonable to implement any or
both of these changes for GNU make?

Thanks!

-- 

  Sergei

Re: hello-2.11 fails multibyte-1 and atexit-1 on musl

[Sergei Trofimovich]

On Mon, 31 Jan 2022 11:39:47 +
Reuben Thomas  wrote:

> Thanks very much for the analysis and suggested patch.
> 
> I don't think however that a code change is needed. Arguably, the
> correct fix would be to check the return value of wprintf, but in fact
> close_stdout() already checks if there was a previous error on the
> stream. I think that arbitrarily setting a large buffer size is a bit
> of a hack, to be honest.
> 
> So I think it's sufficient, at the cost of a slightly less nice error
> message, to allow the code to proceed as at present, and instead to
> change the test so that it does not specifically check for "No space
> left on device", but instead just "write error".
> 
> I attach an updated version of tests/atexit-1; please could you check
> that with it the test now passes with musl? (Also, of course let me
> know if you disagree with my analysis!)

Your varian of atext-1 works on musl for me. Thank you!

-- 

  Sergei

Re: hello-2.11 fails multibyte-1 and atexit-1 on musl

[Sergei Trofimovich]

On Thu, 27 Jan 2022 22:08:57 +
Reuben Thomas  wrote:

> On Wed, 26 Jan 2022 at 12:59, Sergei Trofimovich  wrote:
> >  
> 
> Thanks for the report!
> 
> > From what I understand at
> > 
> > https://wiki.musl-libc.org/functional-differences-from-glibc.html#Character-sets-and-locale
> > musl deliberately supports any char -> wchar_t (and back) conversion for
> > any char in LC_CTYPE=C. And thus the multibyte-1 test will always fail
> > there.  
> 
> I also found this problem on macOS. I tried on my GNU/Linux system
> using an ISO-8859 locale for the test, but the multibyte conversion
> does not fail! (e.g. in en_GB.iso88591). I also tried with
> ru_RU.KOI8-R, same result.
> 
> It seems then that it's not going to be possible in general to find a
> locale that will fail. These results suggest that on some libc's,
> conversion will always succeed.
> 
> Hence, I'm removing the test.
> 
> > Not sure why atexit-1 fails. It seems to rely on
> > close_stdout_set_file_name() call in atexit(), but I don't see where in
> > code it's registered to call.  
> 
> Line 155 registers close_stdout. I can't see anything wrong with the
> way this is done; I'd be very grateful if you could investigate
> further.

Aha, I misinterpreted the atexit handler and did not notice
close_stdout_set_file_name() is part of gnulib.

Attached possible patch. Passes atexit-1 test on musl and glibc.

Some details:

The real reason to failure on musl is stream status and errno value
status in close_stream(). If I add a bit of fprintf() debugging it
becomes clearer:

  void
  close_stdout (void)
  {
fprintf(stderr, "%s: errno=%d\n", __func__, errno);
if (close_stream (stdout) != 0
&& !(ignore_EPIPE && errno == EPIPE))
  {
fprintf(stderr, "%s: after close_stream() errno=%d\n", __func__, errno);
  ...


Execution on glibc:

  close_stdout: errno=0
  close_stdout: after close_stream() errno=28
  ./hello: write error: No space left on device


Execution on glibc:

  close_stdout: errno=28
  close_stdout: after close_stream() errno=0
  hello: write error

On glibc stdout flush did not happen yet by the time we get to atexit().
ltrace also gives the same hint:

On glibc:

$ ltrace ./hello
...
strlen("Hello, world!") = 13
malloc(56)  = 0x10c78c0
mbsrtowcs(0x10c78c0, 0x7ffeb2b630a0, 14, 0x7ffeb2b630a8)= 13
wprintf(0x408298, 0x10c78c0, 177, 0)= 14
free(0x10c78c0) = 
exit(0 
__errno_location()  = 0x7fa894ceb548
fprintf(0x7fa894cdf440, "%s: errno=%d\n", "close_stdout", 0close_stdout: errno=0
) = 22
__fpending(0x7fa894cdf520, 0x7ffeb2b609d0, 0, 0x7fa894c11343) = 14
ferror(0x7fa894cdf520)  = 0
fclose(0x7fa894cdf520)  = -1
fprintf(0x7fa894cdf440, "%s: after close_stream() errno=%"..., "close_stdout", 
28close_stdout: after close_stream() errno=28
) = 44
dcgettext(0, 0x40836e, 5, 0x7fa894c11343)   = 0x40836e
error(0, 28, 0x4080c8, 0x40836e./hello: write error: No space left on device
)= 0
_exit(1 
+++ exited (status 1) +++

Note that wprintf() queued 14 symbols successfully.

On musl queueing did not happen and thus stream close did not return error:

$ ltrace ./hello
...
strlen("Hello, world!") = 13
malloc(56)  = 0x5570a96ddb10
mbsrtowcs(0x5570a96ddb10, 0x7ffdcb755398, 14, 0x7ffdcb7553a0) = 13
wprintf(0x5570a96d9298, 0x5570a96ddb10, 0xff3e, 1)  = 0x
free(0x5570a96ddb10)= 
exit(0 
__errno_location()  = 0x7fad63ed0b1c
fprintf(0x7fad63ece0c0, "%s: errno=%d\n", "close_stdout", 28close_stdout: 
errno=28
) = 23
__fpending(0x7fad63ece2c0, 0x7ffdcb755040, 0x, 0)   = 0
ferror(0x7fad63ece2c0)  = 1
fclose(0x7fad63ece2c0)  = 0
__errno_location()  = 0x7fad63ed0b1c
fprintf(0x7fad63ece0c0, "%s: after close_stream() errno=%"..., "close_stdout", 
0close_stdout: after close_stream() errno=0
) = 43
gettext(0x5570a96d92ba, 0x7ffdcb755040, 0x, 0)  = 0x5570a96d92ba
fileno(0x7fad63ece2c0)  = 1
fcntl(1, 3, 0x636f6c2f7273752f, 0x5570a96d92ba) = -1
fprintf(0x7fad63ece0c0, "%s: ", "hello"hello: )= 7
vfprintf(0x7fad63ece0c0, "%s", 0x7ffdcb755230write error)  = 11

hello-2.11 fails multibyte-1 and atexit-1 on musl

[Sergei Trofimovich]

Hi hello maintainers!

I attempted to package hello-2.11 in NixOS and stumbled on
two test failures on x86_64-unknown-linux-musl:

FAIL: tests/atexit-1
FAIL: tests/multibyte-1

Detailed log:

FAIL: tests/atexit-1


1c1
< hello: write error: No space left on device
---
> hello: write error
FAIL tests/atexit-1 (exit status: 1)

FAIL: tests/multibyte-1
===

1c1
< hello: conversion to a multibyte string failed: Invalid or incomplete 
multibyte or wide character
---
> hello 世界
FAIL tests/multibyte-1 (exit status: 1)

>From what I understand at

https://wiki.musl-libc.org/functional-differences-from-glibc.html#Character-sets-and-locale
musl deliberately supports any char -> wchar_t (and back) conversion for
any char in LC_CTYPE=C. And thus the multibyte-1 test will always fail
there.

Not sure why atexit-1 fails. It seems to rely on
close_stdout_set_file_name() call in atexit(), but I don't see where in
code it's registered to call.

Thanks!

-- 

  Sergei

[dm-devel] [PATCH 4/4] multipath-tools: allow passing non-standard linux-headers location

[Sergei Trofimovich]

On NixOS nothing is installed in /usr/include and instead lives
in it's own prefix.

The change switches linux/nvme_ioctl.h discovery to user provided path.

CC: Martin Wilck 
CC: Benjamin Marzinski 
Signed-off-by: Sergei Trofimovich 
---
 Makefile.inc   | 5 +
 libmultipath/prioritizers/Makefile | 2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/Makefile.inc b/Makefile.inc
index f5509db4..a7d16dfd 100644
--- a/Makefile.inc
+++ b/Makefile.inc
@@ -68,6 +68,11 @@ ifndef LIBUDEV_INCDIR
endif
 endif
 
+# Allow user to override default location.
+ifndef LINUX_HEADERS_INCDIR
+   LINUX_HEADERS_INCDIR = /usr/include
+endif
+
 prefix =
 exec_prefix= $(prefix)
 usr_prefix = $(prefix)
diff --git a/libmultipath/prioritizers/Makefile 
b/libmultipath/prioritizers/Makefile
index 8d34ae32..16c63977 100644
--- a/libmultipath/prioritizers/Makefile
+++ b/libmultipath/prioritizers/Makefile
@@ -23,7 +23,7 @@ LIBS = \
libpriopath_latency.so \
libpriosysfs.so
 
-ifneq ($(call check_file,/usr/include/linux/nvme_ioctl.h),0)
+ifneq ($(call check_file,$(LINUX_HEADERS_INCDIR)/linux/nvme_ioctl.h),0)
LIBS += libprioana.so
CFLAGS += -I../nvme
 endif
-- 
2.34.1

--
dm-devel mailing list
dm-devel@redhat.com
https://listman.redhat.com/mailman/listinfo/dm-devel

[dm-devel] [PATCH 1/4] multipath-tools: avoid using GZIP Makefile variable

[Sergei Trofimovich]

`gzip` supports (deprecated) `GZIP` environment variable. If it's
already present Makefile would override it and pass it through causing
thre breakage:

$ dev>GZIP=-n make
gzip -9 -c mpath_persistent_reserve_in.3 > mpath_persistent_reserve_in.3.gz
gzip: -c: option not valid in GZIP environment variable
Try `gzip --help' for more information.

Fix build by renaming GZIP variable to GZIP_PROG to avoid collision.

CC: Martin Wilck 
CC: Benjamin Marzinski 
Signed-off-by: Sergei Trofimovich 
---
 Makefile.inc | 2 +-
 kpartx/Makefile  | 2 +-
 libmpathpersist/Makefile | 4 ++--
 mpathpersist/Makefile| 2 +-
 multipath/Makefile   | 4 ++--
 multipathd/Makefile  | 2 +-
 6 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/Makefile.inc b/Makefile.inc
index b340f2ae..59856f24 100644
--- a/Makefile.inc
+++ b/Makefile.inc
@@ -75,7 +75,7 @@ nvmedir   = $(TOPDIR)/libmultipath/nvme
 includedir = $(prefix)/usr/include
 pkgconfdir = $(usrlibdir)/pkgconfig
 
-GZIP   = gzip -9 -c
+GZIP_PROG  = gzip -9 -c
 RM = rm -f
 LN = ln -sf
 INSTALL_PROGRAM= install
diff --git a/kpartx/Makefile b/kpartx/Makefile
index 2906a984..9be115d6 100644
--- a/kpartx/Makefile
+++ b/kpartx/Makefile
@@ -21,7 +21,7 @@ all: $(EXEC)
 
 $(EXEC): $(OBJS)
$(CC) $(CFLAGS) $(OBJS) -o $(EXEC) $(LDFLAGS) $(LIBDEPS)
-   $(GZIP) $(EXEC).8 > $(EXEC).8.gz
+   $(GZIP_PROG) $(EXEC).8 > $(EXEC).8.gz
 
 install: $(EXEC) $(EXEC).8
$(INSTALL_PROGRAM) -d $(DESTDIR)$(bindir)
diff --git a/libmpathpersist/Makefile b/libmpathpersist/Makefile
index 1e6399d6..ccdc3223 100644
--- a/libmpathpersist/Makefile
+++ b/libmpathpersist/Makefile
@@ -33,8 +33,8 @@ $(DEVLIB): $(LIBS)
$(LN) $(LIBS) $@
 
 man:
-   $(GZIP) mpath_persistent_reserve_in.3 > mpath_persistent_reserve_in.3.gz
-   $(GZIP) mpath_persistent_reserve_out.3 > 
mpath_persistent_reserve_out.3.gz
+   $(GZIP_PROG) mpath_persistent_reserve_in.3 > 
mpath_persistent_reserve_in.3.gz
+   $(GZIP_PROG) mpath_persistent_reserve_out.3 > 
mpath_persistent_reserve_out.3.gz
 
 install: all
$(INSTALL_PROGRAM) -d $(DESTDIR)$(syslibdir)
diff --git a/mpathpersist/Makefile b/mpathpersist/Makefile
index 51268010..de66b644 100644
--- a/mpathpersist/Makefile
+++ b/mpathpersist/Makefile
@@ -14,7 +14,7 @@ all: $(EXEC)
 
 $(EXEC): $(OBJS)
$(CC) $(OBJS) -o $(EXEC) $(LDFLAGS) $(CFLAGS) $(LIBDEPS)
-   $(GZIP) $(EXEC).8 > $(EXEC).8.gz
+   $(GZIP_PROG) $(EXEC).8 > $(EXEC).8.gz
 
 install:
$(INSTALL_PROGRAM) -d $(DESTDIR)$(bindir)
diff --git a/multipath/Makefile b/multipath/Makefile
index 0828a8f7..3f12d75b 100644
--- a/multipath/Makefile
+++ b/multipath/Makefile
@@ -16,8 +16,8 @@ all: $(EXEC)
 
 $(EXEC): $(OBJS) $(multipathdir)/libmultipath.so $(mpathcmddir)/libmpathcmd.so
$(CC) $(CFLAGS) $(OBJS) -o $(EXEC) $(LDFLAGS) $(LIBDEPS)
-   $(GZIP) $(EXEC).8 > $(EXEC).8.gz
-   $(GZIP) $(EXEC).conf.5 > $(EXEC).conf.5.gz
+   $(GZIP_PROG) $(EXEC).8 > $(EXEC).8.gz
+   $(GZIP_PROG) $(EXEC).conf.5 > $(EXEC).conf.5.gz
 
 install:
$(INSTALL_PROGRAM) -d $(DESTDIR)$(bindir)
diff --git a/multipathd/Makefile b/multipathd/Makefile
index 393b6cbb..64df2214 100644
--- a/multipathd/Makefile
+++ b/multipathd/Makefile
@@ -40,7 +40,7 @@ all : $(EXEC)
 
 $(EXEC): $(OBJS) $(multipathdir)/libmultipath.so $(mpathcmddir)/libmpathcmd.so
$(CC) $(CFLAGS) $(OBJS) $(LDFLAGS) -o $(EXEC) $(LIBDEPS)
-   $(GZIP) $(EXEC).8 > $(EXEC).8.gz
+   $(GZIP_PROG) $(EXEC).8 > $(EXEC).8.gz
 
 cli_handlers.o:cli_handlers.c
$(CC) $(CFLAGS) -Wno-unused-parameter -c -o $@ $<
-- 
2.34.1

--
dm-devel mailing list
dm-devel@redhat.com
https://listman.redhat.com/mailman/listinfo/dm-devel

[dm-devel] [PATCH 3/4] multipath-tools: autodiscover libudev.h headers

[Sergei Trofimovich]

On NixOS nothing is installed in /usr/include and instead lives
in it's own prefix. pkg-config variables are expected to be used
for installation discovery:

$ pkg-config --variable=includedir libudev
/nix/store/27mwkz5zhzw0gip8y7pvjyma5r0hzzaw-systemd-249.7-dev/include

The change switches libudev.h discovery to pkg-config provided path.

CC: Martin Wilck 
CC: Benjamin Marzinski 
Signed-off-by: Sergei Trofimovich 
---
 Makefile.inc  | 8 
 libmultipath/Makefile | 2 +-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/Makefile.inc b/Makefile.inc
index 3b50395f..f5509db4 100644
--- a/Makefile.inc
+++ b/Makefile.inc
@@ -60,6 +60,14 @@ ifndef DEVMAPPER_INCDIR
endif
 endif
 
+ifndef LIBUDEV_INCDIR
+   ifeq ($(shell $(PKGCONFIG) --modversion libudev >/dev/null 2>&1 && echo 
1), 1)
+   LIBUDEV_INCDIR = $(shell $(PKGCONFIG) --variable=includedir 
libudev)
+   else
+   LIBUDEV_INCDIR = /usr/include
+   endif
+endif
+
 prefix =
 exec_prefix= $(prefix)
 usr_prefix = $(prefix)
diff --git a/libmultipath/Makefile b/libmultipath/Makefile
index be48775d..46972faf 100644
--- a/libmultipath/Makefile
+++ b/libmultipath/Makefile
@@ -33,7 +33,7 @@ ifneq ($(call 
check_func,dm_task_set_cookie,$(DEVMAPPER_INCDIR)/libdevmapper.h),
CFLAGS += -DLIBDM_API_COOKIE
 endif
 
-ifneq ($(call 
check_func,udev_monitor_set_receive_buffer_size,/usr/include/libudev.h),0)
+ifneq ($(call 
check_func,udev_monitor_set_receive_buffer_size,$(LIBUDEV_INCDIR)/libudev.h),0)
CFLAGS += -DLIBUDEV_API_RECVBUF
 endif
 
-- 
2.34.1

--
dm-devel mailing list
dm-devel@redhat.com
https://listman.redhat.com/mailman/listinfo/dm-devel