Processed: Re: Bug#993843: [Pkg-zsh-devel] Bug#993843: zsh-static segfaults immediately

[Debian Bug Tracking System]

Processing control commands:

> severity -1 important
Bug #993843 [zsh-static] zsh-static segfaults immediately
Severity set to 'important' from 'serious'

-- 
993843: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993843
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#993843: [Pkg-zsh-devel] Bug#993843: zsh-static segfaults immediately

[Axel Beckert]

Control: severity -1 important

Hi,

Vincent: Thanks for discovering the issue with a statically compiled
binary loading a dynamic library.

Lowering the severity again.

Vincent Lefevre wrote:
> > Please forgive my ignorange in the automated test suites, but perhaps for a
> > future test, would verifying the expected exit status of these commands help
> > intercept this (and any other) weird problem?  :-)
> > 
> >   zsh-static -f -c "exit 0"
> >   zsh-static -f -c "exit 1"
> 
> No, this wouldn't help, because the libc6 version zsh-static has
> been compiled against would be the same as the one for the test.

Of course this will help! The tests are not just run at build time.
The tests are also regularily run against zsh "as installed" via
autopkgtest. See https://ci.debian.net/packages/z/zsh/

> > For comparison, busybox is statically linked:
> > 
> > > strace /bin/busybox -f -c "exit 0" 2>&1 | grep
> > > '^open[a-zA-Z0-9_]*(.*"[^"]*lib.*\.so\>[^"]*"' echo $?
> > 1
> 
> busybox is dynamically linked:

You're both right — depending which busybox-built package you've
isntalled: "busybox" or "busybox-static".

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , https://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Bug#993843: [Pkg-zsh-devel] Bug#993843: zsh-static segfaults immediately

[David]

On Tue, Sep 07, 2021 at 12:23:38PM +0200, Axel Beckert wrote:

...
David wrote:

...


Sorry for the trouble. I (and the tests) admittedly only test the
non-static zsh. And except for some special lookups known to be not
possible with static compiled binaries, I also don't expect any
difference in functionality with zsh-static.


Thank you both so much for looking into it!  And Vincent noticing that 
/lib/x86_64-linux-gnu/libc.so.6 is being opened and mapped into memory 
is very eye opening, for a static binary!


Please forgive my ignorange in the automated test suites, but perhaps 
for a future test, would verifying the expected exit status of these 
commands help intercept this (and any other) weird problem?  :-)


  zsh-static -f -c "exit 0"
  zsh-static -f -c "exit 1"


Now for a more thorough test:  Is it possible (or even desireable?) to 
test the binary to verify it really is statically linked?  Perhaps grep 
strace output looking for potential evidence of .so files getting opened 
in some lib directory?



strace zsh-static -f -c "exit 0" 2>&1 | grep 
'^open[a-zA-Z0-9_]*(.*"[^"]*lib.*\.so\>[^"]*"'

openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libnss_compat.so.2", 
O_RDONLY|O_CLOEXEC) = 11
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 11
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2", 
O_RDONLY|O_CLOEXEC) = 11

echo $?

0

For comparison, busybox is statically linked:

strace /bin/busybox -f -c "exit 0" 2>&1 | grep '^open[a-zA-Z0-9_]*(.*"[^"]*lib.*\.so\>[^"]*"' 
echo $?

1

(And I also just tested a couple small C programs invoking printf and 
compiled with and without "-static".)


There's gotta be a better test to detect run-time opening of shared 
libraries.  (My regex of looking for open...("... .so ...") feels 
hackish.)


- David

Bug#993843: [Pkg-zsh-devel] Bug#993843: zsh-static segfaults immediately

[Vincent Lefevre]

Control: tags -1 upstream

I've found the cause of the use of the shared C library and the crash.
I can notice the following warnings:

gcc -static   -o zsh main.o  `cat stamp-modobjs`   -lgdbm -lpcre -lcap 
-lncursesw -ltinfo -ltinfo -lrt -lm  -lc
/usr/bin/ld: options.o: in function `dosetopt':
./obj-static/Src/../../Src/options.c:830: warning: Using 'initgroups' in 
statically linked applications requires at runtime the shared libraries from 
the glibc version used for linking
/usr/bin/ld: Modules/parameter.o: in function `get_all_groups':
./obj-static/Src/Modules/../../../Src/Modules/parameter.c:2058: warning: Using 
'getgrgid' in statically linked applications requires at runtime the shared 
libraries from the glibc version used for linking
/usr/bin/ld: Modules/files.o: in function `bin_chown':
./obj-static/Src/Modules/../../../Src/Modules/files.c:763: warning: Using 
'getgrnam' in statically linked applications requires at runtime the shared 
libraries from the glibc version used for linking
/usr/bin/ld: params.o: in function `usernamesetfn':
./obj-static/Src/../../Src/params.c:4420: warning: Using 'getpwnam' in 
statically linked applications requires at runtime the shared libraries from 
the glibc version used for linking
/usr/bin/ld: options.o: in function `dosetopt':
./obj-static/Src/../../Src/options.c:822: warning: Using 'getpwuid' in 
statically linked applications requires at runtime the shared libraries from 
the glibc version used for linking

This is exactly the issue we are seeing. Details about why the shared
C library is used with these functions:

  
https://stackoverflow.com/questions/15165306/compile-a-static-binary-which-code-there-a-function-gethostbyname
  https://bugzilla.redhat.com/show_bug.cgi?id=111298

Additional details about the upstream bug in my mail to zsh-workers,
Cc'ed to this bug, i.e.

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993843#46

For the Debian build, I suggest to turn the above warning into
an error (if possible) in order to make sure that the issue is
not left unnoticed (i.e. that the bug does not reappear after
it is fixed, with future changes in zsh).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Processed: Re: Bug#993843: [Pkg-zsh-devel] Bug#993843: zsh-static segfaults immediately

[Debian Bug Tracking System]

Processing control commands:

> tags -1 upstream
Bug #993843 [zsh-static] zsh-static segfaults immediately
Added tag(s) upstream.

-- 
993843: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993843
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#993843: [Pkg-zsh-devel] Bug#993843: zsh-static segfaults immediately

[Vincent Lefevre]

On 2021-09-07 22:22:34 +, David wrote:
> On Tue, Sep 07, 2021 at 12:23:38PM +0200, Axel Beckert wrote:
> > ...
> > David wrote:
> > > ...
> > 
> > Sorry for the trouble. I (and the tests) admittedly only test the
> > non-static zsh. And except for some special lookups known to be not
> > possible with static compiled binaries, I also don't expect any
> > difference in functionality with zsh-static.
> 
> Thank you both so much for looking into it!  And Vincent noticing that
> /lib/x86_64-linux-gnu/libc.so.6 is being opened and mapped into memory is
> very eye opening, for a static binary!
> 
> Please forgive my ignorange in the automated test suites, but perhaps for a
> future test, would verifying the expected exit status of these commands help
> intercept this (and any other) weird problem?  :-)
> 
>   zsh-static -f -c "exit 0"
>   zsh-static -f -c "exit 1"

No, this wouldn't help, because the libc6 version zsh-static has
been compiled against would be the same as the one for the test.

Here, the issue is that zsh-static is built against some libc6
version (at this time, zsh-static would work as expected), then
libc6 is upgraded to some newer version, and zsh-static no longer
works.

I'd say that it would be difficult to design a test framework for
up-to-date Debian/unstable that would be useful in practice (since
as soon as a new version of some package gets in unstable, all the
other packages would need to be tested), except for non-regression
tests about particular bugs like this one.

> Now for a more thorough test:  Is it possible (or even desireable?) to test
> the binary to verify it really is statically linked?  Perhaps grep strace
> output looking for potential evidence of .so files getting opened in some
> lib directory?
> 
> > strace zsh-static -f -c "exit 0" 2>&1 | grep 
> > '^open[a-zA-Z0-9_]*(.*"[^"]*lib.*\.so\>[^"]*"'
> openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libnss_compat.so.2", 
> O_RDONLY|O_CLOEXEC) = 11
> openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 11
> openat(AT_FDCWD, "/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2", 
> O_RDONLY|O_CLOEXEC) = 11
> > echo $?
> 0

Perhaps.

> For comparison, busybox is statically linked:
> 
> > strace /bin/busybox -f -c "exit 0" 2>&1 | grep
> > '^open[a-zA-Z0-9_]*(.*"[^"]*lib.*\.so\>[^"]*"' echo $?
> 1

busybox is dynamically linked:

$ file /bin/busybox
/bin/busybox: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), 
dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, 
BuildID[sha1]=280a5ddfb6379abacc1a6c7a6024876f653cd6bf, for GNU/Linux 3.2.0, 
stripped

> (And I also just tested a couple small C programs invoking printf and
> compiled with and without "-static".)

I confirm that no shared libraries are read with -static on simple
programs.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#993843: [Pkg-zsh-devel] Bug#993843: zsh-static segfaults immediately

[Vincent Lefevre]

On 2021-09-07 13:31:36 +0200, Vincent Lefevre wrote:
[...]
> zsh-static is linked statically. What does it have to do with libc.so.6?
> If it expects some functions from libc.so.6 and others that had been
> linked statically, then with the upgrade of libc6 to 2.32-1, I think
> that this is likely to break.

Indeed, a rebuild solves the issue. My patch at

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993861#15

may be needed due to a FTBFS if libncursesw5-dev is not installed
(now provided by libncurses-dev).

I suppose that the libc6 upgrade was the reason, though I didn't
do a test to check.

A rebuild doesn't really solve the bug, since at the next libc6
upgrade, a similar issue may occur. The use of libc.so.6 should
be removed completely. Now, I don't know whether the libc.so.6
is due to the build rules, binutils, or something else.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#993843: [Pkg-zsh-devel] Bug#993843: zsh-static segfaults immediately

[Vincent Lefevre]

On 2021-09-07 12:23:38 +0200, Axel Beckert wrote:
> A first strace ends as follows:
> 
> mprotect(0x7f75ad335000, 4096, PROT_READ) = 0
> munmap(0x7f75ad337000, 327870)  = 0
> mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
> 0x7f75ad03a000
> openat(AT_FDCWD, "/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 11
> fstat(11, {st_mode=S_IFREG|0644, st_size=526, ...}) = 0
> read(11, "# /etc/nsswitch.conf\n#\n# Example"..., 4096) = 526
> --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0xe0} ---
> +++ killed by SIGSEGV (core dumped) +++

A more interesting part of the strace output:

[...]
223550 openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) 
= 11
223550 read(11, 
"\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\177\2\0\0\0\0\0"..., 832) = 
832
223550 fstat(11, {st_mode=S_IFREG|0755, st_size=1839168, ...}) = 0
223550 mmap(NULL, 1852480, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 11, 0) = 
0x7f8847303000
223550 mprotect(0x7f8847329000, 1658880, PROT_NONE) = 0
223550 mmap(0x7f8847329000, 1347584, PROT_READ|PROT_EXEC, 
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 11, 0x26000) = 0x7f8847329000
223550 mmap(0x7f8847472000, 307200, PROT_READ, 
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 11, 0x16f000) = 0x7f8847472000
223550 mmap(0x7f88474be000, 24576, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 11, 0x1ba000) = 0x7f88474be000
223550 mmap(0x7f88474c4000, 13376, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f88474c4000
223550 close(11)= 0
[...]

zsh-static is linked statically. What does it have to do with libc.so.6?
If it expects some functions from libc.so.6 and others that had been
linked statically, then with the upgrade of libc6 to 2.32-1, I think
that this is likely to break.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)