Bug#796899: Acknowledgement (interesting segfault)

[Joey Hess]

By comparing stack traces under ld-linux.so and not, I was able to determine
that the NULL is coming from global->errors, which is supposed to get
initialized to STDERR but somehow isn't when ld-linux.so runs curl.

While playing with that, I noticed that trying to printf the address of global
causes a segfault, too. Here's a minimal test case for that, which 
replaces src/tool_main.c in curl's source tree.

#include 
#include 

int main(int argc, char *argv[])
{
  FILE *global=0;

  printf("STARTED\n");
  printf("GLOBAL %p\n", global);
}

joey@kite:~/tmp/curl-7.44.0/debian/build>./src/.libs/curl
STARTED
GLOBAL (nil)
joey@kite:~/tmp/curl-7.44.0/debian/build>/lib64/ld-linux-x86-64.so.2 
./src/.libs/curl
STARTED
Segmentation fault

(Building this same code manually, not in curl's source tree, I have not been
able to reproduce the problem. Something about how it's linked as part of 
curl is contributing.)

Here's an even more minimal and strange test case!

joey@kite:~/tmp/curl-7.44.0/debian/build>cat src/tool_main.c 
#include 
#include 

int main(int argc, char *argv[])
{
  fprintf(stdout, "HELLO\n"); 
}
joey@kite:~/tmp/curl-7.44.0/debian/build>./src/.libs/curl
HELLO
joey@kite:~/tmp/curl-7.44.0/debian/build>/lib64/ld-linux-x86-64.so.2 
./src/.libs/curl
Segmentation fault

-- 
see shy jo


signature.asc
Description: Digital signature

Bug#796899: Acknowledgement (interesting segfault)

[Joey Hess]

> int main(int argc, char *argv[])
> {
>   fprintf(stdout, "HELLO\n"); 
> }

Even fdopen(1, 'w') crashes the same way. Maybe whatever initialization
is needed for the stream functions to work isn't happening under
ld-linux.so. 

-- 
see shy jo


signature.asc
Description: Digital signature

Bug#796899: Acknowledgement (interesting segfault)

[Joey Hess]

Joey Hess wrote:
> Even fdopen(1, 'w') crashes the same way.

Er, ignore that, it's obviously wrong.

But, stdout stderr etc are indeed looking very wrong..

joey@kite:~/tmp/curl-7.44.0/debian/build>cat src/tool_main.c 
#include 
#include 

int main(int argc, char *argv[])
{
fprintf(stdout, "hello\n");
}
joey@kite:~/tmp/curl-7.44.0/debian/build>gdb /lib64/ld-linux-x86-64.so.2
(gdb) b __GI__IO_fwrite
Breakpoint 1 at 0x773d0a30: file iofwrite.c, line 35.
(gdb) r ./src/.libs/curl
Starting program: /lib64/ld-linux-x86-64.so.2 ./src/.libs/curl
[Thread debugging using libthread_db enabled]
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, __GI__IO_fwrite (buf=0x77df362b, size=1, count=6, 
fp=0x0) at iofwrite.c:35
   ^^^ stdout

-- 
see shy jo


signature.asc
Description: Digital signature

Bug#796899: Acknowledgement (interesting segfault)

[Joey Hess]

Joey Hess wrote:
> Tried building curl from source to get a useful backtrace, but that
> build didn't have the problem. 
> 
> Since that build was done using gcc 4.9.2-4, it may be another hint in
> the direction of the recent gcc transitions.

Indeed, I built curl with gcc 5.2.1-4 and it has the problem.
So, the gcc upgrade led to this problem.

Here is the backtrace:

joey@kite:~/tmp/curl-7.44.0/debian>LD_LIBRARY_PATH=./build/lib/.libs/ gdb 
/lib64/ld-linux-x86-64.so.2 
GNU gdb (Debian 7.10-1) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from 
/usr/lib/debug//lib/x86_64-linux-gnu/ld-2.19.so...done.
done.
(gdb) run ./build/src/.libs/curl
Starting program: /lib64/ld-linux-x86-64.so.2 ./build/src/.libs/curl
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
fputc (c=99, fp=0x0) at fputc.c:37
37  fputc.c: No such file or directory.
(gdb) bt
#0  fputc (c=99, fp=0x0) at fputc.c:37
#1  0x77b7cd9f in dprintf_formatf (data=, 
stream=0x773d7470 , format=, 
ap_save=ap_save@entry=0x7fffe060) at mprintf.c:616
#2  0x77b7e58e in curl_mfprintf (whereto=, 
format=) at mprintf.c:1121
#3  0x77ddae51 in ?? ()
#4  0x in ?? ()

The code here is quite horrible, kind of looks like the FILE * has
been somehow optimised out, perhaps wrongly?

static int dprintf_formatf(
  void *data, /* untouched by format(), just sent to the stream() function in
 the second argument */
  /* function pointer called for each output character */
  int (*stream)(int, FILE *),

OUTCHAR(*f);

#define OUTCHAR(x) \
  do{ \
if(stream((unsigned char)(x), (FILE *)data) != -1) \
  done++; \
else \
 return done; /* return immediately on failure */ \
  } WHILE_FALSE

-- 
see shy jo


signature.asc
Description: Digital signature

Bug#796899: Acknowledgement (interesting segfault)

[Joey Hess]

Aurelien Jarno wrote:
 The fp pointer is NULL in both of the above functions. Could you please
 try to get a backtrace to see which caller starts to pass a NULL
 pointer?

Tried building curl from source to get a useful backtrace, but that
build didn't have the problem. 

Since that build was done using gcc 4.9.2-4, it may be another hint in
the direction of the recent gcc transitions.

-- 
see shy jo


signature.asc
Description: Digital signature

Bug#796899: Acknowledgement (interesting segfault)

[Joey Hess]

Colin Watson wrote:
 Here's LD_DEBUG=all output, which suggests it might relate to NSS.

  22014:   symbol=fclose;  lookup in file=/lib/x86_64-linux-gnu/libc.so.6 
 [0]
  22014:   binding file /lib/x86_64-linux-gnu/libnss_compat.so.2 [0] to 
 /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `fclose' [GLIBC_2.2.5]

strace shows curl gets as far as reading ~/.curlrc before crashing, while
ssh seems to start running and reads /etc/passwd before crashing.

gdb shows ssh and curl crashing in fwrite and fputc, respectively.

Starting program: /lib64/ld-linux-x86-64.so.2 /usr/bin/ssh
[Thread debugging using libthread_db enabled]
Using host libthread_db library /lib/x86_64-linux-gnu/libthread_db.so.1.

Program received signal SIGSEGV, Segmentation fault.
__GI__IO_fwrite (buf=0x77db4a00, size=1, count=525, fp=0x0)
at iofwrite.c:41
41  iofwrite.c: No such file or directory.

Starting program: /lib64/ld-linux-x86-64.so.2 /usr/bin/curl
[Thread debugging using libthread_db enabled]
Using host libthread_db library /lib/x86_64-linux-gnu/libthread_db.so.1.

Program received signal SIGSEGV, Segmentation fault.
fputc (c=99, fp=0x0) at fputc.c:37
37  fputc.c: No such file or directory.

-- 
see shy jo


signature.asc
Description: Digital signature

Bug#796899: Acknowledgement (interesting segfault)

[Joey Hess]

reassign 796899 libc6
found 796899 2.19-19
thanks

This also happens with curl, not just ssh, so reassigning to libc6.

/lib64/ld-linux-x86-64.so.2 /usr/bin/curl
Segmentation fault

Since curl 7.44.0-1 works when run via ld.so, and curl 7.43.0-1
segfaults, I think this might have to do with the ongoing gcc
transition.

-- 
see shy jo

Bug#796899: Acknowledgement (interesting segfault)

[Aurelien Jarno]

On 2015-08-25 10:58, Joey Hess wrote:
 Colin Watson wrote:
  Here's LD_DEBUG=all output, which suggests it might relate to NSS.
 
   22014: symbol=fclose;  lookup in file=/lib/x86_64-linux-gnu/libc.so.6 
  [0]
   22014: binding file /lib/x86_64-linux-gnu/libnss_compat.so.2 [0] to 
  /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `fclose' [GLIBC_2.2.5]
 
 strace shows curl gets as far as reading ~/.curlrc before crashing, while
 ssh seems to start running and reads /etc/passwd before crashing.
 
 gdb shows ssh and curl crashing in fwrite and fputc, respectively.
 
 Starting program: /lib64/ld-linux-x86-64.so.2 /usr/bin/ssh
 [Thread debugging using libthread_db enabled]
 Using host libthread_db library /lib/x86_64-linux-gnu/libthread_db.so.1.
 
 Program received signal SIGSEGV, Segmentation fault.
 __GI__IO_fwrite (buf=0x77db4a00, size=1, count=525, fp=0x0)
 at iofwrite.c:41
 41iofwrite.c: No such file or directory.
 
 Starting program: /lib64/ld-linux-x86-64.so.2 /usr/bin/curl
 [Thread debugging using libthread_db enabled]
 Using host libthread_db library /lib/x86_64-linux-gnu/libthread_db.so.1.
 
 Program received signal SIGSEGV, Segmentation fault.
 fputc (c=99, fp=0x0) at fputc.c:37
 37fputc.c: No such file or directory.

The fp pointer is NULL in both of the above functions. Could you please
try to get a backtrace to see which caller starts to pass a NULL
pointer?

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net


signature.asc
Description: Digital signature