Re: [PATCH] Serious glibc sparc rlimits bug

2011-08-14 Thread Jurij Smakov
On Sun, Aug 14, 2011 at 12:29:18AM -0700, David Miller wrote:
> 
> Unfortunately, the sparc32 definition of the 64-bit RLIM_INFINITY
> value is wrong, and it's been wrong for a while.
> 
> This causes serious problems, the worst of which is that no 64-bit
> pthread program executing as a child of 'make' can succeed.
> 
> GLIBC's pthread_create() uses the current RLIMIT_STACK value as a hint
> for sizing thread stacks.  It handles RLIM_INFINITY specially instead
> of trying to allocate an enormous stack.
> 
> But if RLIM_INFINITY is wrong... right, nothing works.
> 
> How this happens is that libpam corrupts the kernel default rlimits
> when it creates a login session, because even if you have no explicit
> settings in /etc/security/limits.conf it still reads every rlimit
> then sets it right back to the same value.  It even has some pre-cooked
> defaults, and the default for RLIMIT_STACK is cur=8MB max=RLIM_INFINITY
> 
> So if RLIM_INFINITY is wrong (for 32-bit binaries it's too small), the
> values will all get truncated down to this incorrect RLIM_INFINITY
> value.
> 
> Next, make sets the current RLIMIT_STACK 'cur' value to the maximum,
> so now the current RLIMIT_STACK has this wrong RLIM_INFINITY value
> too.
> 
> Then 64-bit glibc doesn't recognize this value as RLIM_INFINITY during
> pthread_create() and it tries to allocate a thread stack of size
> 0x8000 bytes.  This, of course, fails.
> 
> A bunch of binaries are going to need to be rebuilt because of this issue
> once the glibc build goes through, I would prioritize libpam0g, make,
> and emacs23.
> 
> I'll be committing this soon to glibc GIT.  I tested this by doing a
> glibc deb rebuild with this patch applied, installing, then rebuilding
> libpam0g and make.

Thanks for the information and the fix. I've opened a bug to keep
track of the issue:

http://bugs.debian.org/637767

If you have any references to upstream bug or commit(s), please add them
to it.

> 
> 2011-08-14  David S. Miller  
> 
>   * sysdeps/unix/sysv/linux/sparc/bits/resource.h (RLIM_INFINITY,
>   RLIM64_INFINITY): Fix 64-bit values for 32-bit sparc.
> 
> diff --git a/sysdeps/unix/sysv/linux/sparc/bits/resource.h 
> b/sysdeps/unix/sysv/linux/sparc/bits/resource.h
> index 04d33e4..5c00b8f 100644
> --- a/sysdeps/unix/sysv/linux/sparc/bits/resource.h
> +++ b/sysdeps/unix/sysv/linux/sparc/bits/resource.h
> @@ -130,11 +130,11 @@ enum __rlimit_resource
>  #ifndef __USE_FILE_OFFSET64
>  # define RLIM_INFINITY ((long int)(~0UL >> 1))
>  #else
> -# define RLIM_INFINITY 0x7fffLL
> +# define RLIM_INFINITY 0xLL
>  #endif
>  
>  #ifdef __USE_LARGEFILE64
> -# define RLIM64_INFINITY 0x7fffLL
> +# define RLIM64_INFINITY 0xLL
>  #endif
>  
>  #endif

-- 
Jurij Smakov   ju...@wooyd.org
Key: http://www.wooyd.org/pgpkey/  KeyID: C99E03CC


-- 
To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110814090617.ga3...@droopy.oc.cox.net



[PATCH] Serious glibc sparc rlimits bug

2011-08-14 Thread David Miller

Unfortunately, the sparc32 definition of the 64-bit RLIM_INFINITY
value is wrong, and it's been wrong for a while.

This causes serious problems, the worst of which is that no 64-bit
pthread program executing as a child of 'make' can succeed.

GLIBC's pthread_create() uses the current RLIMIT_STACK value as a hint
for sizing thread stacks.  It handles RLIM_INFINITY specially instead
of trying to allocate an enormous stack.

But if RLIM_INFINITY is wrong... right, nothing works.

How this happens is that libpam corrupts the kernel default rlimits
when it creates a login session, because even if you have no explicit
settings in /etc/security/limits.conf it still reads every rlimit
then sets it right back to the same value.  It even has some pre-cooked
defaults, and the default for RLIMIT_STACK is cur=8MB max=RLIM_INFINITY

So if RLIM_INFINITY is wrong (for 32-bit binaries it's too small), the
values will all get truncated down to this incorrect RLIM_INFINITY
value.

Next, make sets the current RLIMIT_STACK 'cur' value to the maximum,
so now the current RLIMIT_STACK has this wrong RLIM_INFINITY value
too.

Then 64-bit glibc doesn't recognize this value as RLIM_INFINITY during
pthread_create() and it tries to allocate a thread stack of size
0x8000 bytes.  This, of course, fails.

A bunch of binaries are going to need to be rebuilt because of this issue
once the glibc build goes through, I would prioritize libpam0g, make,
and emacs23.

I'll be committing this soon to glibc GIT.  I tested this by doing a
glibc deb rebuild with this patch applied, installing, then rebuilding
libpam0g and make.

2011-08-14  David S. Miller  

* sysdeps/unix/sysv/linux/sparc/bits/resource.h (RLIM_INFINITY,
RLIM64_INFINITY): Fix 64-bit values for 32-bit sparc.

diff --git a/sysdeps/unix/sysv/linux/sparc/bits/resource.h 
b/sysdeps/unix/sysv/linux/sparc/bits/resource.h
index 04d33e4..5c00b8f 100644
--- a/sysdeps/unix/sysv/linux/sparc/bits/resource.h
+++ b/sysdeps/unix/sysv/linux/sparc/bits/resource.h
@@ -130,11 +130,11 @@ enum __rlimit_resource
 #ifndef __USE_FILE_OFFSET64
 # define RLIM_INFINITY ((long int)(~0UL >> 1))
 #else
-# define RLIM_INFINITY 0x7fffLL
+# define RLIM_INFINITY 0xLL
 #endif
 
 #ifdef __USE_LARGEFILE64
-# define RLIM64_INFINITY 0x7fffLL
+# define RLIM64_INFINITY 0xLL
 #endif
 
 #endif


-- 
To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20110814.002918.773510966891265864.da...@davemloft.net