Bug#633458: gcc-4.6 miscompiles libgcrypt11 on armel

2011-07-27 Thread Andreas Metzler
On 2011-07-14 Matthias Klose  wrote:
> On 07/14/2011 08:17 PM, Andreas Metzler wrote:
> > (sid)ametzler@abel:~/GNUTLS/GCR/libgcrypt11-1.4.6$ 
> > ~/GNUTLS/2.10.5-2/usr/bin/gnutls-cli db.debian.org
> > Resolving 'db.debian.org'...
> > Connecting to '82.195.75.106:443'...
> > *** Fatal error: Decryption has failed.
> > *** Handshake has failed
> > GnuTLS error: Decryption has failed.
> > ---
> > 
> > ~/GNUTLS/2.10.5-2/ is not a local build, I just extracted (dpkg -x) the
> > official deb since abel does not have gnutls-bin installed.

> strange. this does work for me on a local machine (Ubuntu 11.04 kernel, armv7
> processor, Genesi Efika MX), sid chroot

I have no idea why that is the case. It is 100% reproducible on abel.

Anyway, I have just uploaded libgcrypt11 1.4.6-8 including the
attached workaround.

cu andreas
-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'
Description: Disable inlining on do_decrypt_aligned() on ARM. This works
 around gcc issue #633458, which had broken gnutls on ARM.
Author: Andreas Metzler 
Bug-Debian: http://bugs.debian.org/633458

--- libgcrypt11-1.4.6.orig/cipher/rijndael.c
+++ libgcrypt11-1.4.6/cipher/rijndael.c
@@ -569,6 +569,9 @@ _gcry_aes_cbc_enc (void *context, unsign
 /* Decrypt one block.  A and B need to be aligned on a 4 byte boundary
and the decryption must have been prepared.  A and B may be the
same. */
+#ifdef ARMWORKAROUND633458
+__attribute__((noinline))
+#endif
 static void
 do_decrypt_aligned (RIJNDAEL_context *ctx, 
 unsigned char *b, const unsigned char *a)


Bug#633458: gcc-4.6 miscompiles libgcrypt11 on armel

2011-07-15 Thread Andreas Metzler
On 2011-07-14 Matthias Klose  wrote:
> On 07/12/2011 07:56 PM, Andreas Metzler wrote:
> > On 2011-07-10 Andreas Metzler  wrote:
[...]
> >>> Building with -fno-strict-aliasing does not help.
[...]

> maybe try not to inline, or turn off an optimization for this
> specific function(s).  Then the assembler output of a working and a
> non-working function would help.

> __attribute__((noinline))
> __attribute__((optimize("O0")))
> __attribute__((optimize("no-inline")))
> __attribute__((optimize("no-strict-aliasing")))

> see
> http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Function-Attributes.html#Function-Attributes

Hello,

[helps]Building rijndael.c with -O0
[nochange] Building rijndael.c with -O1
[nochange] Building rijndael.c (or all gcrypt) with
   -fno-strict-aliasing
[helps]Building rijndael.c with -fno-inline
[helps]Setting __attribute__((noinline)) on do_decrypt_aligned [1]
[nochange] Setting __attribute__((noinline)) on do_decrypt
[helps]Setting __attribute__((optimize("O0"))) on do_decrypt_aligned
[helps]Setting __attribute__((optimize("O0"))) on do_decrypt
[helps]Setting __attribute__((optimize("no-inline"))) on do_decrypt_aligned
[helps]Setting __attribute__((optimize("no-inline"))) on do_decrypt

[...]

For which of these would like to see further info, and how exactly
should I generate it?

thanks, cu andreas

[1] Example diff:
-
--- ../rijndael.c.orig  2011-07-14 17:56:54.0 +
+++ cipher/rijndael.c   2011-07-15 17:54:46.0 +
@@ -569,7 +569,7 @@ _gcry_aes_cbc_enc (void *context, unsign
 /* Decrypt one block.  A and B need to be aligned on a 4 byte boundary
and the decryption must have been prepared.  A and B may be the
same. */
-static void
+__attribute__((noinline)) static void
 do_decrypt_aligned (RIJNDAEL_context *ctx,
 unsigned char *b, const unsigned char *a)
 {



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#633458: gcc-4.6 miscompiles libgcrypt11 on armel

2011-07-14 Thread Matthias Klose
On 07/14/2011 08:17 PM, Andreas Metzler wrote:
> (sid)ametzler@abel:~/GNUTLS/GCR/libgcrypt11-1.4.6$ 
> ~/GNUTLS/2.10.5-2/usr/bin/gnutls-cli db.debian.org
> Resolving 'db.debian.org'...
> Connecting to '82.195.75.106:443'...
> *** Fatal error: Decryption has failed.
> *** Handshake has failed
> GnuTLS error: Decryption has failed.
> ---
> 
> ~/GNUTLS/2.10.5-2/ is not a local build, I just extracted (dpkg -x) the
> official deb since abel does not have gnutls-bin installed.

strange. this does work for me on a local machine (Ubuntu 11.04 kernel, armv7
processor, Genesi Efika MX), sid chroot



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#633458: gcc-4.6 miscompiles libgcrypt11 on armel

2011-07-14 Thread Andreas Metzler
On 2011-07-14 Matthias Klose  wrote:
> On 07/12/2011 07:56 PM, Andreas Metzler wrote:
[...]
> >> The culprit is cipher/rijndael.c. If I build this file with
> >> gcc-4.5 the bug goes away.

> > I do not know how to go on now.

> could you recheck with 4.6.1-3 and gcc-snapshot?
> is this exposed on armhf as well (CC'ing Markos)?

No difference with either 4.6.1-3 or gcc-snapshot 20110709-1.
(Removing the two "static" still fixes the testcase.)

[ deleted some stuff I will try later this week ]

> the binary in the archive seems to work. gcc-4.6.0-13 was used for
> the build.
[...]

Actually, it does not:
---
+++-==-==-
ii  gcc-4.64.6.1-3GNU C compiler
ii  libgcrypt111.4.6-7LGPL Crypto library - runtime library
ii  libgnutls262.10.5-2   the GNU TLS library - runtime library
(sid)ametzler@abel:~/GNUTLS/GCR/libgcrypt11-1.4.6$ 
~/GNUTLS/2.10.5-2/usr/bin/gnutls-cli db.debian.org
Resolving 'db.debian.org'...
Connecting to '82.195.75.106:443'...
*** Fatal error: Decryption has failed.
*** Handshake has failed
GnuTLS error: Decryption has failed.
---

~/GNUTLS/2.10.5-2/ is not a local build, I just extracted (dpkg -x) the
official deb since abel does not have gnutls-bin installed.

cu andreas

-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#633458: gcc-4.6 miscompiles libgcrypt11 on armel

2011-07-14 Thread Matthias Klose
On 07/12/2011 07:56 PM, Andreas Metzler wrote:
> On 2011-07-10 Andreas Metzler  wrote:
>> On 2011-07-10 Andreas Metzler  wrote:
>>> On 2011-07-10 Andreas Metzler  wrote:
 Package: gcc-4.6
 Version: 4.6.1-1
 Severity: normal
 Blocks: 633373
> 
 gcc-4.6 seems to miscompile libgcrypt11 on armel, causing total
 failure of gnutls26.
> 
 ---
 (sid)ametzler@abel:~/GNUTLS$ ~/GNUTLS/2.10.5-2/usr/bin/gnutls-cli 
 db.debian.org Resolving 'db.debian.org'...
 Connecting to '82.195.75.106:443'...
 *** Fatal error: Decryption has failed.
 *** Handshake has failed
 GnuTLS error: Decryption has failed.
 ---
>>> [...]
> 
>>> Building with -fno-strict-aliasing does not help.
>> [...]
> 
>> The culprit is cipher/rijndael.c. If I build this file with
>> gcc-4.5 the bug goes away.
> 
> Hello,
> 
> I do not know how to go on now.

could you recheck with 4.6.1-3 and gcc-snapshot?
is this exposed on armhf as well (CC'ing Markos)?

> What I tried was splitting rijndael.c into multiple separate files.
> However this made the error go away.
> 
> It seems that the significant change I made when spliting was
> dropping the static qualifier on the function declarations.

maybe try not to inline, or turn off an optimization for this specific
function(s).  Then the assembler output of a working and a non-working function
would help.

__attribute__((noinline))
__attribute__((optimize("O0")))
__attribute__((optimize("no-inline")))
__attribute__((optimize("no-strict-aliasing")))

see
http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Function-Attributes.html#Function-Attributes

> 
> The minimal change I found to make my testcase (gnutls-cli
> db.debian.org) go away was dropping the static qualifier from these
> two functions:
> 
> 
> /* Decrypt one block.  A and B need to be aligned on a 4 byte boundary
>and the decryption must have been prepared.  A and B may be the
>same. */
> static void
> do_decrypt_aligned (RIJNDAEL_context *ctx, 
> unsigned char *b, const unsigned char *a)
> 
> 
> /* Decrypt one block.  AX and BX may be the same. */
> static void
> do_decrypt (RIJNDAEL_context *ctx, byte *bx, const byte *ax)
> 

the binary in the archive seems to work. gcc-4.6.0-13 was used for the build.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#633458: gcc-4.6 miscompiles libgcrypt11 on armel

2011-07-12 Thread Andreas Metzler
On 2011-07-10 Andreas Metzler  wrote:
> On 2011-07-10 Andreas Metzler  wrote:
> > On 2011-07-10 Andreas Metzler  wrote:
> > > Package: gcc-4.6
> > > Version: 4.6.1-1
> > > Severity: normal
> > > Blocks: 633373

> > > gcc-4.6 seems to miscompile libgcrypt11 on armel, causing total
> > > failure of gnutls26.

> > > ---
> > > (sid)ametzler@abel:~/GNUTLS$ ~/GNUTLS/2.10.5-2/usr/bin/gnutls-cli 
> > > db.debian.org Resolving 'db.debian.org'...
> > > Connecting to '82.195.75.106:443'...
> > > *** Fatal error: Decryption has failed.
> > > *** Handshake has failed
> > > GnuTLS error: Decryption has failed.
> > > ---
> > [...]

> > Building with -fno-strict-aliasing does not help.
> [...]

> The culprit is cipher/rijndael.c. If I build this file with
> gcc-4.5 the bug goes away.

Hello,

I do not know how to go on now.

What I tried was splitting rijndael.c into multiple separate files.
However this made the error go away.

It seems that the significant change I made when spliting was
dropping the static qualifier on the function declarations.

The minimal change I found to make my testcase (gnutls-cli
db.debian.org) go away was dropping the static qualifier from these
two functions:


/* Decrypt one block.  A and B need to be aligned on a 4 byte boundary
   and the decryption must have been prepared.  A and B may be the
   same. */
static void
do_decrypt_aligned (RIJNDAEL_context *ctx, 
unsigned char *b, const unsigned char *a)


/* Decrypt one block.  AX and BX may be the same. */
static void
do_decrypt (RIJNDAEL_context *ctx, byte *bx, const byte *ax)


cu andreas



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#633458: gcc-4.6 miscompiles libgcrypt11 on armel

2011-07-10 Thread Andreas Metzler
On 2011-07-10 Andreas Metzler  wrote:
> On 2011-07-10 Andreas Metzler  wrote:
> > Package: gcc-4.6
> > Version: 4.6.1-1
> > Severity: normal
> > Blocks: 633373

> > gcc-4.6 seems to miscompile libgcrypt11 on armel, causing total
> > failure of gnutls26.

> > ---
> > (sid)ametzler@abel:~/GNUTLS$ ~/GNUTLS/2.10.5-2/usr/bin/gnutls-cli 
> > db.debian.org Resolving 'db.debian.org'...
> > Connecting to '82.195.75.106:443'...
> > *** Fatal error: Decryption has failed.
> > *** Handshake has failed
> > GnuTLS error: Decryption has failed.
> > ---
> [...]

> Building with -fno-strict-aliasing does not help.
[...]

The culprit is cipher/rijndael.c. If I build this file with
gcc-4.5 the bug goes away.

cu andreas



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#633458: gcc-4.6 miscompiles libgcrypt11 on armel

2011-07-10 Thread Andreas Metzler
On 2011-07-10 Andreas Metzler  wrote:
> Package: gcc-4.6
> Version: 4.6.1-1
> Severity: normal
> Blocks: 633373

> gcc-4.6 seems to miscompile libgcrypt11 on armel, causing total
> failure of gnutls26.

> ---
> (sid)ametzler@abel:~/GNUTLS$ ~/GNUTLS/2.10.5-2/usr/bin/gnutls-cli 
> db.debian.org Resolving 'db.debian.org'...
> Connecting to '82.195.75.106:443'...
> *** Fatal error: Decryption has failed.
> *** Handshake has failed
> GnuTLS error: Decryption has failed.
> ---
[...]

Building with -fno-strict-aliasing does not help. running gnutls-cli
with debugging provides a little bit more info on the code triggering
the issue:

[...]
|<7>| RB: Requested 53 bytes
|<2>| ASSERT: gnutls_cipher.c:530
|<4>| REC[0x2e338]: Short record length 166 > 48 - 20 (under attack?)
[...]

cu andreas



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#633458: gcc-4.6 miscompiles libgcrypt11 on armel

2011-07-10 Thread Andreas Metzler
Package: gcc-4.6
Version: 4.6.1-1
Severity: normal
Blocks: 633373

gcc-4.6 seems to miscompile libgcrypt11 on armel, causing total
failure of gnutls26.

---
(sid)ametzler@abel:~/GNUTLS$ ~/GNUTLS/2.10.5-2/usr/bin/gnutls-cli db.debian.org 
Resolving 'db.debian.org'...
Connecting to '82.195.75.106:443'...
*** Fatal error: Decryption has failed.
*** Handshake has failed
GnuTLS error: Decryption has failed.
---

libgcrypt11 1.4.6-5 (no source changes compared to 1.4.6-7, just a
switch to multiarch directories) works:
---
(sid)ametzler@abel:~/GNUTLS$ LD_LIBRARY_PATH=~/GNUTLS/gcr-1.4.6-5/lib  ~/GNUTLS/
2.10.5-2/usr/bin/gnutls-cli db.debian.org
Resolving 'db.debian.org'...
Connecting to '82.195.75.106:443'...
- Ephemeral Diffie-Hellman parameters
[...]
- Handshake was completed
---

I have also rebuilt libgcrypt11 with gcc-4.5, this also fixed the issue:
---
 LD_LIBRARY_PATH=/home/ametzler/GNUTLS/GCR/libgcrypt11-1.4.6+gcc45/src/.libs  
~/GNUTLS/2.10.5-2/usr/bin/gnutls-cli db.debian.org
Resolving 'db.debian.org'...
Connecting to '82.195.75.106:443'...
- Ephemeral Diffie-Hellman parameters
[...]
- Handshake was completed
---

All tests done in the sid chroot of the Debian porterbox (abel).

cu andreas

ii  gcc4:4.6.0-6  GNU C compiler
ii  gcc-4.54.5.3-3The GNU C compiler
ii  gcc-4.64.6.1-1GNU C compiler
ii  libgcrypt111.4.6-7LGPL Crypto library - runtime library
ii  libgnutls262.10.5-2   the GNU TLS library - runtime library
ii  libtasn1-3 2.9-4  Manage ASN.1 structures (runtime)



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org