[openssl.org #2778] [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0
Hi, I'm trying to run a simple DTLS client/server test using OpenSSL 1.0.1, but with the same Cipher Suites that OpenSSL 1.0.0 uses, (to compare the two handshakes). This works fine with a 32 bit, (i686), build, but fails on 64 bit, (x86_64) with the following error: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 If I do *not* override the default Cipher Suites, the 64 bit test works. I've attached packet captures from the various tests: dtls-assert-fail-x86_64.pcap -- failing handshake, (64 bit OpenSSL 1.0.1, using 1.0.0 Ciphers) dtls-assert-ok-i686.pcap -- working handshake, (32 bit OpenSSL 1.0.1, using 1.0.0 Ciphers) dtls-assert-ok-x86_64.pcap -- working handshake, (64 bit OpenSSL 1.0.1, using 1.0.1 Ciphers) Looking at the working/failing x86_64 pcaps, the MD differs for the chosen suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (works) vs. TLS_RSA_WITH_AES_256_CBC_SHA (fails) My OpenSSL 1.0.1 code is built from the original sources, and linked directly into a test harness, with patches/overrides for the following: 1) The random number generator is controlled by the test harness 2) Time-of-day is controlled by the test harness 3) Memory allocation/freeing is handled by the test harness The 64 bit code is built on Fedora 16 using: gcc version 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) I'd be interested to hear if other people experience the same problem with OpenSSL 1.0.1 x86_64 DTLS using TLS_RSA_WITH_AES_256_CBC_SHA, (or am I on my own here). Thanks, John Fitzgibbon Hi,I'm trying to run a simple DTLS client/server test using OpenSSL 1.0.1, but with the same Cipher Suites that OpenSSL 1.0.0 uses, (to compare the two handshakes). This works fine with a 32 bit, (i686), build, but fails on 64 bit, (x86_64) with the following error:d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0If I do *not* override the default Cipher Suites, the 64 bit test works.I've attached packet captures from the various tests:dtls-assert-fail-x86_64.pcap -- failing handshake, (64 bit OpenSSL 1.0.1, using 1.0.0 Ciphers)dtls-assert-ok-i686.pcap -- working handshake, (32 bit OpenSSL 1.0.1, using 1.0.0 Ciphers)dtls-assert-ok-x86_64.pcap -- working handshake, (64 bit OpenSSL 1.0.1, using 1.0.1 Ciphers)Looking at the working/failing x86_64 pcaps, the MD differs for the chosen suite:TLS_RSA_WITH_AES_256_CBC_SHA256 (works)vs.TLS_RSA_WITH_AES_256_CBC_SHA (fails)My OpenSSL 1.0.1 code is built from the original sources, and linked directly into a test harness, with patches/overrides for the following:1) The random number generator is controlled by the test harness2) Time-of-day is controlled by the test harness3) Memory allocation/freeing is handled by the test harnessThe 64 bit code is built on Fedora 16 using:gcc version 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) I'd be interested to hear if other people experience the same problem with OpenSSL 1.0.1 x86_64 DTLS using TLS_RSA_WITH_AES_256_CBC_SHA, (or am I on my own here).Thanks,John Fitzgibbon dtls-assert-bug.tgz Description: application/compressed
Re: [openssl.org #2778] [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0
Never mind... found a 64 bit memory alignment error in the test harness. I'm not entirely clear how/why the alignment problem was impacting OpenSSL, but with that bug fixed, the DTLS problem goes away. Apologies for the false alarm, John Fitzgibbon From: John Fitzgibbon john_fitzgib...@yahoo.com To: OpenSSL Response Team r...@openssl.org Sent: Wednesday, March 28, 2012 12:42 PM Subject: [openssl.org #2778] [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 Hi, I'm trying to run a simple DTLS client/server test using OpenSSL 1.0.1, but with the same Cipher Suites that OpenSSL 1.0.0 uses, (to compare the two handshakes). This works fine with a 32 bit, (i686), build, but fails on 64 bit, (x86_64) with the following error: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 If I do *not* override the default Cipher Suites, the 64 bit test works. I've attached packet captures from the various tests: dtls-assert-fail-x86_64.pcap -- failing handshake, (64 bit OpenSSL 1.0.1, using 1.0.0 Ciphers) dtls-assert-ok-i686.pcap -- working handshake, (32 bit OpenSSL 1.0.1, using 1.0.0 Ciphers) dtls-assert-ok-x86_64.pcap -- working handshake, (64 bit OpenSSL 1.0.1, using 1.0.1 Ciphers) Looking at the working/failing x86_64 pcaps, the MD differs for the chosen suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (works) vs. TLS_RSA_WITH_AES_256_CBC_SHA (fails) My OpenSSL 1.0.1 code is built from the original sources, and linked directly into a test harness, with patches/overrides for the following: 1) The random number generator is controlled by the test harness 2) Time-of-day is controlled by the test harness 3) Memory allocation/freeing is handled by the test harness The 64 bit code is built on Fedora 16 using: gcc version 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) I'd be interested to hear if other people experience the same problem with OpenSSL 1.0.1 x86_64 DTLS using TLS_RSA_WITH_AES_256_CBC_SHA, (or am I on my own here). Thanks, John Fitzgibbon Never mind... found a 64 bit memory alignment error in the test harness. I'm not entirely clear how/why the alignment problem was impacting OpenSSL, but with that bug fixed, the DTLS problem goes away.Apologies for the false alarm,John FitzgibbonFrom: John Fitzgibbon john_fitzgib...@yahoo.com To: OpenSSL Response Team r...@openssl.org Sent: Wednesday, March 28, 2012 12:42 PM Subject: [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 Hi,I'm trying to run a simple DTLS client/server test using OpenSSL 1.0.1, but with the same Cipher Suites that OpenSSL 1.0.0 uses, (to compare the two handshakes). This works fine with a 32 bit, (i686), build, but fails on 64 bit, (x86_64) with the following error:d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0If I do *not* override the default Cipher Suites, the 64 bit test works.I've attached packet captures from the various tests:dtls-assert-fail-x86_64.pcap -- failing handshake, (64 bit OpenSSL 1.0.1, using 1.0.0 Ciphers)dtls-assert-ok-i686.pcap -- working handshake, (32 bit OpenSSL 1.0.1, using 1.0.0 Ciphers)dtls-assert-ok-x86_64.pcap -- working handshake, (64 bit OpenSSL 1.0.1, using 1.0.1 Ciphers)Looking at the working/failing x86_64 pcaps, the MD differs for the chosen suite:TLS_RSA_WITH_AES_256_CBC_SHA256 (works)vs.TLS_RSA_WITH_AES_256_CBC_SHA (fails)My OpenSSL 1.0.1 code is built from the original sources, and linked directly into a test harness, with patches/overrides for the following:1) The random number generator is controlled by the test harness2) Time-of-day is controlled by the test harness3) Memory allocation/freeing is handled by the test harnessThe 64 bit code is built on Fedora 16 using:gcc version 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) I'd be interested to hear if other people experience the same problem with OpenSSL 1.0.1 x86_64 DTLS using TLS_RSA_WITH_AES_256_CBC_SHA, (or am I on my own here).Thanks,John Fitzgibbon
Re: [openssl.org #2778] [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0
Geez, not my best day... this *is* still a problem. I forgot to reset my test cases to use the 1.0.0 list of Cipher Suites. The alignment bug had nothing whatsoever to do with this particular OpenSSL failure, and it still fails even with my now-squeaky-clean test harness. From: John Fitzgibbon john_fitzgib...@yahoo.com To: OpenSSL Response Team r...@openssl.org Sent: Wednesday, March 28, 2012 2:29 PM Subject: Re: [openssl.org #2778] [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 Never mind... found a 64 bit memory alignment error in the test harness. I'm not entirely clear how/why the alignment problem was impacting OpenSSL, but with that bug fixed, the DTLS problem goes away. Apologies for the false alarm, John Fitzgibbon From: John Fitzgibbon john_fitzgib...@yahoo.com To: OpenSSL Response Team r...@openssl.org Sent: Wednesday, March 28, 2012 12:42 PM Subject: [openssl.org #2778] [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 Hi, I'm trying to run a simple DTLS client/server test using OpenSSL 1.0.1, but with the same Cipher Suites that OpenSSL 1.0.0 uses, (to compare the two handshakes). This works fine with a 32 bit, (i686), build, but fails on 64 bit, (x86_64) with the following error: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 If I do *not* override the default Cipher Suites, the 64 bit test works. I've attached packet captures from the various tests: dtls-assert-fail-x86_64.pcap -- failing handshake, (64 bit OpenSSL 1.0.1, using 1.0.0 Ciphers) dtls-assert-ok-i686.pcap -- working handshake, (32 bit OpenSSL 1.0.1, using 1.0.0 Ciphers) dtls-assert-ok-x86_64.pcap -- working handshake, (64 bit OpenSSL 1.0.1, using 1.0.1 Ciphers) Looking at the working/failing x86_64 pcaps, the MD differs for the chosen suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (works) vs. TLS_RSA_WITH_AES_256_CBC_SHA (fails) My OpenSSL 1.0.1 code is built from the original sources, and linked directly into a test harness, with patches/overrides for the following: 1) The random number generator is controlled by the test harness 2) Time-of-day is controlled by the test harness 3) Memory allocation/freeing is handled by the test harness The 64 bit code is built on Fedora 16 using: gcc version 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) I'd be interested to hear if other people experience the same problem with OpenSSL 1.0.1 x86_64 DTLS using TLS_RSA_WITH_AES_256_CBC_SHA, (or am I on my own here). Thanks, John Fitzgibbon Geez, not my best day... this *is* still a problem. I forgot to reset my test cases to use the 1.0.0 list of Cipher Suites. The alignment bug had nothing whatsoever to do with this particular OpenSSL failure, and it still fails even with my now-squeaky-clean test harness.From: John Fitzgibbon john_fitzgib...@yahoo.com To: OpenSSL Response Team r...@openssl.org Sent: Wednesday, March 28, 2012 2:29 PM Subject: Re: [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 Never mind... found a 64 bit memory alignment error in the test harness. I'm not entirely clear how/why the alignment problem was impacting OpenSSL, but with that bug fixed, the DTLS problem goes away.Apologies for the false alarm,John FitzgibbonFrom: John Fitzgibbon john_fitzgib...@yahoo.com To: OpenSSL Response Team r...@openssl.org Sent: Wednesday, March 28, 2012 12:42 PM Subject: [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 Hi,I'm trying to run a simple DTLS client/server test using OpenSSL 1.0.1, but with the same Cipher Suites that OpenSSL 1.0.0 uses, (to compare the two handshakes). This works fine with a 32 bit, (i686), build, but fails on 64 bit, (x86_64) with the following error:d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0If I do *not* override the default Cipher Suites, the 64 bit test works.I've attached packet captures from the various tests:dtls-assert-fail-x86_64.pcap -- failing handshake, (64 bit OpenSSL 1.0.1, using 1.0.0 Ciphers)dtls-assert-ok-i686.pcap -- working handshake, (32 bit OpenSSL 1.0.1, using 1.0.0 Ciphers)dtls-assert-ok-x86_64.pcap -- working handshake, (64 bit OpenSSL 1.0.1, using 1.0.1 Ciphers)Looking at the working/failing x86_64 pcaps, the MD differs for the chosen suite:TLS_RSA_WITH_AES_256_CBC_SHA256 (works)vs.TLS_RSA_WITH_AES_256_CBC_SHA (fails)My OpenSSL 1.0.1 code is built from the original sources, and linked directly into a test harness, with patches/overrides for the following:1) The random number generator is controlled by the test harness2) Time-of-day is controlled by the test harness3) Memory allocation/freeing is handled by the test harnessThe 64 bit code is built on Fedora 16 using:gcc version 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) I'd be interested to hear if other people
Re: [openssl.org #2778] [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0
Don't know if this is related or not, but I'm also running a very similar test that uses TLS instead of DTLS, (same scenario, OpenSSL 1.0.1 with 1.0.0 Cipher Suites selected). That works fine, except that the 64 bit version of the test looks like it doesn't include the Empty Fragments security countermeasure, (at least the telltale 32 byte record at the start of each packet isn't there). My config command is the same for 32/64 bits: ./config --prefix=../../ssl --openssldir=../../ssl/openssl no-threads no-shared no-sse2 no-idea no-mdc2 no-rc5 $NO_HEARTBEATS ... where NO_HEARTBEATS=no-heartbeats for OpenSSL 1.0.1 only, (option didn't exist for 1.0.0). My understanding was that the Empty Fragments countermeasure should be on by default for both builds?? From: John Fitzgibbon john_fitzgib...@yahoo.com To: OpenSSL Response Team r...@openssl.org Sent: Wednesday, March 28, 2012 2:37 PM Subject: Re: [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 Geez, not my best day... this *is* still a problem. I forgot to reset my test cases to use the 1.0.0 list of Cipher Suites. The alignment bug had nothing whatsoever to do with this particular OpenSSL failure, and it still fails even with my now-squeaky-clean test harness. From: John Fitzgibbon john_fitzgib...@yahoo.com To: OpenSSL Response Team r...@openssl.org Sent: Wednesday, March 28, 2012 2:29 PM Subject: Re: [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 Never mind... found a 64 bit memory alignment error in the test harness. I'm not entirely clear how/why the alignment problem was impacting OpenSSL, but with that bug fixed, the DTLS problem goes away. Apologies for the false alarm, John Fitzgibbon From: John Fitzgibbon john_fitzgib...@yahoo.com To: OpenSSL Response Team r...@openssl.org Sent: Wednesday, March 28, 2012 12:42 PM Subject: [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 Hi, I'm trying to run a simple DTLS client/server test using OpenSSL 1.0.1, but with the same Cipher Suites that OpenSSL 1.0.0 uses, (to compare the two handshakes). This works fine with a 32 bit, (i686), build, but fails on 64 bit, (x86_64) with the following error: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 If I do *not* override the default Cipher Suites, the 64 bit test works. I've attached packet captures from the various tests: dtls-assert-fail-x86_64.pcap -- failing handshake, (64 bit OpenSSL 1.0.1, using 1.0.0 Ciphers) dtls-assert-ok-i686.pcap -- working handshake, (32 bit OpenSSL 1.0.1, using 1.0.0 Ciphers) dtls-assert-ok-x86_64.pcap -- working handshake, (64 bit OpenSSL 1.0.1, using 1.0.1 Ciphers) Looking at the working/failing x86_64 pcaps, the MD differs for the chosen suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (works) vs. TLS_RSA_WITH_AES_256_CBC_SHA (fails) My OpenSSL 1.0.1 code is built from the original sources, and linked directly into a test harness, with patches/overrides for the following: 1) The random number generator is controlled by the test harness 2) Time-of-day is controlled by the test harness 3) Memory allocation/freeing is handled by the test harness The 64 bit code is built on Fedora 16 using: gcc version 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) I'd be interested to hear if other people experience the same problem with OpenSSL 1.0.1 x86_64 DTLS using TLS_RSA_WITH_AES_256_CBC_SHA, (or am I on my own here). Thanks, John Fitzgibbon Don't know if this is related or not, but I'm also running a very similar test that uses TLS instead of DTLS, (same scenario, OpenSSL 1.0.1 with 1.0.0 Cipher Suites selected). That works fine, except that the 64 bit version of the test looks like it doesn't include the "Empty Fragments" security countermeasure, (at least the telltale 32 byte record at the start of each packet isn't there).My config command is the same for 32/64 bits:./config --prefix=../../ssl --openssldir=../../ssl/openssl no-threads no-shared no-sse2 no-idea no-mdc2 no-rc5 $NO_HEARTBEATS... where NO_HEARTBEATS="no-heartbeats" for OpenSSL 1.0.1 only, (option didn't exist for 1.0.0).My understanding was that the "Empty Fragments" countermeasure should be "on" by default for both builds??From: John Fitzgibbon john_fitzgib...@yahoo.com To: OpenSSL Response Team r...@openssl.org Sent: Wednesday, March 28, 2012 2:37 PM Subject: Re: [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0 Geez, not my best day... this *is* still a problem. I forgot to reset my test cases to use the 1.0.0 list of Cipher Suites. The alignment bug had nothing whatsoever to do with this particular OpenSSL failure, and it still fails even with my now-squeaky-clean test harness.From: John Fitzgibbon john_fitzgib...@yahoo.com To:
[openssl.org #2778] [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0
[john_fitzgib...@yahoo.com - Fri Mar 30 09:21:50 2012]: Don't know if this is related or not, but I'm also running a very similar test that uses TLS instead of DTLS, (same scenario, OpenSSL 1.0.1 with 1.0.0 Cipher Suites selected). That works fine, except that the 64 bit version of the test looks like it doesn't include the Empty Fragments security countermeasure, (at least the telltale 32 byte record at the start of each packet isn't there). If you're using TLS v1.1 or 1.2 then you shouldn't encounter empty fragments on any version as they are not required any more as CBC mode includes an explicit IV. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [openssl.org #2778] [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0
the 64 bit version of the test looks like it doesn't include the Empty Fragments security countermeasure If you're using TLS v1.1 or 1.2 then you shouldn't encounter empty fragments on any version as they are not required any more as CBC mode includes an explicit IV. The TLS tests are 1.0. The same code built on a 32 bit platform gives different results, though the initial negotiation looks the same. I've attached the 32/64 bit pcaps. The tests that give problems, (DTLS and TLS), have three things in common: 1) They use OpenSSL 1.0.1 2) They are built on x86_64 3) They set the cipher list with a call to: SSL_CTX_set_cipher_list(ctx, [a very long cipher list]) The very long cipher list in question is a copy of this string: static const char ssl_1_0_0_ciphers[]= ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA: DHE-RSA-CAMELLIA256-SHA:DHE-DSS-CAMELLIA256-SHA:ECDH-RSA-AES256-SHA:ECDH-ECDSA-AES256-SHA: AES256-SHA:CAMELLIA256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA: EDH-RSA-DES-CBC3-SHA:EDH-DSS-DES-CBC3-SHA:ECDH-RSA-DES-CBC3-SHA:ECDH-ECDSA-DES-CBC3-SHA: DES-CBC3-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:DHE-RSA-AES128-SHA: DHE-DSS-AES128-SHA:DHE-RSA-SEED-SHA:DHE-DSS-SEED-SHA:DHE-RSA-CAMELLIA128-SHA: DHE-DSS-CAMELLIA128-SHA:ECDH-RSA-AES128-SHA:ECDH-ECDSA-AES128-SHA:AES128-SHA: SEED-SHA:CAMELLIA128-SHA:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-RC4-SHA:ECDH-RSA-RC4-SHA: ECDH-ECDSA-RC4-SHA:RC4-SHA:RC4-MD5:EDH-RSA-DES-CBC-SHA:EDH-DSS-DES-CBC-SHA:DES-CBC-SHA: EXP-EDH-RSA-DES-CBC-SHA:EXP-EDH-DSS-DES-CBC-SHA:EXP-DES-CBC-SHA:EXP-RC2-CBC-MD5:EXP-RC4-MD5; If I do any of the following the tests work as expected: - revert to OpenSSL 1.0.0h - build 32 bit - remove the cipher list, (use defaults) tls-1.0-i686.pcap Description: Binary data tls-1.0-x86_64.pcap Description: Binary data
[openssl.org #2775] Additional information for openssh segfault
I too am experiencing a segfault in openssh when connecting to a particular Linux host. This did work fine with the previous version of libssl (I'm running Ubuntu 12.04 so this was 1.0.0 before upgrading to 1.0.1). No core file is created but I can do the gdb backtrace and disassemble. You've correctly guessed the failing instruction :) but not sure why it's happening. Program received signal SIGSEGV, Segmentation fault. _vpaes_decrypt_core () at vpaes-x86.s:221 221vpaes-x86.s: No such file or directory. (gdb) bt #0 _vpaes_decrypt_core () at vpaes-x86.s:221 #1 0xb7e369e5 in vpaes_cbc_encrypt () at vpaes-x86.s:641 #2 0x732d6361 in ?? () (gdb) disassemble Dump of assembler code for function _vpaes_decrypt_core: 0xb7e36310 +0: mov0xf0(%edx),%eax 0xb7e36316 +6: lea0x260(%ebp),%ebx 0xb7e3631c +12: movdqa %xmm6,%xmm1 0xb7e36320 +16: movdqa -0x40(%ebx),%xmm2 0xb7e36325 +21: pandn %xmm0,%xmm1 0xb7e36329 +25: mov%eax,%ecx 0xb7e3632b +27: psrld $0x4,%xmm1 0xb7e36330 +32: movdqu (%edx),%xmm5 0xb7e36334 +36: shl$0x4,%ecx 0xb7e36337 +39: pand %xmm6,%xmm0 0xb7e3633b +43: pshufb %xmm0,%xmm2 0xb7e36340 +48: movdqa -0x30(%ebx),%xmm0 0xb7e36345 +53: xor$0x30,%ecx 0xb7e36348 +56: pshufb %xmm1,%xmm0 0xb7e3634d +61: and$0x30,%ecx 0xb7e36350 +64: pxor %xmm5,%xmm2 0xb7e36354 +68: movdqa 0xb0(%ebp),%xmm5 0xb7e3635c +76: pxor %xmm2,%xmm0 0xb7e36360 +80: add$0x10,%edx 0xb7e36363 +83: lea-0x160(%ebx,%ecx,1),%ecx 0xb7e3636a +90: jmp0xb7e363fa _vpaes_decrypt_core+234 0xb7e3636f +95:nop 0xb7e36370 +96: movdqa -0x20(%ebx),%xmm4 ---Type return to continue, or q return to quit--- 0xb7e36375 +101: pshufb %xmm2,%xmm4 0xb7e3637a +106: pxor %xmm0,%xmm4 0xb7e3637e +110: movdqa -0x10(%ebx),%xmm0 0xb7e36383 +115: pshufb %xmm3,%xmm0 0xb7e36388 +120: pxor %xmm4,%xmm0 0xb7e3638c +124: add$0x10,%edx 0xb7e3638f +127: pshufb %xmm5,%xmm0 0xb7e36394 +132: movdqa (%ebx),%xmm4 0xb7e36398 +136: pshufb %xmm2,%xmm4 0xb7e3639d +141: pxor %xmm0,%xmm4 0xb7e363a1 +145: movdqa 0x10(%ebx),%xmm0 0xb7e363a6 +150: pshufb %xmm3,%xmm0 0xb7e363ab +155: pxor %xmm4,%xmm0 0xb7e363af +159: sub$0x1,%eax 0xb7e363b2 +162: pshufb %xmm5,%xmm0 0xb7e363b7 +167: movdqa 0x20(%ebx),%xmm4 0xb7e363bc +172: pshufb %xmm2,%xmm4 0xb7e363c1 +177: pxor %xmm0,%xmm4 0xb7e363c5 +181: movdqa 0x30(%ebx),%xmm0 0xb7e363ca +186: pshufb %xmm3,%xmm0 0xb7e363cf +191: pxor %xmm4,%xmm0 0xb7e363d3 +195: pshufb %xmm5,%xmm0 0xb7e363d8 +200: movdqa 0x40(%ebx),%xmm4 0xb7e363dd +205: pshufb %xmm2,%xmm4 ---Type return to continue, or q return to quit--- 0xb7e363e2 +210: pxor %xmm0,%xmm4 0xb7e363e6 +214: movdqa 0x50(%ebx),%xmm0 0xb7e363eb +219: pshufb %xmm3,%xmm0 0xb7e363f0 +224: pxor %xmm4,%xmm0 0xb7e363f4 +228: palignr $0xc,%xmm5,%xmm5 0xb7e363fa +234: movdqa %xmm6,%xmm1 0xb7e363fe +238: pandn %xmm0,%xmm1 0xb7e36402 +242: psrld $0x4,%xmm1 0xb7e36407 +247: pand %xmm6,%xmm0 0xb7e3640b +251: movdqa -0x20(%ebp),%xmm2 0xb7e36410 +256: pshufb %xmm0,%xmm2 0xb7e36415 +261: pxor %xmm1,%xmm0 0xb7e36419 +265: movdqa %xmm7,%xmm3 0xb7e3641d +269: pshufb %xmm1,%xmm3 0xb7e36422 +274: pxor %xmm2,%xmm3 0xb7e36426 +278: movdqa %xmm7,%xmm4 0xb7e3642a +282: pshufb %xmm0,%xmm4 0xb7e3642f +287: pxor %xmm2,%xmm4 0xb7e36433 +291: movdqa %xmm7,%xmm2 0xb7e36437 +295: pshufb %xmm3,%xmm2 0xb7e3643c +300: pxor %xmm0,%xmm2 0xb7e36440 +304: movdqa %xmm7,%xmm3 0xb7e36444 +308: pshufb %xmm4,%xmm3 0xb7e36449 +313: pxor %xmm1,%xmm3 ---Type return to continue, or q return to quit--- = 0xb7e3644d +317:movdqu (%edx),%xmm0 0xb7e36451 +321: jne0xb7e36370 _vpaes_decrypt_core+96 0xb7e36457 +327: movdqa 0x60(%ebx),%xmm4 0xb7e3645c +332: pshufb %xmm2,%xmm4 0xb7e36461 +337: pxor %xmm0,%xmm4 0xb7e36465 +341: movdqa 0x70(%ebx),%xmm0 0xb7e3646a +346: movdqa (%ecx),%xmm2 0xb7e3646e +350: pshufb %xmm3,%xmm0 0xb7e36473 +355: pxor %xmm4,%xmm0 0xb7e36477 +359: pshufb %xmm2,%xmm0 0xb7e3647c +364: ret End of assembler dump. (gdb) info reg eax0x277a77b5 662337461 ecx0xb7e35fa0 -1209835616 edx0x80090ff8 -2146889736 ebx0xb7e360d0 -1209835312 esp0xbfffb05c 0xbfffb05c ebp0xb7e35e70 0xb7e35e70 esi0x80081bd8 -2146952232 edi0xeba0 -5216 eip0xb7e3644d 0xb7e3644d _vpaes_decrypt_core+317 eflags 0x10202 [ IF RF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 Sincerely, Michael Russo, Systems Engineer PaperSolve, Inc. 268 Watchogue Road Staten Island, NY 10314 I too am experiencing a segfault in
Re: [openssl.org #2775] Additional information for openssh segfault
I too am experiencing a segfault in openssh when connecting to a particular Linux host. So you mean you can login to say 2-3-4-5 other hosts, but not one specific. Tough break... This did work fine with the previous version of libssl (I'm running Ubuntu 12.04 so this was 1.0.0 before upgrading to 1.0.1). No core file is created but I can do the gdb backtrace and disassemble. Presumably you have to 'ulimit -c unlimited' (if you run sh) in order to get the core. But running under debugger is as good and the only way to do that I'm about to suggest... You've correctly guessed the failing instruction :) but not sure why it's happening. Program received signal SIGSEGV, Segmentation fault. _vpaes_decrypt_core () at vpaes-x86.s:221 (gdb) disassemble Dump of assembler code for function _vpaes_decrypt_core: 0xb7e36310 +0: mov0xf0(%edx),%eax ... 0xb7e363af +159: sub$0x1,%eax ... = 0xb7e3644d +317:movdqu (%edx),%xmm0 0xb7e36451 +321: jne0xb7e36370 _vpaes_decrypt_core+96 ... End of assembler dump. (gdb) info reg eax0x277a77b5 662337461 ... edx0x80090ff8 -2146889736 As you can see 'eax' is crazy, 'edx' lies on page boundary and next page ought to be unaccessible causing SEGV. Now under gdb. First see if it fails upon first call. I.e. set breakpoint to _vpaes_decrypt_core, 'b _vpaes_decrypt_core', start program, when it hits the breakpoint, write down value in 'edx' register and issue 'continue'. Does it SEGV or stops at break point again? If latter, count 'continue's writing down values in 'edx' register. Is it same 'edx'? Best would be if it's same 'edx' and it's not first call... Once you know which invocation fails, restart program with same breakpoint, at first invocation check 'edx' and set watch point to 0xf0(%edx), 'watch *((int *)(value-in-edx+240)), 'continue'. Eventually it should break at watch point. 'where'? If already first call to _vpaes_decrypt_core SEGVs, then you would have to set breakpoint to vpaes_set_decrypt_key, find out pointer to key schedule, set watchpoint as above and see where it breaks at watchpoint. In order to find pointer to key schedule, issue 'stepi' 9 times after it breaks at vpaes_set_decrypt_key and examine 'edx'. Naturally vpaes_set_decrypt_key will hit the watch point, you should 'continue' to find out who *else* is modifying it. Also double-check that it's same value in 'edx' about entry to _vpaes_decrypt_core. Do you get the drift? __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
[openssl.org #2775] Segmentation fault libcrypto.so.1.0.0
Damn, I knew I should have taken that assembly language course all those years ago. And yes, it does appear that it's only old versions of SSH that I'm having a problem connecting to (eg OpenSSH_3.6.1p2 w/ OpenSSL 0.9.7f, another host running 4.3p2 and 0.9.8e is fine). Well I set the breakpoint and I can continue on, so it is not crashing at the first breakpoint, and I get the same value in edx until I'm asked to enter my passcode (this server and the other one running 3.6.1p2 are running the RSA PAM modules, which I really hope aren't related): (gdb) b _vpaes_decrypt_core Function _vpaes_decrypt_core not defined. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (_vpaes_decrypt_core) pending. (gdb) run mi...@smtp.readq.com Starting program: /usr/bin/ssh mi...@smtp.readq.com [Thread debugging using libthread_db enabled] Breakpoint 1, _vpaes_decrypt_core () at vpaes-x86.s:148 148vpaes-x86.s: No such file or directory. (gdb) info reg eax0x0 0 ecx0x0 0 edx0x80084ae8 -2146940184 ebx0xbfffb0cc -1073762100 esp0xbfffb08c 0xbfffb08c ebp0xb7e35e70 0xb7e35e70 esi0x80080768 -2146957464 edi0x0 0 eip0xb7e36310 0xb7e36310 _vpaes_decrypt_core eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 (gdb) continue Continuing. Breakpoint 1, _vpaes_decrypt_core () at vpaes-x86.s:148 148in vpaes-x86.s (gdb) info reg eax0x0 0 ecx0x0 0 edx0x80084ae8 -2146940184 ebx0xbfffb0cc -1073762100 esp0xbfffb08c 0xbfffb08c ebp0xb7e35e70 0xb7e35e70 esi0x80080778 -2146957448 edi0x0 0 eip0xb7e36310 0xb7e36310 _vpaes_decrypt_core eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 (gdb) continue Continuing. edx0x80084ae8 -2146940184 Continuing. edx0x80084ae8 -2146940184 Continuing. edx0x80084ae8 -2146940184 Continuing. edx0x80084ae8 -2146940184 Continuing. edx0x80084ae8 -2146940184 Continuing. edx0x80084ae8 -2146940184 Continuing. Enter PASSCODE: I snipped a bunch of continue and info reg's up there to show you the value in edx is not changing. Breakpoint 1, _vpaes_decrypt_core () at vpaes-x86.s:148 148in vpaes-x86.s (gdb) info reg eax0x0 0 ecx0x0 0 edx0x80084ae8 -2146940184 ebx0xbfffb0ac -1073762132 esp0xbfffb06c 0xbfffb06c ebp0xb7e35e70 0xb7e35e70 esi0x80080768 -2146957464 edi0x0 0 eip0xb7e36310 0xb7e36310 _vpaes_decrypt_core eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 (gdb) continue Continuing. I set the watchpoint: (gdb) watch *((int *)(-2146940184+240)) Hardware watchpoint 2: *((int *)(-2146940184+240)) But after about 15 'continue's I type 'help continue' and realize I can give it a number. Some number of 'continue's later: (gdb) continue 100 Will ignore next 99 crossings of breakpoint 1. Continuing. Hardware watchpoint 2: *((int *)(-2146940184+240)) Old value = 9 New value = 915002721 vpaes_cbc_encrypt () at vpaes-x86.s:647 647in vpaes-x86.s (gdb) where #0 vpaes_cbc_encrypt () at vpaes-x86.s:647 #1 0x68732d63 in ?? () Guess I've got to see if I can figure out if it's an incompatibility with the old version of SSH or the equally ancient RSA PAM modules on those 2 servers where I'm having the trouble. Sincerely, Michael Russo, Systems Engineer PaperSolve, Inc. 268 Watchogue Road Staten Island, NY 10314 Damn, I knew I should have taken that assembly language course all those years ago. And yes, it does appear that its only old versions of SSH that Im having a problem connecting to (eg OpenSSH_3.6.1p2 w/ OpenSSL 0.9.7f, another host running 4.3p2 and 0.9.8e is fine). Well I set the breakpoint and I can continue on, so it is not crashing at the first breakpoint, and I get the same value in edx until Im asked to enter my passcode (this server and the other one running 3.6.1p2 are running the RSA PAM modules, which I really hope arent related): (gdb) b _vpaes_decrypt_core Function _vpaes_decrypt_core not defined. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (_vpaes_decrypt_core) pending. (gdb) run mi...@smtp.readq.com Starting program: /usr/bin/ssh mi...@smtp.readq.com [Thread debugging using
Re: [openssl.org #2775] Segmentation fault libcrypto.so.1.0.0
(gdb) continue 100 Will ignore next 99 crossings of breakpoint 1. Continuing. Hardware watchpoint 2: *((int *)(-2146940184+240)) Old value = 9 New value = 915002721 vpaes_cbc_encrypt () at vpaes-x86.s:647 647in vpaes-x86.s (gdb) where #0 vpaes_cbc_encrypt () at vpaes-x86.s:647 #1 0x68732d63 in ?? () 'disassemble' [till you see offending instruction] and 'info reg' at this point, please. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
[openssl.org #2775]
Well I executed this right after the 'where' from last time (still had it up in a window though the connection has long since timed out): (gdb) info reg eax0x0 0 ecx0xb7e35f90 -1209835632 edx0x80084ae8 -2146940184 ebx0x3018 12312 esp0xbfffb0700xbfffb070 ebp0xb7e35e70 0xb7e35e70 esi0x80081bb8 -2146952264 edi0xebb0 -5200 eip0xb7e369fd0xb7e369fd vpaes_cbc_encrypt+189 eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 (gdb) disassemble Dump of assembler code for function vpaes_cbc_encrypt: 0xb7e36940 +0: push %ebp 0xb7e36941 +1: push %ebx 0xb7e36942 +2: push %esi 0xb7e36943 +3: push %edi 0xb7e36944 +4: mov0x14(%esp),%esi 0xb7e36948 +8: mov0x18(%esp),%edi 0xb7e3694c +12: mov0x1c(%esp),%eax 0xb7e36950 +16: mov0x20(%esp),%edx 0xb7e36954 +20: lea-0x38(%esp),%ebx 0xb7e36958 +24: mov0x24(%esp),%ebp 0xb7e3695c +28: and$0xfff0,%ebx 0xb7e3695f +31:mov0x28(%esp),%ecx 0xb7e36963 +35: xchg %esp,%ebx 0xb7e36965 +37: movdqu 0x0(%ebp),%xmm1 0xb7e3696a +42: sub%esi,%edi 0xb7e3696c +44: mov%ebx,0x30(%esp) 0xb7e36970 +48: mov%edi,(%esp) 0xb7e36973 +51: sub$0x10,%eax 0xb7e36976 +54: mov%edx,0x4(%esp) 0xb7e3697a +58: mov%ebp,0x8(%esp) 0xb7e3697e +62: mov%eax,%edi 0xb7e36980 +64: lea0xf4e5,%ebp 0xb7e36986 +70: call 0xb7e361c0 _vpaes_preheat ---Type return to continue, or q return to quit--- 0xb7e3698b +75: cmp$0x0,%ecx 0xb7e3698e +78: je 0xb7e369d0 vpaes_cbc_encrypt+144 0xb7e36990 +80: jmp0xb7e369a0 vpaes_cbc_encrypt+96 0xb7e36992 +82: lea0x0(%esi,%eiz,1),%esi 0xb7e36999 +89: lea0x0(%edi,%eiz,1),%edi 0xb7e369a0 +96: movdqu (%esi),%xmm0 0xb7e369a4 +100: pxor %xmm1,%xmm0 0xb7e369a8 +104: call 0xb7e361d0 _vpaes_encrypt_core 0xb7e369ad +109: mov(%esp),%ebx 0xb7e369b0 +112: mov0x4(%esp),%edx 0xb7e369b4 +116: movdqa %xmm0,%xmm1 0xb7e369b8 +120: movdqu %xmm0,(%ebx,%esi,1) 0xb7e369bd +125: lea0x10(%esi),%esi 0xb7e369c0 +128: sub$0x10,%edi 0xb7e369c3 +131: jae0xb7e369a0 vpaes_cbc_encrypt+96 0xb7e369c5 +133: jmp0xb7e36a05 vpaes_cbc_encrypt+197 0xb7e369c7 +135: mov%esi,%esi 0xb7e369c9 +137: lea0x0(%edi,%eiz,1),%edi 0xb7e369d0 +144: movdqu (%esi),%xmm0 0xb7e369d4 +148: movdqa %xmm1,0x10(%esp) 0xb7e369da +154: movdqa %xmm0,0x20(%esp) 0xb7e369e0 +160: call 0xb7e36310 _vpaes_decrypt_core 0xb7e369e5 +165: mov(%esp),%ebx 0xb7e369e8 +168: mov0x4(%esp),%edx ---Type return to continue, or q return to quit--- 0xb7e369ec +172: pxor 0x10(%esp),%xmm0 0xb7e369f2 +178: movdqa 0x20(%esp),%xmm1 0xb7e369f8 +184: movdqu %xmm0,(%ebx,%esi,1) = 0xb7e369fd +189: lea0x10(%esi),%esi 0xb7e36a00 +192: sub$0x10,%edi 0xb7e36a03 +195: jae0xb7e369d0 vpaes_cbc_encrypt+144 0xb7e36a05 +197: mov0x8(%esp),%ebx 0xb7e36a09 +201: mov0x30(%esp),%esp 0xb7e36a0d +205: movdqu %xmm1,(%ebx) 0xb7e36a11 +209: pop%edi 0xb7e36a12 +210: pop%esi 0xb7e36a13 +211: pop%ebx 0xb7e36a14 +212: pop%ebp 0xb7e36a15 +213: ret End of assembler dump. (gdb) info reg eax0x0 0 ecx0xb7e35f90 -1209835632 edx0x80084ae8 -2146940184 ebx0x3018 12312 esp0xbfffb0700xbfffb070 ebp0xb7e35e70 0xb7e35e70 esi0x80081bb8 -2146952264 edi0xebb0 -5200 eip0xb7e369fd0xb7e369fd vpaes_cbc_encrypt+189 eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 Sincerely, Michael Russo, Systems Engineer PaperSolve, Inc. 268 Watchogue Road Staten Island, NY 10314 Well I executed this right after the where from last time (still had it up in a window though the connection has long since timed out): (gdb) info reg eax 0x0 0 ecx 0xb7e35f90 -1209835632 edx 0x80084ae8 -2146940184 ebx 0x3018 12312 esp 0xbfffb070 0xbfffb070 ebp 0xb7e35e70 0xb7e35e70 esi 0x80081bb8 -2146952264 edi 0xebb0 -5200 eip 0xb7e369fd 0xb7e369fd vpaes_cbc_encrypt189 eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 (gdb) disassemble Dump of assembler code for function vpaes_cbc_encrypt: 0xb7e36940 0: push %ebp 0xb7e36941 1: push %ebx 0xb7e36942 2: push %esi 0xb7e36943 3: push %edi 0xb7e36944 4: mov 0x14(%esp),%esi 0xb7e36948 8: mov 0x18(%esp),%edi 0xb7e3694c 12: mov 0x1c(%esp),%eax
Re: [openssl.org #2775] [openssh 5.9p1-8] Segmentation fault libcrypto.so.1.0.0
On Wed, Mar 28, 2012 at 10:34:49AM +0200, Joe Bew via RT wrote: Please, consider this bugreport: https://bugs.archlinux.org/task/29111 There is also: http://bugs.debian.org/665836 Kurt __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [openssl.org #2778] [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0
Some interesting observations: 1) Changed the cipher lists to much simpler values: ciphers = AES256-SHA256 = works ciphers = AES256-SHA = fails 2) On a hunch, I tried adding no-asm to the config line: 2.1) TLS test now works and yields a perfect match with the 32 bit test 2.2) DTLS test works, but there is an interesting side-effect... ... the random bytes field differs in the server hello. There should be no difference because the test harness is supplying a non-random PRG, (and time-of-day is also strictly controlled). I'm guessing that OpenSSL uses a SHA-based PRF on random bytes? If so, then it looks like something might be up with 64-bit SHA, (or PRF??) in 1.0.1. I've attached the no-asm-64bit pkt cap and the reference 32 bit equivalent. dtls-no-asm-x86_64.pcap Description: Binary data dtls-i686.pcap Description: Binary data
Re: [openssl.org #2778] [BUG:] OpenSSL 1.0.1 x86_64: d1_pkt.c(444): OpenSSL internal error, assertion failed: t = 0
DTLS test works, but the random bytes field differs in the server hello. There should be no difference because the test harness is supplying a non-random PRNG. This is happening because of the following, (which looks like a bug), in ssl/d1_srvr.c, line 923: Time=(unsigned long)time(NULL); /* Time */ l2n(Time,p); RAND_pseudo_bytes(p,SSL3_RANDOM_SIZE-sizeof(Time)); sizeof(Time) is 8 bytes in x86_84, but l2n() has only advanced the pointer 4 bytes. This leaves 4 bytes of uninitialized data exposed in the random bytes field for x68_64. Unless l2n() can do something different on other platforms, line 923 should be more explicit: RAND_pseudo_bytes(p,SSL3_RANDOM_SIZE-4); With this fix, my no-asm 64bit DTLS test yields a pcap that matches the 32 bit build. ... that isolates the problem to the 64 bit assember changes between 1.0.0 and 1.0.1, specifically, (it would seem), for SHA-1. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
