[openssl.org #2775]
Woo-hoo! Success! I was able to connect to both machines that I could not before the patch. Thank you so much Andy! root@hawty:/usr/src/openssl-1.0.1# ssh mi...@smtp.readq.com ssh: /usr/src/openssl-1.0.1/libcrypto.so.1.0.0: no version information available (required by ssh) Enter PASSCODE: Last login: Wed Mar 28 14:39:13 2012 from static-98-113-84-130.nycmny.fios.verizon.net [miker@turing miker]$ Sincerely, Michael Russo, Systems Engineer PaperSolve, Inc. 268 Watchogue Road Staten Island, NY 10314 Woo-hoo! Success! I was able to connect to both machines that I could not before the patch. Thank you so much Andy! root@hawty:/usr/src/openssl-1.0.1# ssh mi...@smtp.readq.com ssh: /usr/src/openssl-1.0.1/libcrypto.so.1.0.0: no version information available (required by ssh) Enter PASSCODE: Last login: Wed Mar 28 14:39:13 2012 from static-98-113-84-130.nycmny.fios.verizon.net [miker@turing miker]$ Sincerely, Michael Russo, Systems Engineer PaperSolve, Inc. 268 Watchogue Road Staten Island, NY 10314
[openssl.org #2775] Additional information for openssh segfault
I too am experiencing a segfault in openssh when connecting to a particular Linux host. This did work fine with the previous version of libssl (I'm running Ubuntu 12.04 so this was 1.0.0 before upgrading to 1.0.1). No core file is created but I can do the gdb backtrace and disassemble. You've correctly guessed the failing instruction :) but not sure why it's happening. Program received signal SIGSEGV, Segmentation fault. _vpaes_decrypt_core () at vpaes-x86.s:221 221vpaes-x86.s: No such file or directory. (gdb) bt #0 _vpaes_decrypt_core () at vpaes-x86.s:221 #1 0xb7e369e5 in vpaes_cbc_encrypt () at vpaes-x86.s:641 #2 0x732d6361 in ?? () (gdb) disassemble Dump of assembler code for function _vpaes_decrypt_core: 0xb7e36310 +0: mov0xf0(%edx),%eax 0xb7e36316 +6: lea0x260(%ebp),%ebx 0xb7e3631c +12: movdqa %xmm6,%xmm1 0xb7e36320 +16: movdqa -0x40(%ebx),%xmm2 0xb7e36325 +21: pandn %xmm0,%xmm1 0xb7e36329 +25: mov%eax,%ecx 0xb7e3632b +27: psrld $0x4,%xmm1 0xb7e36330 +32: movdqu (%edx),%xmm5 0xb7e36334 +36: shl$0x4,%ecx 0xb7e36337 +39: pand %xmm6,%xmm0 0xb7e3633b +43: pshufb %xmm0,%xmm2 0xb7e36340 +48: movdqa -0x30(%ebx),%xmm0 0xb7e36345 +53: xor$0x30,%ecx 0xb7e36348 +56: pshufb %xmm1,%xmm0 0xb7e3634d +61: and$0x30,%ecx 0xb7e36350 +64: pxor %xmm5,%xmm2 0xb7e36354 +68: movdqa 0xb0(%ebp),%xmm5 0xb7e3635c +76: pxor %xmm2,%xmm0 0xb7e36360 +80: add$0x10,%edx 0xb7e36363 +83: lea-0x160(%ebx,%ecx,1),%ecx 0xb7e3636a +90: jmp0xb7e363fa _vpaes_decrypt_core+234 0xb7e3636f +95:nop 0xb7e36370 +96: movdqa -0x20(%ebx),%xmm4 ---Type return to continue, or q return to quit--- 0xb7e36375 +101: pshufb %xmm2,%xmm4 0xb7e3637a +106: pxor %xmm0,%xmm4 0xb7e3637e +110: movdqa -0x10(%ebx),%xmm0 0xb7e36383 +115: pshufb %xmm3,%xmm0 0xb7e36388 +120: pxor %xmm4,%xmm0 0xb7e3638c +124: add$0x10,%edx 0xb7e3638f +127: pshufb %xmm5,%xmm0 0xb7e36394 +132: movdqa (%ebx),%xmm4 0xb7e36398 +136: pshufb %xmm2,%xmm4 0xb7e3639d +141: pxor %xmm0,%xmm4 0xb7e363a1 +145: movdqa 0x10(%ebx),%xmm0 0xb7e363a6 +150: pshufb %xmm3,%xmm0 0xb7e363ab +155: pxor %xmm4,%xmm0 0xb7e363af +159: sub$0x1,%eax 0xb7e363b2 +162: pshufb %xmm5,%xmm0 0xb7e363b7 +167: movdqa 0x20(%ebx),%xmm4 0xb7e363bc +172: pshufb %xmm2,%xmm4 0xb7e363c1 +177: pxor %xmm0,%xmm4 0xb7e363c5 +181: movdqa 0x30(%ebx),%xmm0 0xb7e363ca +186: pshufb %xmm3,%xmm0 0xb7e363cf +191: pxor %xmm4,%xmm0 0xb7e363d3 +195: pshufb %xmm5,%xmm0 0xb7e363d8 +200: movdqa 0x40(%ebx),%xmm4 0xb7e363dd +205: pshufb %xmm2,%xmm4 ---Type return to continue, or q return to quit--- 0xb7e363e2 +210: pxor %xmm0,%xmm4 0xb7e363e6 +214: movdqa 0x50(%ebx),%xmm0 0xb7e363eb +219: pshufb %xmm3,%xmm0 0xb7e363f0 +224: pxor %xmm4,%xmm0 0xb7e363f4 +228: palignr $0xc,%xmm5,%xmm5 0xb7e363fa +234: movdqa %xmm6,%xmm1 0xb7e363fe +238: pandn %xmm0,%xmm1 0xb7e36402 +242: psrld $0x4,%xmm1 0xb7e36407 +247: pand %xmm6,%xmm0 0xb7e3640b +251: movdqa -0x20(%ebp),%xmm2 0xb7e36410 +256: pshufb %xmm0,%xmm2 0xb7e36415 +261: pxor %xmm1,%xmm0 0xb7e36419 +265: movdqa %xmm7,%xmm3 0xb7e3641d +269: pshufb %xmm1,%xmm3 0xb7e36422 +274: pxor %xmm2,%xmm3 0xb7e36426 +278: movdqa %xmm7,%xmm4 0xb7e3642a +282: pshufb %xmm0,%xmm4 0xb7e3642f +287: pxor %xmm2,%xmm4 0xb7e36433 +291: movdqa %xmm7,%xmm2 0xb7e36437 +295: pshufb %xmm3,%xmm2 0xb7e3643c +300: pxor %xmm0,%xmm2 0xb7e36440 +304: movdqa %xmm7,%xmm3 0xb7e36444 +308: pshufb %xmm4,%xmm3 0xb7e36449 +313: pxor %xmm1,%xmm3 ---Type return to continue, or q return to quit--- = 0xb7e3644d +317:movdqu (%edx),%xmm0 0xb7e36451 +321: jne0xb7e36370 _vpaes_decrypt_core+96 0xb7e36457 +327: movdqa 0x60(%ebx),%xmm4 0xb7e3645c +332: pshufb %xmm2,%xmm4 0xb7e36461 +337: pxor %xmm0,%xmm4 0xb7e36465 +341: movdqa 0x70(%ebx),%xmm0 0xb7e3646a +346: movdqa (%ecx),%xmm2 0xb7e3646e +350: pshufb %xmm3,%xmm0 0xb7e36473 +355: pxor %xmm4,%xmm0 0xb7e36477 +359: pshufb %xmm2,%xmm0 0xb7e3647c +364: ret End of assembler dump. (gdb) info reg eax0x277a77b5 662337461 ecx0xb7e35fa0 -1209835616 edx0x80090ff8 -2146889736 ebx0xb7e360d0 -1209835312 esp0xbfffb05c 0xbfffb05c ebp0xb7e35e70 0xb7e35e70 esi0x80081bd8 -2146952232 edi0xeba0 -5216 eip0xb7e3644d 0xb7e3644d _vpaes_decrypt_core+317 eflags 0x10202 [ IF RF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 Sincerely, Michael Russo, Systems Engineer PaperSolve, Inc. 268 Watchogue Road Staten Island, NY 10314 I too am experiencing a segfault in
[openssl.org #2775] Segmentation fault libcrypto.so.1.0.0
Damn, I knew I should have taken that assembly language course all those years ago. And yes, it does appear that it's only old versions of SSH that I'm having a problem connecting to (eg OpenSSH_3.6.1p2 w/ OpenSSL 0.9.7f, another host running 4.3p2 and 0.9.8e is fine). Well I set the breakpoint and I can continue on, so it is not crashing at the first breakpoint, and I get the same value in edx until I'm asked to enter my passcode (this server and the other one running 3.6.1p2 are running the RSA PAM modules, which I really hope aren't related): (gdb) b _vpaes_decrypt_core Function _vpaes_decrypt_core not defined. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (_vpaes_decrypt_core) pending. (gdb) run mi...@smtp.readq.com Starting program: /usr/bin/ssh mi...@smtp.readq.com [Thread debugging using libthread_db enabled] Breakpoint 1, _vpaes_decrypt_core () at vpaes-x86.s:148 148vpaes-x86.s: No such file or directory. (gdb) info reg eax0x0 0 ecx0x0 0 edx0x80084ae8 -2146940184 ebx0xbfffb0cc -1073762100 esp0xbfffb08c 0xbfffb08c ebp0xb7e35e70 0xb7e35e70 esi0x80080768 -2146957464 edi0x0 0 eip0xb7e36310 0xb7e36310 _vpaes_decrypt_core eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 (gdb) continue Continuing. Breakpoint 1, _vpaes_decrypt_core () at vpaes-x86.s:148 148in vpaes-x86.s (gdb) info reg eax0x0 0 ecx0x0 0 edx0x80084ae8 -2146940184 ebx0xbfffb0cc -1073762100 esp0xbfffb08c 0xbfffb08c ebp0xb7e35e70 0xb7e35e70 esi0x80080778 -2146957448 edi0x0 0 eip0xb7e36310 0xb7e36310 _vpaes_decrypt_core eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 (gdb) continue Continuing. edx0x80084ae8 -2146940184 Continuing. edx0x80084ae8 -2146940184 Continuing. edx0x80084ae8 -2146940184 Continuing. edx0x80084ae8 -2146940184 Continuing. edx0x80084ae8 -2146940184 Continuing. edx0x80084ae8 -2146940184 Continuing. Enter PASSCODE: I snipped a bunch of continue and info reg's up there to show you the value in edx is not changing. Breakpoint 1, _vpaes_decrypt_core () at vpaes-x86.s:148 148in vpaes-x86.s (gdb) info reg eax0x0 0 ecx0x0 0 edx0x80084ae8 -2146940184 ebx0xbfffb0ac -1073762132 esp0xbfffb06c 0xbfffb06c ebp0xb7e35e70 0xb7e35e70 esi0x80080768 -2146957464 edi0x0 0 eip0xb7e36310 0xb7e36310 _vpaes_decrypt_core eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 (gdb) continue Continuing. I set the watchpoint: (gdb) watch *((int *)(-2146940184+240)) Hardware watchpoint 2: *((int *)(-2146940184+240)) But after about 15 'continue's I type 'help continue' and realize I can give it a number. Some number of 'continue's later: (gdb) continue 100 Will ignore next 99 crossings of breakpoint 1. Continuing. Hardware watchpoint 2: *((int *)(-2146940184+240)) Old value = 9 New value = 915002721 vpaes_cbc_encrypt () at vpaes-x86.s:647 647in vpaes-x86.s (gdb) where #0 vpaes_cbc_encrypt () at vpaes-x86.s:647 #1 0x68732d63 in ?? () Guess I've got to see if I can figure out if it's an incompatibility with the old version of SSH or the equally ancient RSA PAM modules on those 2 servers where I'm having the trouble. Sincerely, Michael Russo, Systems Engineer PaperSolve, Inc. 268 Watchogue Road Staten Island, NY 10314 Damn, I knew I should have taken that assembly language course all those years ago. And yes, it does appear that its only old versions of SSH that Im having a problem connecting to (eg OpenSSH_3.6.1p2 w/ OpenSSL 0.9.7f, another host running 4.3p2 and 0.9.8e is fine). Well I set the breakpoint and I can continue on, so it is not crashing at the first breakpoint, and I get the same value in edx until Im asked to enter my passcode (this server and the other one running 3.6.1p2 are running the RSA PAM modules, which I really hope arent related): (gdb) b _vpaes_decrypt_core Function _vpaes_decrypt_core not defined. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (_vpaes_decrypt_core) pending. (gdb) run mi...@smtp.readq.com Starting program: /usr/bin/ssh mi...@smtp.readq.com [Thread debugging using
[openssl.org #2775]
Well I executed this right after the 'where' from last time (still had it up in a window though the connection has long since timed out): (gdb) info reg eax0x0 0 ecx0xb7e35f90 -1209835632 edx0x80084ae8 -2146940184 ebx0x3018 12312 esp0xbfffb0700xbfffb070 ebp0xb7e35e70 0xb7e35e70 esi0x80081bb8 -2146952264 edi0xebb0 -5200 eip0xb7e369fd0xb7e369fd vpaes_cbc_encrypt+189 eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 (gdb) disassemble Dump of assembler code for function vpaes_cbc_encrypt: 0xb7e36940 +0: push %ebp 0xb7e36941 +1: push %ebx 0xb7e36942 +2: push %esi 0xb7e36943 +3: push %edi 0xb7e36944 +4: mov0x14(%esp),%esi 0xb7e36948 +8: mov0x18(%esp),%edi 0xb7e3694c +12: mov0x1c(%esp),%eax 0xb7e36950 +16: mov0x20(%esp),%edx 0xb7e36954 +20: lea-0x38(%esp),%ebx 0xb7e36958 +24: mov0x24(%esp),%ebp 0xb7e3695c +28: and$0xfff0,%ebx 0xb7e3695f +31:mov0x28(%esp),%ecx 0xb7e36963 +35: xchg %esp,%ebx 0xb7e36965 +37: movdqu 0x0(%ebp),%xmm1 0xb7e3696a +42: sub%esi,%edi 0xb7e3696c +44: mov%ebx,0x30(%esp) 0xb7e36970 +48: mov%edi,(%esp) 0xb7e36973 +51: sub$0x10,%eax 0xb7e36976 +54: mov%edx,0x4(%esp) 0xb7e3697a +58: mov%ebp,0x8(%esp) 0xb7e3697e +62: mov%eax,%edi 0xb7e36980 +64: lea0xf4e5,%ebp 0xb7e36986 +70: call 0xb7e361c0 _vpaes_preheat ---Type return to continue, or q return to quit--- 0xb7e3698b +75: cmp$0x0,%ecx 0xb7e3698e +78: je 0xb7e369d0 vpaes_cbc_encrypt+144 0xb7e36990 +80: jmp0xb7e369a0 vpaes_cbc_encrypt+96 0xb7e36992 +82: lea0x0(%esi,%eiz,1),%esi 0xb7e36999 +89: lea0x0(%edi,%eiz,1),%edi 0xb7e369a0 +96: movdqu (%esi),%xmm0 0xb7e369a4 +100: pxor %xmm1,%xmm0 0xb7e369a8 +104: call 0xb7e361d0 _vpaes_encrypt_core 0xb7e369ad +109: mov(%esp),%ebx 0xb7e369b0 +112: mov0x4(%esp),%edx 0xb7e369b4 +116: movdqa %xmm0,%xmm1 0xb7e369b8 +120: movdqu %xmm0,(%ebx,%esi,1) 0xb7e369bd +125: lea0x10(%esi),%esi 0xb7e369c0 +128: sub$0x10,%edi 0xb7e369c3 +131: jae0xb7e369a0 vpaes_cbc_encrypt+96 0xb7e369c5 +133: jmp0xb7e36a05 vpaes_cbc_encrypt+197 0xb7e369c7 +135: mov%esi,%esi 0xb7e369c9 +137: lea0x0(%edi,%eiz,1),%edi 0xb7e369d0 +144: movdqu (%esi),%xmm0 0xb7e369d4 +148: movdqa %xmm1,0x10(%esp) 0xb7e369da +154: movdqa %xmm0,0x20(%esp) 0xb7e369e0 +160: call 0xb7e36310 _vpaes_decrypt_core 0xb7e369e5 +165: mov(%esp),%ebx 0xb7e369e8 +168: mov0x4(%esp),%edx ---Type return to continue, or q return to quit--- 0xb7e369ec +172: pxor 0x10(%esp),%xmm0 0xb7e369f2 +178: movdqa 0x20(%esp),%xmm1 0xb7e369f8 +184: movdqu %xmm0,(%ebx,%esi,1) = 0xb7e369fd +189: lea0x10(%esi),%esi 0xb7e36a00 +192: sub$0x10,%edi 0xb7e36a03 +195: jae0xb7e369d0 vpaes_cbc_encrypt+144 0xb7e36a05 +197: mov0x8(%esp),%ebx 0xb7e36a09 +201: mov0x30(%esp),%esp 0xb7e36a0d +205: movdqu %xmm1,(%ebx) 0xb7e36a11 +209: pop%edi 0xb7e36a12 +210: pop%esi 0xb7e36a13 +211: pop%ebx 0xb7e36a14 +212: pop%ebp 0xb7e36a15 +213: ret End of assembler dump. (gdb) info reg eax0x0 0 ecx0xb7e35f90 -1209835632 edx0x80084ae8 -2146940184 ebx0x3018 12312 esp0xbfffb0700xbfffb070 ebp0xb7e35e70 0xb7e35e70 esi0x80081bb8 -2146952264 edi0xebb0 -5200 eip0xb7e369fd0xb7e369fd vpaes_cbc_encrypt+189 eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 Sincerely, Michael Russo, Systems Engineer PaperSolve, Inc. 268 Watchogue Road Staten Island, NY 10314 Well I executed this right after the where from last time (still had it up in a window though the connection has long since timed out): (gdb) info reg eax 0x0 0 ecx 0xb7e35f90 -1209835632 edx 0x80084ae8 -2146940184 ebx 0x3018 12312 esp 0xbfffb070 0xbfffb070 ebp 0xb7e35e70 0xb7e35e70 esi 0x80081bb8 -2146952264 edi 0xebb0 -5200 eip 0xb7e369fd 0xb7e369fd vpaes_cbc_encrypt189 eflags 0x246 [ PF ZF IF ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 (gdb) disassemble Dump of assembler code for function vpaes_cbc_encrypt: 0xb7e36940 0: push %ebp 0xb7e36941 1: push %ebx 0xb7e36942 2: push %esi 0xb7e36943 3: push %edi 0xb7e36944 4: mov 0x14(%esp),%esi 0xb7e36948 8: mov 0x18(%esp),%edi 0xb7e3694c 12: mov 0x1c(%esp),%eax
