[openssl.org #2775] Additional information for openssh segfault

[Mike Russo via RT]

I too am experiencing a segfault in openssh when connecting to a particular 
Linux host. This did work fine with the previous version of libssl (I'm running 
Ubuntu 12.04 so this was 1.0.0 before upgrading to 1.0.1). No core file is 
created but I can do the gdb backtrace and disassemble.

You've correctly guessed the failing instruction :) but not sure why it's 
happening.

Program received signal SIGSEGV, Segmentation fault.
_vpaes_decrypt_core () at vpaes-x86.s:221
221vpaes-x86.s: No such file or directory.
(gdb) bt
#0  _vpaes_decrypt_core () at vpaes-x86.s:221
#1  0xb7e369e5 in vpaes_cbc_encrypt () at vpaes-x86.s:641
#2  0x732d6361 in ?? ()
(gdb) disassemble
Dump of assembler code for function _vpaes_decrypt_core:
   0xb7e36310 +0: mov0xf0(%edx),%eax
   0xb7e36316 +6: lea0x260(%ebp),%ebx
   0xb7e3631c +12:   movdqa %xmm6,%xmm1
   0xb7e36320 +16:   movdqa -0x40(%ebx),%xmm2
   0xb7e36325 +21:   pandn  %xmm0,%xmm1
   0xb7e36329 +25:   mov%eax,%ecx
   0xb7e3632b +27:   psrld  $0x4,%xmm1
   0xb7e36330 +32:   movdqu (%edx),%xmm5
   0xb7e36334 +36:   shl$0x4,%ecx
   0xb7e36337 +39:   pand   %xmm6,%xmm0
   0xb7e3633b +43:   pshufb %xmm0,%xmm2
   0xb7e36340 +48:   movdqa -0x30(%ebx),%xmm0
   0xb7e36345 +53:   xor$0x30,%ecx
   0xb7e36348 +56:   pshufb %xmm1,%xmm0
   0xb7e3634d +61:   and$0x30,%ecx
   0xb7e36350 +64:   pxor   %xmm5,%xmm2
   0xb7e36354 +68:   movdqa 0xb0(%ebp),%xmm5
   0xb7e3635c +76:   pxor   %xmm2,%xmm0
   0xb7e36360 +80:   add$0x10,%edx
   0xb7e36363 +83:   lea-0x160(%ebx,%ecx,1),%ecx
   0xb7e3636a +90:   jmp0xb7e363fa _vpaes_decrypt_core+234
   0xb7e3636f +95:nop
   0xb7e36370 +96:   movdqa -0x20(%ebx),%xmm4
---Type return to continue, or q return to quit---
   0xb7e36375 +101: pshufb %xmm2,%xmm4
   0xb7e3637a +106: pxor   %xmm0,%xmm4
   0xb7e3637e +110: movdqa -0x10(%ebx),%xmm0
   0xb7e36383 +115: pshufb %xmm3,%xmm0
   0xb7e36388 +120: pxor   %xmm4,%xmm0
   0xb7e3638c +124: add$0x10,%edx
   0xb7e3638f +127:  pshufb %xmm5,%xmm0
   0xb7e36394 +132: movdqa (%ebx),%xmm4
   0xb7e36398 +136: pshufb %xmm2,%xmm4
   0xb7e3639d +141: pxor   %xmm0,%xmm4
   0xb7e363a1 +145: movdqa 0x10(%ebx),%xmm0
   0xb7e363a6 +150: pshufb %xmm3,%xmm0
   0xb7e363ab +155:  pxor   %xmm4,%xmm0
   0xb7e363af +159:  sub$0x1,%eax
   0xb7e363b2 +162: pshufb %xmm5,%xmm0
   0xb7e363b7 +167: movdqa 0x20(%ebx),%xmm4
   0xb7e363bc +172:  pshufb %xmm2,%xmm4
   0xb7e363c1 +177: pxor   %xmm0,%xmm4
   0xb7e363c5 +181: movdqa 0x30(%ebx),%xmm0
   0xb7e363ca +186:  pshufb %xmm3,%xmm0
   0xb7e363cf +191:  pxor   %xmm4,%xmm0
   0xb7e363d3 +195: pshufb %xmm5,%xmm0
   0xb7e363d8 +200: movdqa 0x40(%ebx),%xmm4
   0xb7e363dd +205: pshufb %xmm2,%xmm4
---Type return to continue, or q return to quit---
   0xb7e363e2 +210: pxor   %xmm0,%xmm4
   0xb7e363e6 +214: movdqa 0x50(%ebx),%xmm0
   0xb7e363eb +219:  pshufb %xmm3,%xmm0
   0xb7e363f0 +224:  pxor   %xmm4,%xmm0
   0xb7e363f4 +228:  palignr $0xc,%xmm5,%xmm5
   0xb7e363fa +234:  movdqa %xmm6,%xmm1
   0xb7e363fe +238:  pandn  %xmm0,%xmm1
   0xb7e36402 +242: psrld  $0x4,%xmm1
   0xb7e36407 +247: pand   %xmm6,%xmm0
   0xb7e3640b +251: movdqa -0x20(%ebp),%xmm2
   0xb7e36410 +256: pshufb %xmm0,%xmm2
   0xb7e36415 +261: pxor   %xmm1,%xmm0
   0xb7e36419 +265: movdqa %xmm7,%xmm3
   0xb7e3641d +269: pshufb %xmm1,%xmm3
   0xb7e36422 +274: pxor   %xmm2,%xmm3
   0xb7e36426 +278: movdqa %xmm7,%xmm4
   0xb7e3642a +282: pshufb %xmm0,%xmm4
   0xb7e3642f +287:  pxor   %xmm2,%xmm4
   0xb7e36433 +291: movdqa %xmm7,%xmm2
   0xb7e36437 +295: pshufb %xmm3,%xmm2
   0xb7e3643c +300: pxor   %xmm0,%xmm2
   0xb7e36440 +304: movdqa %xmm7,%xmm3
   0xb7e36444 +308: pshufb %xmm4,%xmm3
   0xb7e36449 +313: pxor   %xmm1,%xmm3
---Type return to continue, or q return to quit---
= 0xb7e3644d +317:movdqu (%edx),%xmm0
   0xb7e36451 +321: jne0xb7e36370 _vpaes_decrypt_core+96
   0xb7e36457 +327: movdqa 0x60(%ebx),%xmm4
   0xb7e3645c +332: pshufb %xmm2,%xmm4
   0xb7e36461 +337: pxor   %xmm0,%xmm4
   0xb7e36465 +341: movdqa 0x70(%ebx),%xmm0
   0xb7e3646a +346: movdqa (%ecx),%xmm2
   0xb7e3646e +350: pshufb %xmm3,%xmm0
   0xb7e36473 +355: pxor   %xmm4,%xmm0
   0xb7e36477 +359: pshufb %xmm2,%xmm0
   0xb7e3647c +364: ret
End of assembler dump.
(gdb) info reg
eax0x277a77b5  662337461
ecx0xb7e35fa0   -1209835616
edx0x80090ff8   -2146889736
ebx0xb7e360d0  -1209835312
esp0xbfffb05c 0xbfffb05c
ebp0xb7e35e70  0xb7e35e70
esi0x80081bd8   -2146952232
edi0xeba0   -5216
eip0xb7e3644d   0xb7e3644d _vpaes_decrypt_core+317
eflags 0x10202   [ IF RF ]
cs 0x73 115
ss 0x7b 123
ds 0x7b 123
es 0x7b 123
fs 0x0   0
gs 0x33 51

Sincerely,

Michael Russo, Systems Engineer
PaperSolve, Inc.
268 Watchogue Road
Staten Island, NY 10314






I too am experiencing a segfault in 

Re: [openssl.org #2775] Additional information for openssh segfault

[Andy Polyakov via RT]

 I too am experiencing a segfault in openssh when connecting to a
 particular Linux host.

So you mean you can login to say 2-3-4-5 other hosts, but not one
specific. Tough break...

 This did work fine with the previous version
 of libssl (I'm running Ubuntu 12.04 so this was 1.0.0 before
 upgrading to 1.0.1). No core file is created but I can do the gdb
 backtrace and disassemble.

Presumably you have to 'ulimit -c unlimited' (if you run sh) in order to
get the core. But running under debugger is as good and the only way to
do that I'm about to suggest...

 You've correctly guessed the failing instruction :) but not sure why
 it's happening.
 
 Program received signal SIGSEGV, Segmentation fault.
 _vpaes_decrypt_core () at vpaes-x86.s:221
 (gdb) disassemble
 Dump of assembler code for function _vpaes_decrypt_core:
0xb7e36310 +0: mov0xf0(%edx),%eax
...
0xb7e363af +159:  sub$0x1,%eax
...
 = 0xb7e3644d +317:movdqu (%edx),%xmm0
0xb7e36451 +321: jne0xb7e36370 _vpaes_decrypt_core+96
...
 End of assembler dump.
 (gdb) info reg
 eax0x277a77b5  662337461
 ...
 edx0x80090ff8   -2146889736

As you can see 'eax' is crazy, 'edx' lies on page boundary and next page
ought to be unaccessible causing SEGV. Now under gdb. First see if it
fails upon first call. I.e. set breakpoint to _vpaes_decrypt_core, 'b
_vpaes_decrypt_core', start program, when it hits the breakpoint, write
down value in 'edx' register and issue 'continue'. Does it SEGV or stops
at break point again? If latter, count 'continue's writing down values
in 'edx' register. Is it same 'edx'? Best would be if it's same 'edx'
and it's not first call... Once you know which invocation fails, restart
program with same breakpoint, at first invocation check 'edx' and set
watch point to 0xf0(%edx), 'watch *((int *)(value-in-edx+240)),
'continue'. Eventually it should break at watch point. 'where'?

If already first call to _vpaes_decrypt_core SEGVs, then you would have
to set breakpoint to vpaes_set_decrypt_key, find out pointer to key
schedule, set watchpoint as above and see where it breaks at watchpoint.
 In order to find pointer to key schedule, issue 'stepi' 9 times after
it breaks at vpaes_set_decrypt_key and examine 'edx'. Naturally
vpaes_set_decrypt_key will hit the watch point, you should 'continue' to
find out who *else* is modifying it. Also double-check that it's same
value in 'edx' about entry to _vpaes_decrypt_core. Do you get the drift?


__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org