[openssl.org #2775] Segmentation fault libcrypto.so.1.0.0

2012-03-30 Thread Mike Russo via RT
Damn, I knew I should have taken that assembly language course all those years 
ago.  And yes, it does appear that it's only old versions of SSH that I'm 
having a problem connecting to (eg OpenSSH_3.6.1p2 w/ OpenSSL 0.9.7f, another 
host running 4.3p2 and 0.9.8e is fine).

Well I set the breakpoint and I can continue on, so it is not crashing at the 
first breakpoint, and I get the same value in edx until I'm asked to enter my 
passcode (this server and the other one running 3.6.1p2 are running the RSA PAM 
modules, which I really hope aren't related):

(gdb) b _vpaes_decrypt_core
Function _vpaes_decrypt_core not defined.
Make breakpoint pending on future shared library load? (y or [n]) y

Breakpoint 1 (_vpaes_decrypt_core) pending.
(gdb) run mi...@smtp.readq.com
Starting program: /usr/bin/ssh mi...@smtp.readq.com
[Thread debugging using libthread_db enabled]

Breakpoint 1, _vpaes_decrypt_core () at vpaes-x86.s:148
148vpaes-x86.s: No such file or directory.
(gdb) info reg
eax0x0  0
ecx0x0  0
edx0x80084ae8  -2146940184
ebx0xbfffb0cc -1073762100
esp0xbfffb08c 0xbfffb08c
ebp0xb7e35e70  0xb7e35e70
esi0x80080768   -2146957464
edi0x0   0
eip0xb7e36310   0xb7e36310 _vpaes_decrypt_core
eflags 0x246 [ PF ZF IF ]
cs 0x73 115
ss 0x7b 123
ds 0x7b 123
es 0x7b 123
fs 0x0   0
gs 0x33 51
(gdb) continue
Continuing.

Breakpoint 1, _vpaes_decrypt_core () at vpaes-x86.s:148
148in vpaes-x86.s
(gdb) info reg
eax0x0  0
ecx0x0  0
edx0x80084ae8  -2146940184
ebx0xbfffb0cc -1073762100
esp0xbfffb08c 0xbfffb08c
ebp0xb7e35e70  0xb7e35e70
esi0x80080778   -2146957448
edi0x0   0
eip0xb7e36310   0xb7e36310 _vpaes_decrypt_core
eflags 0x246 [ PF ZF IF ]
cs 0x73 115
ss 0x7b 123
ds 0x7b 123
es 0x7b 123
fs 0x0   0
gs 0x33 51
(gdb) continue
Continuing.
edx0x80084ae8  -2146940184
Continuing.
edx0x80084ae8  -2146940184
Continuing.
edx0x80084ae8  -2146940184
Continuing.
edx0x80084ae8  -2146940184
Continuing.
edx0x80084ae8  -2146940184
Continuing.
edx0x80084ae8  -2146940184
Continuing.
Enter PASSCODE:

I snipped a bunch of continue and info reg's up there to show you the value in 
edx is not changing.


Breakpoint 1, _vpaes_decrypt_core () at vpaes-x86.s:148
148in vpaes-x86.s
(gdb) info reg
eax0x0  0
ecx0x0  0
edx0x80084ae8  -2146940184
ebx0xbfffb0ac -1073762132
esp0xbfffb06c 0xbfffb06c
ebp0xb7e35e70  0xb7e35e70
esi0x80080768   -2146957464
edi0x0   0
eip0xb7e36310   0xb7e36310 _vpaes_decrypt_core
eflags 0x246 [ PF ZF IF ]
cs 0x73 115
ss 0x7b 123
ds 0x7b 123
es 0x7b 123
fs 0x0   0
gs 0x33 51
(gdb) continue
Continuing.


I set the watchpoint:
(gdb) watch *((int *)(-2146940184+240))
Hardware watchpoint 2: *((int *)(-2146940184+240))


But after about 15 'continue's I type 'help continue' and realize I can give it 
a number.  Some number of 'continue's later:

(gdb) continue 100
Will ignore next 99 crossings of breakpoint 1.  Continuing.
Hardware watchpoint 2: *((int *)(-2146940184+240))

Old value = 9
New value = 915002721
vpaes_cbc_encrypt () at vpaes-x86.s:647
647in vpaes-x86.s
(gdb) where
#0  vpaes_cbc_encrypt () at vpaes-x86.s:647
#1  0x68732d63 in ?? ()


Guess I've got to see if I can figure out if it's an incompatibility with the 
old version of SSH or the equally ancient RSA PAM modules on those 2 servers 
where I'm having the trouble.


Sincerely,

Michael Russo, Systems Engineer
PaperSolve, Inc.
268 Watchogue Road
Staten Island, NY 10314






Damn, I knew I should have taken that assembly language course all those years ago. And yes, it does appear that its only old versions of SSH that Im having a problem
 connecting to (eg OpenSSH_3.6.1p2 w/ OpenSSL 0.9.7f, another host running 4.3p2 and 0.9.8e is fine).


Well I set the breakpoint and I can continue on, so it is not crashing at the first breakpoint, and I get the same value in edx until Im asked to enter my passcode (this
 server and the other one running 3.6.1p2 are running the RSA PAM modules, which I really hope arent related):

(gdb) b _vpaes_decrypt_core
Function _vpaes_decrypt_core not defined.
Make breakpoint pending on future shared library load? (y or [n]) y

Breakpoint 1 (_vpaes_decrypt_core) pending.
(gdb) run mi...@smtp.readq.com
Starting program: /usr/bin/ssh mi...@smtp.readq.com
[Thread debugging using 

Re: [openssl.org #2775] Segmentation fault libcrypto.so.1.0.0

2012-03-30 Thread Andy Polyakov via RT
 (gdb) continue 100
 Will ignore next 99 crossings of breakpoint 1.  Continuing.
 Hardware watchpoint 2: *((int *)(-2146940184+240))
 
 Old value = 9
 New value = 915002721
 vpaes_cbc_encrypt () at vpaes-x86.s:647
 647in vpaes-x86.s
 (gdb) where
 #0  vpaes_cbc_encrypt () at vpaes-x86.s:647
 #1  0x68732d63 in ?? ()

'disassemble' [till you see offending instruction] and 'info reg' at
this point, please.


__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org