Bug#412886: RFH gnutls related crash of exim4 on x86_64 (Bug#412886)

[Simon Josefsson]

Ronny Adsetts <[EMAIL PROTECTED]> writes:

> I have tcpdumps from a couple of earlier crashing sessions. They're attached 
> to the Cc'd Debian bug:
>
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=412886;msg=15;filename=20070228_190841_crasher.pcap;att=1
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=412886;msg=20;filename=20070228_200842_crasher.pcap;att=1
>
> Hopefully that'll be helpful.

The tcpdump's are only in one direction, and ethereal can't decode it as
a SSL stream then...  It would be useful to capture both what is sent
and what is received.  Now it appears to be only what the other end is
sending.

> NB. My posts are being rejected from [EMAIL PROTECTED] so I assume the list 
> is subscriber only

I have added you to the whitelist.

/Simon


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#412886: RFH gnutls related crash of exim4 on x86_64 (Bug#412886)

[Ronny Adsetts]

Simon Josefsson said at 15/05/2007 21:53:
> On 15 maj 2007, at 22.24, Ronny Adsetts wrote:
>> Simon Josefsson said at 15/05/2007 20:53:
>> [snip lots of helpful debug data]

 Please let me know if you want any more information.
>>>
>>> If you can reproduce the crash in gdb, try getting a backtrace using 'bt
>>> full', and invoke 'up' until you are in the _gnutls_recv_handshake stack
>>> frame, then 'p recv_type'.  If it is indeed
>>> GNUTLS_HANDSHAKE_CLIENT_HELLO or GNUTLS_HANDSHAKE_SERVER_HELLO then I
>>> think we have diagnosed the cause.
>> [...]
>>
>> Simon, thanks for your help so far.
>>
>> Assuming the delivery starts again, it stopped this morning, it'll
>> keep trying every hour for about three days...
>>
>> Unless you know if it's possible to take a tcpdump and recreate the
>> transaction?
> 
> Yes, do try to get a tcpdump trace of the session.  It may not be
> possible to recreate the transaction with live code, but at least we can
> look at what the other end is sending, and especially if it is sending a
> client hello again when GnuTLS expects something else.

I have tcpdumps from a couple of earlier crashing sessions. They're attached to 
the Cc'd Debian bug:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=412886;msg=15;filename=20070228_190841_crasher.pcap;att=1
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=412886;msg=20;filename=20070228_200842_crasher.pcap;att=1

Hopefully that'll be helpful.

NB. My posts are being rejected from [EMAIL PROTECTED] so I assume the list is 
subscriber only

Ronny
-- 
Ronny Adsetts
Technical Director
Amazing Internet Ltd, London
t: +44 20 8607 9535
f: +44 20 8607 9536
w: www.amazinginternet.com

Registered office: UK House, 82 Heath Road, Twickenham TW1 4BW
Registered in England. Company No. 4042957 




signature.asc
Description: OpenPGP digital signature

Bug#412886: RFH gnutls related crash of exim4 on x86_64 (Bug#412886)

[Simon Josefsson]

On 15 maj 2007, at 22.24, Ronny Adsetts wrote:


Simon Josefsson said at 15/05/2007 20:53:
[snip lots of helpful debug data]


Please let me know if you want any more information.


If you can reproduce the crash in gdb, try getting a backtrace  
using 'bt
full', and invoke 'up' until you are in the _gnutls_recv_handshake  
stack

frame, then 'p recv_type'.  If it is indeed
GNUTLS_HANDSHAKE_CLIENT_HELLO or GNUTLS_HANDSHAKE_SERVER_HELLO then I
think we have diagnosed the cause.

[...]

Simon, thanks for your help so far.

Assuming the delivery starts again, it stopped this morning, it'll  
keep trying every hour for about three days...


Unless you know if it's possible to take a tcpdump and recreate the  
transaction?


Yes, do try to get a tcpdump trace of the session.  It may not be  
possible to recreate the transaction with live code, but at least we  
can look at what the other end is sending, and especially if it is  
sending a client hello again when GnuTLS expects something else.


Thanks,
Simon


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#412886: RFH gnutls related crash of exim4 on x86_64 (Bug#412886)

[Ronny Adsetts]

Simon Josefsson said at 15/05/2007 20:53:
[snip lots of helpful debug data]
>>
>> Please let me know if you want any more information.
> 
> If you can reproduce the crash in gdb, try getting a backtrace using 'bt
> full', and invoke 'up' until you are in the _gnutls_recv_handshake stack
> frame, then 'p recv_type'.  If it is indeed
> GNUTLS_HANDSHAKE_CLIENT_HELLO or GNUTLS_HANDSHAKE_SERVER_HELLO then I
> think we have diagnosed the cause.
[...]

Simon, thanks for your help so far.

Assuming the delivery starts again, it stopped this morning, it'll keep trying 
every hour for about three days...

Unless you know if it's possible to take a tcpdump and recreate the transaction?

> Any chance to contact the remote host to find out what software it is
> using?

I tried but didn't get very far. Maybe I'll give it another go.

If I manage to get any more data as outlined above, I'll reply to the thread.

Thanks again.

Ronny
-- 
Ronny Adsetts
Technical Director
Amazing Internet Ltd, London
t: +44 20 8607 9535
f: +44 20 8607 9536
w: www.amazinginternet.com

Registered office: UK House, 82 Heath Road, Twickenham TW1 4BW
Registered in England. Company No. 4042957 




signature.asc
Description: OpenPGP digital signature

Bug#412886: RFH gnutls related crash of exim4 on x86_64 (Bug#412886)

[Simon Josefsson]

Andreas Metzler <[EMAIL PROTECTED]> writes:

> #0  _gnutls_read_uint16 (data=0x0) at gnutls_num.c:120
> 120 gnutls_num.c: No such file or directory.
> in gnutls_num.c
> (gdb) bt
> #0  _gnutls_read_uint16 (data=0x0) at gnutls_num.c:120
> #1  0x2ba3e841bfc9 in _gnutls_proc_rsa_client_kx (session=0x629e10,
> data=0x0, _data_size=61) at auth_rsa.c:213
  ^^^

Interesting, data is NULL here, as invoked from this function:

> #2  0x2ba3e84171e9 in _gnutls_recv_client_kx_message (session=0x629e10)
> at gnutls_kx.c:333

The function reads:

  ret =
_gnutls_recv_handshake (session, &data,
&datasize,
GNUTLS_HANDSHAKE_CLIENT_KEY_EXCHANGE,
MANDATORY_PACKET);
  if (ret < 0)
return ret;

  ret =
session->internals.auth_struct->
gnutls_process_client_kx (session, data, datasize);
  gnutls_free (data);
  if (ret < 0)
return ret;

Reading the source for _gnutls_recv_handshake, it seems to be possible
for it to return >= 0 if the code received a
GNUTLS_HANDSHAKE_CLIENT_HELLO or GNUTLS_HANDSHAKE_SERVER_HELLO packet.
I'm not 100 % certain of this though.

> #3  0x2ba3e8412c72 in _gnutls_handshake_server (session=0x629e10)
> at gnutls_handshake.c:2259
> #4  0x2ba3e841236b in gnutls_handshake (session=0x629e10)
> at gnutls_handshake.c:1908
> #5  0x004604a7 in tls_server_start (require_ciphers=0x0)
> at tls-gnu.c:838
> #6  0x00459339 in smtp_setup_msg () at smtp_in.c:3212
> #7  0x00418fc3 in handle_smtp_call (listen_sockets=0x5e3cd0,
> listen_socket_count=6, accept_socket=0, accepted=0x0) at daemon.c:495
> #8  0x0041a55c in daemon_go () at daemon.c:1815
> #9  0x0042848b in main (argc=7, cargv=0x0) at exim.c:3922
> ---
>
> Please let me know if you want any more information.

If you can reproduce the crash in gdb, try getting a backtrace using 'bt
full', and invoke 'up' until you are in the _gnutls_recv_handshake stack
frame, then 'p recv_type'.  If it is indeed
GNUTLS_HANDSHAKE_CLIENT_HELLO or GNUTLS_HANDSHAKE_SERVER_HELLO then I
think we have diagnosed the cause.

It may be that the client got confused and started sending client
hello's all over, and this confused the GnuTLS state machine.  Of
course, it shouldn't be possible to crash it.

Debugging exactly why the other end sends these funny messages would be
the next step, I'm not sure that if we would fix the crash that the
hosts will be able to handshake properly.

Any chance to contact the remote host to find out what software it is
using?

/Simon


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#412886: RFH gnutls related crash of exim4 on x86_64 (Bug#412886)

[Andreas Metzler]

Hello,

Ronny Adsetts has been plagued by an exim4 crash in the gnutls code
when receiving mail from a specific server. - It seems like gnutls
does not like the client certificate and crashes. The complete bug
history is on , featuring strace output
and a tcpdump capture.

Ronny has been able to get the following backtrace, with the segfault
happening due to null-pointer dereferencing in _gnutls_read_uint16.

I do not know how debug this efficiently, the machine in question is
a production machine and the bug only occurs on specific third party
hosts connecting.

TIA for your help. This is gnutls 1.4.x, BTW)
cu andreas

- Forwarded message from Ronny Adsetts <[EMAIL PROTECTED]> -
Message-ID: <[EMAIL PROTECTED]>
Date: Sun, 13 May 2007 18:34:20 +0100
From: Ronny Adsetts <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED], Marc Haber <[EMAIL PROTECTED]>,
Andreas Metzler <[EMAIL PROTECTED]>

Hi Marc/Andreas,

I've finally managed to get a core file on this segfault:

---
$ sudo gdb /usr/sbin/exim4 core
GNU gdb 6.3-debian
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-linux"...Using host libthread_db library "/l.

Core was generated by `/usr/sbin/exim4 -bd -q30m -oX 587:465:25 -oP /var/run/ex.
Program terminated with signal 11, Segmentation fault.

[...]

#0  _gnutls_read_uint16 (data=0x0) at gnutls_num.c:120
120 gnutls_num.c: No such file or directory.
in gnutls_num.c
(gdb) bt
#0  _gnutls_read_uint16 (data=0x0) at gnutls_num.c:120
#1  0x2ba3e841bfc9 in _gnutls_proc_rsa_client_kx (session=0x629e10,
data=0x0, _data_size=61) at auth_rsa.c:213
#2  0x2ba3e84171e9 in _gnutls_recv_client_kx_message (session=0x629e10)
at gnutls_kx.c:333
#3  0x2ba3e8412c72 in _gnutls_handshake_server (session=0x629e10)
at gnutls_handshake.c:2259
#4  0x2ba3e841236b in gnutls_handshake (session=0x629e10)
at gnutls_handshake.c:1908
#5  0x004604a7 in tls_server_start (require_ciphers=0x0)
at tls-gnu.c:838
#6  0x00459339 in smtp_setup_msg () at smtp_in.c:3212
#7  0x00418fc3 in handle_smtp_call (listen_sockets=0x5e3cd0,
listen_socket_count=6, accept_socket=0, accepted=0x0) at daemon.c:495
#8  0x0041a55c in daemon_go () at daemon.c:1815
#9  0x0042848b in main (argc=7, cargv=0x0) at exim.c:3922
---

Please let me know if you want any more information.

Thanks.

Ronny
-- 
Ronny Adsetts
Technical Director
Amazing Internet Ltd, London
t: +44 20 8607 9535
f: +44 20 8607 9536
w: www.amazinginternet.com

Registered office: UK House, 82 Heath Road, Twickenham TW1 4BW
Registered in England. Company No. 4042957 
- End forwarded message -


signature.asc
Description: Digital signature