RE: ssl handshake failure in 1.0.1 but not 1.0.0
From: Dave Thompson Yes, the server has a custom root cert that isn't installed on this machine. I am happy that the server cert is correct. For testing that's okay, but I hope in real use you are verifying. Otherwise an active attacker may be able to MITM your connections. Production environments do a peer verification. I disabled that for development purposes. The ServerHello does indeed contain the secure-renegotiation extension in one pcap and not the other. Assuming there isn't some really weird logic on the server that supports 5746 only sometimes, this might be due to the (much) larger cipherlist -- OpenSSL puts ERI-SCSV at the end of the cipherlist, so if the server can only handle maybe 32 or 50 or so entries in the cipherlist it might not see ERI in the default-ciphers case. You could experiment with intermediate size cipherlists -- my suggestion of forcing -tls1 by itself takes you down from 80 to 52 (because it implicitly disables the TLSv1.2-only SHA2 and GCM suites), or so does explicit -cipher DEFAULT:!TLSv1.2 . Removing more things you shouldn't want anyway goes lower e.g. DEFAULT:!TLSv1.2:!EXPORT:!LOW:!SRP:!kECDH should be 30. [snip] If the problem is the length of the ClientHello and/or cipherlist -- as is consistent with but not conclusively proven by what you've seen so far, and is somewhat similar to the fact that other servers have already been found to fail or hang *initial* negotiation when ClientHello = 256 bytes (although this server did *not* fail there), just using a shorter cipherlist should work. A few akRSA, one or two DHE-RSA and ECDHE-RSA because a server with RSA can still do akRSA unless KU prohibits, a few ECDHE-ECDSA and perhaps a few DHE-DSS -- maybe 20 total -- should handle any sane server. That's great, thank you for the detailed explanations. Your hunch that the problem lies with the length of the cipherlist seems to bear out; I removed some of the ciphers you suggested and the server still happily connects. It creates a Client Hello of 198 bytes which should also avoid the other problem you mention (that I haven't seen on this particular server). Thanks for all the help, Ben __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: ssl handshake failure in 1.0.1 but not 1.0.0
From: owner-openssl-users On Behalf Of Ben Arnold Sent: Friday, November 08, 2013 10:45 snip I have tried using s_client and it fails with the same handshake failure. Please see below. Attaching a PCAP file of the traffic is much more useful than hex packet dumps. You're right of course, that is much more sensible. I have attached two pcap traces from s_connect, one success and one failure. From: Dave Thompson snip: server cert Yes, the server has a custom root cert that isn't installed on this machine. I am happy that the server cert is correct. For testing that's okay, but I hope in real use you are verifying. Otherwise an active attacker may be able to MITM your connections. To OP: If you can try to reproduce with s_client default (which is TLSv1.2 or less) and again specifying -tls1 (or -no_tls1_2 -no_tls1_1). That might narrow it down pretty close. I can reproduce the failure with s_client in the same manner. It looks (to me) like the server only asks for the client certificate once the GET command has been issued, the initial negotiation doesn't ask for it. I don't know the TLS protocol so that might be normal. Once I send the GET ... command it tries to Yes. More exactly, on the initial negotiation the server does not request client auth (and thus openssl doesn't obtain and send it). After curl or s_client/you sends the GET request, the server initiates renegotiation and does request client auth except in the case using 1.0.1 where it fails before getting to that point. Renegotiation is a standard capability of SSL/TLS and can be initiated by either client or server. Whether and when it is used depends on the applications using SSL/TLS. *For HTTPS*, it is not uncommon for webservers to allow connection without client auth that can access public resources but require renegotiation with client auth for private resources, and it certainly appears this particular webserver is doing that. renegotiate but fails. Looking at the output from s_client -state I see this during the first negotiation... --- No client certificate CA names sent --- SSL handshake has read 2884 bytes and written 639 bytes --- New, TLSv1/SSLv3, Cipher is DES-CBC3-SHA Server public key is 2048 bit Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE And then I send GET /directory HTTP/1.1 and see... SSL_connect:SSL renegotiate ciphers SSL_connect:unknown state SSL_connect:failed in unknown state 16444:error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure:.\ssl\s3_pkt.c:1156: From: Krzysztof Kwiatkowski Do you still see an error if you specify one cipher? f.e. AES256-SHA? I do get an error when using AES256-SHA, in fact a much earlier error as the server does not support that cipher, nor does it support AES-128-SHA. However I took the idea and if I add -cipher DES-CBC3-SHA (as selected by the server in the previous run) to s_connect then everything works OK, and if I add the same cipher selection to my program that that works too. When I do specify DES-CBC3-SHA, s_client also reports Secure Renegotiation IS supported This sounds important to me! :) Notice that the failure case reports renegotiation is NOT supported. To be exact it reports *secure* renegotiation is not supported. There are two slightly different renegotiation protocols, the original one (now usually called legacy or unsafe) which was found to be somewhat vulnerable to a MITM-splicing attack, and the updated RFC5746 one (sometimes called by the RFC#, but often just called secure). Client can detect from ServerHello whether the server supports RFC5746 (or at least claims to) and the display tells you that. The client can't determine if legacy renegotiation is supported except by trying it, and even then can't be 100% certain because it could fail for another reason. The ServerHello does indeed contain the secure-renegotiation extension in one pcap and not the other. Assuming there isn't some really weird logic on the server that supports 5746 only sometimes, this might be due to the (much) larger cipherlist -- OpenSSL puts ERI-SCSV at the end of the cipherlist, so if the server can only handle maybe 32 or 50 or so entries in the cipherlist it might not see ERI in the default-ciphers case. You could experiment with intermediate size cipherlists -- my suggestion of forcing -tls1 by itself takes you down from 80 to 52 (because it implicitly disables the TLSv1.2-only SHA2 and GCM suites), or so does explicit -cipher DEFAULT:!TLSv1.2 . Removing more things you shouldn't want anyway goes lower e.g. DEFAULT:!TLSv1.2:!EXPORT:!LOW:!SRP:!kECDH should be 30. Or you could try writing a Java SSL client, which allows you to position ERI-SCSV anywhere you want in the list (i.e. end of 80, end of 40, 40th of 80, etc, etc). The renegotiation ClientHello is longer than the initial one because of the session-id and
RE: ssl handshake failure in 1.0.1 but not 1.0.0
From: Viktor Dukhovni You can test with s_client(1) and compare results. Is your client certificate an RSA certificate? How many bits of public key? Is its signature SHA1 or SHA256? It's a 2048 bit RSA SHA1 certificate, but I think Dave Thompson's right and it's not getting that far. I have tried using s_client and it fails with the same handshake failure. Please see below. Attaching a PCAP file of the traffic is much more useful than hex packet dumps. You're right of course, that is much more sensible. I have attached two pcap traces from s_connect, one success and one failure. From: Dave Thompson To OP: is the logged server cert info true? I note the log shows the client verification of the server failed; did this website give you a custom root to trust and did you simply not set that up (or perhaps not in the environment you're testing in)? Yes, the server has a custom root cert that isn't installed on this machine. I am happy that the server cert is correct. To OP: If you can try to reproduce with s_client default (which is TLSv1.2 or less) and again specifying -tls1 (or -no_tls1_2 -no_tls1_1). That might narrow it down pretty close. I can reproduce the failure with s_client in the same manner. It looks (to me) like the server only asks for the client certificate once the GET command has been issued, the initial negotiation doesn't ask for it. I don't know the TLS protocol so that might be normal. Once I send the GET ... command it tries to renegotiate but fails. Looking at the output from s_client -state I see this during the first negotiation... --- No client certificate CA names sent --- SSL handshake has read 2884 bytes and written 639 bytes --- New, TLSv1/SSLv3, Cipher is DES-CBC3-SHA Server public key is 2048 bit Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE And then I send GET /directory HTTP/1.1 and see... SSL_connect:SSL renegotiate ciphers SSL_connect:unknown state SSL_connect:failed in unknown state 16444:error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure:.\ssl\s3_pkt.c:1156: From: Krzysztof Kwiatkowski Do you still see an error if you specify one cipher? f.e. AES256-SHA? I do get an error when using AES256-SHA, in fact a much earlier error as the server does not support that cipher, nor does it support AES-128-SHA. However I took the idea and if I add -cipher DES-CBC3-SHA (as selected by the server in the previous run) to s_connect then everything works OK, and if I add the same cipher selection to my program that that works too. When I do specify DES-CBC3-SHA, s_client also reports Secure Renegotiation IS supported This sounds important to me! :) Notice that the failure case reports renegotiation is NOT supported. So the question now is, how come negotiation *is* supported when I manually specify DES-CBC3-SHA and *not* supported when I leave default ciphers, when the server ends up picking DES-CBC3-SHA anyway. I have attached two pcap files, both captured using s_client - one with the default ciphers and one with -cipher DES_CBC3-SHA. I notice that the Client Hello is slightly different, when using all ciphers there are two extra extensions: ec_point_formats and elliptic_curves. Whether or not this is relevant I have no idea, I imagine not as they are likely omitted because DES isn't ECC. For now all the servers I need to talk to support DES-CBC3-SHA, this may not always be the case. Any idea how likely it is I will find a server that doesn't? Or is there something else I can set to make renegotiation supported without specifying the cipher. Thanks for all your suggestions so far, Ben s_client_default.pcap Description: s_client_default.pcap s_client_des_cbc3_sha.pcap Description: s_client_des_cbc3_sha.pcap
Re: ssl handshake failure in 1.0.1 but not 1.0.0
On Thu, Nov 07, 2013 at 12:29:13PM +, Ben Arnold wrote: I am using SSL_CTX_set_client_cert_cb to provide the client certificate when needed. I have a problem in that OpenSSL 1.0.1e does not trigger this callback for all websites that I expect it to, only some. Instead on the failing sites there is an SSL handshake failure after the client verifies the server certificate: You can test with s_client(1) and compare results. Is your client certificate an RSA certificate? How many bits of public key? Is its signature SHA1 or SHA256? SSL read: error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure, errno 0 Interestingly if I compile against 1.0.0k then there is no failure and the callback *is* triggered for all sites (that I have tried so far anyway). Sounds like a problem with TLSv1.2. If your client certificate is only 512-bits it may not be wide enough to sign a SHA384 digest, or some other TLSv1.2 parameter issue. Attaching a PCAP file of the traffic is much more useful than hex packet dumps. Capture the traffic with tcpdump -s0 -w file ... and look with wireshark -r file. If you don't understand the wireshark output, post the (binary) PCAP file containing just one failed TLS handshake, perhaps also a PCAP file with a succesful TLS handshake. -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: ssl handshake failure in 1.0.1 but not 1.0.0
From: owner-openssl-users On Behalf Of Viktor Dukhovni Sent: Thursday, November 07, 2013 11:02 On Thu, Nov 07, 2013 at 12:29:13PM +, Ben Arnold wrote: I am using SSL_CTX_set_client_cert_cb to provide the client certificate when needed. I have a problem in that OpenSSL 1.0.1e does not trigger this callback for all websites that I expect it to, only some. Instead on the failing sites there is an SSL handshake failure after the client verifies the server certificate: You can test with s_client(1) and compare results. Is your client certificate an RSA certificate? How many bits of public key? Is its signature SHA1 or SHA256? OP's log shows protocol hasn't reached the CertReq - ClientCert steps so at this point nothing about the client cert should matter. (And in any case the signature *on* the cert is by the CA key, unless it's self signed, in which case using it for client-auth would be really silly. From the log, unless OP fixed it, the server cert looks like a DIY CA, and if someone does that for the server I would expect it for clients also.) To OP: is the logged server cert info true? I note the log shows the client verification of the server failed; did this website give you a custom root to trust and did you simply not set that up (or perhaps not in the environment you're testing in)? SSL read: error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure, errno 0 Interestingly if I compile against 1.0.0k then there is no failure and the callback *is* triggered for all sites (that I have tried so far anyway). Sounds like a problem with TLSv1.2. If your client certificate is only 512-bits it may not be wide enough to sign a SHA384 digest, or some other TLSv1.2 parameter issue. Client hasn't even selected the cert yet. And although I agree a PCAP is better in general than a less-complete program log, just looking at the headers in the log you can see the client offered TLSv1.2 (0303) but the server only accepted TLSv1 (0301) so there definitely isn't any SHA-2 issue. But something else related to TLSv1.2 does seem likely. Possibly the new larger size (as OP observed), or the new extensions, although the initial negotiation apparently worked fine with both. Attaching a PCAP file of the traffic is much more useful than hex packet dumps. Capture the traffic with tcpdump -s0 -w file ... and look with wireshark -r file. If you don't understand the wireshark output, post the (binary) PCAP file containing just one failed TLS handshake, perhaps also a PCAP file with a succesful TLS handshake. I do notice the second ClientHello -- in response to server request -- using 1.0.1 re-offers TLSv1.2. I thought it was good practice when we know the server has previously rejected something not to re-offer it, but I don't recall where I saw this and I might well be wrong. Even so the server should negotiate down (as it did initially) or at minimum send an alert, not just close. To OP: If you can try to reproduce with s_client default (which is TLSv1.2 or less) and again specifying -tls1 (or -no_tls1_2 -no_tls1_1). That might narrow it down pretty close. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: ssl handshake failure in 1.0.1 but not 1.0.0
Do you still see an error if you specify one cipher? f.e. AES256-SHA? On 2013-11-07 22:26, Dave Thompson wrote: From: owner-openssl-users On Behalf Of Viktor Dukhovni Sent: Thursday, November 07, 2013 11:02 On Thu, Nov 07, 2013 at 12:29:13PM +, Ben Arnold wrote: I am using SSL_CTX_set_client_cert_cb to provide the client certificate when needed. I have a problem in that OpenSSL 1.0.1e does not trigger this callback for all websites that I expect it to, only some. Instead on the failing sites there is an SSL handshake failure after the client verifies the server certificate: You can test with s_client(1) and compare results. Is your client certificate an RSA certificate? How many bits of public key? Is its signature SHA1 or SHA256? OP's log shows protocol hasn't reached the CertReq - ClientCert steps so at this point nothing about the client cert should matter. (And in any case the signature *on* the cert is by the CA key, unless it's self signed, in which case using it for client-auth would be really silly. From the log, unless OP fixed it, the server cert looks like a DIY CA, and if someone does that for the server I would expect it for clients also.) To OP: is the logged server cert info true? I note the log shows the client verification of the server failed; did this website give you a custom root to trust and did you simply not set that up (or perhaps not in the environment you're testing in)? SSL read: error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure, errno 0 Interestingly if I compile against 1.0.0k then there is no failure and the callback *is* triggered for all sites (that I have tried so far anyway). Sounds like a problem with TLSv1.2. If your client certificate is only 512-bits it may not be wide enough to sign a SHA384 digest, or some other TLSv1.2 parameter issue. Client hasn't even selected the cert yet. And although I agree a PCAP is better in general than a less-complete program log, just looking at the headers in the log you can see the client offered TLSv1.2 (0303) but the server only accepted TLSv1 (0301) so there definitely isn't any SHA-2 issue. But something else related to TLSv1.2 does seem likely. Possibly the new larger size (as OP observed), or the new extensions, although the initial negotiation apparently worked fine with both. Attaching a PCAP file of the traffic is much more useful than hex packet dumps. Capture the traffic with tcpdump -s0 -w file ... and look with wireshark -r file. If you don't understand the wireshark output, post the (binary) PCAP file containing just one failed TLS handshake, perhaps also a PCAP file with a succesful TLS handshake. I do notice the second ClientHello -- in response to server request -- using 1.0.1 re-offers TLSv1.2. I thought it was good practice when we know the server has previously rejected something not to re-offer it, but I don't recall where I saw this and I might well be wrong. Even so the server should negotiate down (as it did initially) or at minimum send an alert, not just close. To OP: If you can try to reproduce with s_client default (which is TLSv1.2 or less) and again specifying -tls1 (or -no_tls1_2 -no_tls1_1). That might narrow it down pretty close. __ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
