[openssl.org #2844] TLS 1.2 server sends abort when TLS 1.2 client doesn't include Signature Algorithm extension

[John Foley via RT]

RFC 5246 allows a TLS 1.2 client to omit the Signature Algorithm 
extension.  See section 7.4.1.4.1 for details.  This creates a problem 
for OpenSSL 1.0.1 when acting as a server and either a DSA or ECDSA 
certificate is used.  Because the ClientHello does not contain the 
Signature Algorithm extension, tls1_process_sigalgs() is never invoked 
on the server side to set the digest used for the session context.  
Later during the handshake when the server sends the ServerKeyExchange, 
the call to tls12_get_sigandhash() fails, which causes the server to 
abort the handshake.

The attached patch to tls12_get_sigandhash() resolves the problem.  
However, the OpenSSL maintainers may have a better solution.


Index: t1_lib.c
===================================================================
--- t1_lib.c	(revision 585)
+++ t1_lib.c	(working copy)
@@ -2830,6 +2830,20 @@
 		return 0;
 	md_id = tls12_find_id(EVP_MD_type(md), tls12_md,
 				sizeof(tls12_md)/sizeof(tls12_lookup));
+
+	/*
+	 * There's a special case that we need to check.  If
+	 * the client didn't send a signature algorithm extension, which
+	 * is possible under RFC5246 guidelines, the above check to 
+	 * find the hash algorithm will fail if the server's cert is
+	 * either DSA or ECDSA.  In this case, we default to SHA1
+	 * as the hash algorithm
+	 */
+	if (md_id == -1 && 
+	    (EVP_MD_type(md) == NID_ecdsa_with_SHA1 || 
+	     EVP_MD_type(md) == NID_dsa)) {
+	    md_id = TLSEXT_hash_sha1;
+	}
 	if (md_id == -1)
 		return 0;
 	sig_id = tls12_get_sigid(pk);

<<inline: foleyj.vcf>>