A bit of context:
I have this endpoint behind an AWS ALB. I do SSL termination at the ALB.
To my surprise, when looking at the client_tlsnegotiation_error_count
metric for the ALB, I've noticed a substantial amount of failed
connection attempts due to TLS negotiation errors - perhaps around 5% of
total traffic but this estimate could be wrong by a substantial margin.
The clients are mostly an average cross-section of current mobile
devices in use these days in the US.
The ALB is configured with the default TLS policy that Amazon provides:
| ssl-enum-ciphers:
| TLSv1.0:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| compressors:
| NULL
| cipher preference: server
| TLSv1.1:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| compressors:
| NULL
| cipher preference: server
| TLSv1.2:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
| TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| compressors:
| NULL
| cipher preference: server
|_ least strength: A
Unfortunately, ALBs do not provide error logs, so I could not directly
identify why some clients are failing the SSL negotiation.
Instead, I've pushed SSL termination deeper into the stack, to the Nginx
frontend, and replaced the ALB with a plain TCP-based ELB. Now
connections to port 443 are forwarded directly to Nginx for SSL
negotiation.
I've configured Nginx with the exact same protocol versions and ciphers
like the ALB:
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers
'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA';
ssl_prefer_server_ciphers on;
I've verified with nmap and I get the same ssl-enum-ciphers list from
Nginx.
Now in the Nginx error log I get lots of lines like this:
SSL_do_handshake() failed (SSL: error:140A1175:SSL
routines:ssl_bytes_to_cipher_list:inappropriate fallback) while SSL
handshaking
Still not very informative, so I've run tcpdump on port 443 on the Nginx
instances. As expected, there's some amount of SSL errors like this:
Secure Sockets Layer
TLSv1 Record Layer: Alert (Level: Fatal, Description:
Inappropriate Fallback)
Content Type: Alert (21)
Version: TLS 1.0 (0x0301)
Length: 2
Alert Message
Level: Fatal (2)
Description: Inappropriate Fallback (86)
In that same TCP stream there's this Client Hello packet:
Secure Sockets Layer
TLSv1 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 165
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 161
Version: TLS 1.0 (0x0301)
Random
GMT Unix Time: Jun 7, 2050 12:50:05.000000000 PST
Random Bytes:
da03ff7045a5f76e78edf61c097c75e4e141df6649ef1861...
Session ID Length: 0
Cipher Suites Length: 28
Cipher Suites (14 suites)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
(0xc00a)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
(0xc009)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
(0xc013)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
(0xc014)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
(0xc007)
Cipher Suite: TLS_ECDHE_RSA_WITH_RC4_128_SHA
(0xc011)
Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA
(0x0033)
Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA
(0x0039)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
Cipher Suite: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)
Cipher Suite: TLS_RSA_WITH_RC4_128_SHA (0x0005)
Cipher Suite: TLS_RSA_WITH_RC4_128_MD5 (0x0004)
Cipher Suite: TLS_FALLBACK_SCSV (0x5600)
Compression Methods Length: 1
Compression Methods (1 method)
Compression Method: null (0)
Extensions Length: 92
Extension: renegotiation_info
Type: renegotiation_info (0xff01)
Length: 1
Renegotiation Info extension
Renegotiation info extension length: 0
Extension: server_name
Type: server_name (0x0000)
Length: 27
Server Name Indication extension
Server Name list length: 25
Server Name Type: host_name (0)
Server Name length: 22
Server Name: [REDACTED]
Extension: Extended Master Secret
Type: Extended Master Secret (0x0017)
Length: 0
Extension: SessionTicket TLS
Type: SessionTicket TLS (0x0023)
Length: 0
Data (0 bytes)
Extension: Application Layer Protocol Negotiation
Type: Application Layer Protocol Negotiation
(0x0010)
Length: 26
ALPN Extension Length: 24
ALPN Protocol
ALPN string length: 5
ALPN Next Protocol: h2-16
ALPN string length: 8
ALPN Next Protocol: spdy/3.1
ALPN string length: 8
ALPN Next Protocol: http/1.1
Extension: ec_point_formats
Type: ec_point_formats (0x000b)
Length: 2
EC point formats Length: 1
Elliptic curves point formats (1)
EC point format: uncompressed (0)
Extension: elliptic_curves
Type: elliptic_curves (0x000a)
Length: 8
Elliptic Curves Length: 6
Elliptic curves (3 curves)
Elliptic curve: secp256r1 (0x0017)
Elliptic curve: secp384r1 (0x0018)
Elliptic curve: secp521r1 (0x0019)
It's a little puzzling because the exchange of crypto messages uses TLS
1.0 which the server definitely supports, and the client should be very
likely to support too.
I've seen discussions online saying that the presence of the
TLS_FALLBACK_SCSV cipher suite is an indication that this failed
connection is related to anti-POODLE security measures, and the
communication is likely to be retried again. Is that correct?
For the most part, what I'm trying to do is find the answers to these
questions:
1. Is this bad? If yes, why? (what are the things that the clients need
that the endpoint is not providing)
2. If it's not bad, then why do we get this constant stream of SSL
errors?
It's a little difficult to search the capture file and try to correlate
the failed SSL handshake with other, successful connections, because the
source IPs are masked by the ELB. There might be a way to rely on the
PROXY protocol header to identify IPs, but I'll have to figure out how
to do that.
--
Florin Andrei
http://florin.myip.org/
--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
