Thanks for the help!
Yes, the client certificate is directly signed by the certificate in ca.pem.
There is no intermediary. I did not know about the "ssl" debug flag but this is
very helpful. It seems your theory about failing the client cert verification
was correct.
[Jan 6 08:21:22.635] {0x7ff241b10700} DEBUG:
<SSLNetVConnection.cc:1213<http://sslnetvconnection.cc:1213>
(sslServerHandShakeEvent)> (ssl-diag) SSL handshake error: SSL_ERROR_WANT_READ
(2), errno=0
[Jan 6 08:21:22.636] {0x7ff241b10700} DEBUG:
<SSLNetVConnection.cc:1139<http://sslnetvconnection.cc:1139>
(sslServerHandShakeEvent)> (ssl) Go on with the handshake state=5
[Jan 6 08:21:22.636] {0x7ff241b10700} DEBUG:
<SSLUtils.cc:1510<http://sslutils.cc:1510> (ssl_callback_info)> (ssl)
ssl_callback_info ssl: 0x7ff228028ab0 where: 8193 ret: 1 State: SSLv3/TLS write
server done
[Jan 6 08:21:22.636] {0x7ff241b10700} DEBUG:
<SSLUtils.cc:401<http://sslutils.cc:401> (ssl_verify_client_callback)> (ssl)
Callback: verify client cert
[Jan 6 08:21:22.636] {0x7ff241b10700} DEBUG:
<SSLNetVConnection.cc:1603<http://sslnetvconnection.cc:1603> (callHooks)> (ssl)
callHooks sslHandshakeHookState=5
[Jan 6 08:21:22.636] {0x7ff241b10700} DEBUG:
<SSLNetVConnection.cc:1675<http://sslnetvconnection.cc:1675> (callHooks)> (ssl)
callHooks iterated to curHook=(nil)
[Jan 6 08:21:22.636] {0x7ff241b10700} DEBUG:
<SSLUtils.cc:1510<http://sslutils.cc:1510> (ssl_callback_info)> (ssl)
ssl_callback_info ssl: 0x7ff228028ab0 where: 16392 ret: 592 State: error
[Jan 6 08:21:22.636] {0x7ff241b10700} DEBUG:
<SSLUtils.cc:1510<http://sslutils.cc:1510> (ssl_callback_info)> (ssl)
ssl_callback_info ssl: 0x7ff228028ab0 where: 8194 ret: -1 State: error
[Jan 6 08:21:22.636] {0x7ff241b10700} DEBUG:
<SSLUtils.cc:2344<http://sslutils.cc:2344> (SSLAccept)> (ssl.error.accept) SSL
accept returned -1, ssl_error=1, ERR_get_error=337100934 (error:1417C086:SSL
routines:tls_process_client_certificate:certificate verify failed)
[Jan 6 08:21:22.636] {0x7ff241b10700} DEBUG:
<SSLNetVConnection.cc:1213<http://sslnetvconnection.cc:1213>
(sslServerHandShakeEvent)> (ssl-diag) SSL::140678460933888:error:1417C086:SSL
routines:tls_process_client_certificate:certificate verify
failed:../ssl/statem/statem_srvr.c:3701: peer address is 192.168.1.4
[Jan 6 08:21:22.636] {0x7ff241b10700} DEBUG:
<SSLNetVConnection.cc:1213<http://sslnetvconnection.cc:1213>
(sslServerHandShakeEvent)> (ssl-diag) SSL handshake error: SSL_ERROR_SSL (1),
errno=0
[Jan 6 08:21:22.636] {0x7ff241b10700} DEBUG:
<SSLNetVConnection.cc:1339<http://sslnetvconnection.cc:1339>
(sslServerHandShakeEvent)> (ssl-diag)
SSLNetVConnection::sslServerHandShakeEvent, SSL_ERROR_SSL errno=0
The certificate is quite old but valid.
Validity
Not Before: Aug 18 14:29:55 2017 GMT
Not After : Aug 16 14:29:55 2027 GMT
There is nothing noteworthy in diags.log nor any startup errors or warnings.
Thanks again,
Zack
On Jan 6, 2021, at 7:38 AM, Susan Hinrichs
<[email protected]<mailto:[email protected]>> wrote:
The config looks good to me. There are autests that exercise ATS requiring
certs from the client with a variety of CA verification configurations. I
haven't worked with the 8.x branch, but those tests are run against that branch
I believe.
Is your client certificate directly signed by the certificate in ca.pem? Is
there an intermediate cert floating around someplace? Assuming the certs are
exchanged as in the working case, it seems likely that the failure is occurring
in the client certificate verification. I assume that ATS is sending the SSL
alert.
Any interesting start up failure messages in diags.log?
The next thing to check is turning on debugging and setting the ssl debug tag.
On Tue, Jan 5, 2021 at 4:49 PM Zack Bartel
<[email protected]<mailto:[email protected]>> wrote:
Hello,
We are using client -> TS SSL termination and using client certificates and a
CA. While upgrading to 8.x this has stopped working and I'm having trouble
figuring out why. I was wondering if there may have been some change in 8.x or
error with my config which would be obvious to someone.
The error is "SSL alert number 80" internal error.
SSL_connect:SSLv3/TLS read server certificate request
SSL_connect:SSLv3/TLS read server done
SSL_connect:SSLv3/TLS write client certificate
SSL_connect:SSLv3/TLS write client key exchange
SSL_connect:SSLv3/TLS write certificate verify
SSL_connect:SSLv3/TLS write change cipher spec
SSL_connect:SSLv3/TLS write finished
read from 0x564393d04170 [0x564393d0e6a3] (5 bytes => 5 (0x5))
0000 - 15 03 03 00 02 .....
read from 0x564393d04170 [0x564393d0e6a8] (2 bytes => 2 (0x2))
0000 - 02 50 .P
SSL3 alert read:fatal:internal error
SSL_connect:error in error
139713745085760:error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert
internal error:../ssl/record/rec_layer_s3.c:1543:SSL alert number 80
I've inspected the traffic with Wireshark to confirm the exchange is identical
between the versions up until the "Internal Error" response. Tried up to
7.1.12-rc0 which works and 8.0.0 -> 8.1.x which does not. I've also tried on
Ubuntu and Centos8 with the same results.
Relevant config:
records.config:
CONFIG proxy.config.ssl.CA.cert.path STRING /ssl/
CONFIG proxy.config.ssl.CA.cert.filename STRING ca.pem
CONFIG proxy.config.ssl.client.verify.server INT 1
CONFIG proxy.config.ssl.client.certification_level INT 2
CONFIG proxy.config.ssl.client.CA.cert.path STRING /etc/pki/tls/certs/
CONFIG proxy.config.ssl.client.CA.cert.filename STRING ca-bundle.crt
CONFIG proxy.config.ssl.client.cipher_suite STRING
TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:DES-CBC3-SHA
CONFIG proxy.config.ssl.server.private_key.path STRING /ssl/
CONFIG proxy.config.ssl.server.cert.path STRING /ssl/
CONFIG proxy.config.ssl.server.cipher_suite STRING
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-DSS-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
ssl_multicert.config:
ssl_cert_name=server.pem
Any help on this greatly appreciated, thank you,
Zack
