Thanks Mark,
I was going to try to build tcnative with debug this weekend to try to help out
but life begged to disagree...i will try to step in that code next week as
well...
Appreciate it!
George
-----Original Message-----
From: Mark Thomas <ma...@apache.org>
Sent: Sunday, October 20, 2019 2:38 PM
To: users@tomcat.apache.org
Subject: Re: Client Cert TLS issue
Just a note to say I haven't forgotten this. I hope to look at this this week
unless someone beats me to it.
Mark
On 16/10/2019 17:55, George Stanchev wrote:
>
> On 15/10/2019 22:15, George Stanchev wrote:
>> Hi,
>>
>> I would need some help with tracking an issue with TC 8.5.47 (windows x64,
>> java: azul 1.8.0_222) configured with [1] and tcnative-1.dll. When a simple
>> client tries to connect to the server, the server hangs on SSL handshake
>> until either the client times out on read or the server times out (if I set
>> the HttpsURLConnection#setConnectTimeout(0) and ...#setReadTimeout(0)). I
>> have enabled the client side SSL trace and everything goes well until ECDH
>> key exchange (for brevity I have enabled only one TLS suite
>> "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA"). I can provide the debug logs if
>> requested. The cacerts we use is whatever comes with Azul's Java distro
>> which has ~150 entries + the custom cert for testing. The issue comes from
>> how the connector deals with trusted certs because if I reduce the cacerts
>> to contain only the test certificate, the request is being served without an
>> issue. Also if I remove the tcnative-1.dll from TC there is no issue either.
>>
>> Perhaps I am missing something. Any help is appreciated.
>
> <Mark wrote>
> This sounds like it is repeatable and that you have a system you can test
> with. On that basis here are a few things to try:
>
> 1. Take a 3 thread dumps ~5s apart of the Tomcat process when TLS handshake
> is hanging.
>
> 2. Try a binary search to try and determine if the issue is the number of
> certificates in the trust store or is caused by a specific certificate.
>
> It sounds like there might be an issue converting one or more of the trusted
> certs in the trust store to a format OpenSSL can work with.
>
> </Mark wrote>
>
> So the thread dumps didn't prove to be very useful - at least I couldn't see
> anything. I attached one of them (dump3.txt). But the trial and error in the
> binary searched proved it is not any certificate but a magic number of
> certificates in cacerts - 125 hang and 124 work. I don't know if it is a
> memory issue or buffer size issue - I tried removing and adding random certs
> around the 124/125 boundary and which one is out and in doesn't make a
> difference. I realize that a cert is rather large blob so it still could be
> memory issue with the size of the cert doesn't matter that much. In
> 8.5.47-src\java\org\apache\tomcat\util\net\openssl\OpenSSLContext.java#init(kms,
> tms, sr) we pass the DER-encoded certs to OpenSSL:
>
> // Pass along the DER encoded certificates of the accepted
> client
> // certificate issuers, so that their subjects can be
> presented
> // by the server during the handshake to allow the client
> choosing
> // an acceptable certificate
> for (X509Certificate caCert :
> x509TrustManager.getAcceptedIssuers()) {
> SSLContext.addClientCACertificateRaw(ctx,
> caCert.getEncoded());
> if (log.isDebugEnabled())
> log.debug(sm.getString("openssl.addedClientCaCert",
> caCert.toString()));
> }
>
> And this is not where it hangs. I stepped through the code through the
> handshaker but still haven't been able to figure out the hang point as I am
> not familiar with the details of that portion of TC code . I've attached two
> cacerts that you can plug in as truststore to try out the issue.
>
> George
>
>
>
>>
>> George
>>
>> [1]
>>
>> <Connector
>> port="8443" SSLEnabled="true" maxHttpHeaderSize="8192"
>> maxThreads="150" acceptCount="100" enableLookups="false"
>> disableUploadTimeout="true"
>> scheme="https" secure="true" clientAuth="true" sslProtocol="TLS"
>> sslEnabledProtocols="+TLSv1 +TLSv1.1 +TLSv1.2"
>> protocol="org.apache.coyote.http11.Http11NioProtocol"
>> keystoreType="JKS"
>> keystoreFile="${server.conf.dir}/serena.keystore" keystorePass="changeit"
>> keyAlias="jboss" URIEncoding="UTF-8" useServerCipherSuitesOrder="true"
>> ciphers="TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384,
>> TLS_CHACHA20_POLY1305_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
>> TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
>> TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
>> TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
>> TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,
>> TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,
>> TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,
>> TLS_DHE_RSA_WITH_AES_128_CBC_SHA256,
>> TLS_DHE_RSA_WITH_AES_256_CBC_SHA256,
>> TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256,
>> TLS_DHE_DSS_WITH_AES_128_GCM_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,
>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
>> TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,
>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
>> TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256,
>> TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_128_CBC_SHA256,
>> TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA,
>> TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA,
>> TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_128_SHA_GCM_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_128_SHA_CBC_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_128_CCM, TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8,
>> TLS_ECDHE_ECDSA_WITH_AES_256_GCM_CCM_8,
>> TLS_ECDHE_ECDSA_WITH_AES_256_CCM, TLS_ECDHE_ECDSA_WITH_AES_256_SHA,
>> TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8,
>> TLS_DHE_DSS_WITH_AES_256_CBC_SHA256,
>> TLS_RSA_WITH_CAMELLIA_128_CBC_SHA256,
>> TLS_RSA_WITH_CAMELLIA_256_CBC_SHA256,
>> TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA256,
>> TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
>> TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA256,
>> TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256,
>> TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384,
>> TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
>> TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384" />
>>
>>
>
> B‹KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
> KCB•È [œÝXœØÜšX™KK[XZ[
> ˆ \Ù\œË][œÝXœØÜšX™P ÛXØ]
> ˜\XÚK›Ü™ÃB‘›ÜˆY][Û˜[ÛÛ[X[™ ËK[XZ[
> ˆ \Ù\œËZ[ ÛXØ]
> ˜\XÚK›Ü™ÃBƒ
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
