Jack, Regarding compiling 6.2.x with OpenSSL 1.1.0f, I submitted PR #2570 which back-ports your fix for the sslheaders plugin that was required due to X509 being made opaque in OpenSSL 1.1. There was also a linking issue with Ubuntu when compiling with the 'with-openssl' configure option. Apparently, this might not occur on Red Hat which is what Jeremy used below.
Thanks, Peter -----Original Message----- From: Jeremy Payne [mailto:[email protected]] Sent: Friday, September 22, 2017 10:52 AM To: [email protected] Subject: Re: 回复：Re: Openssl 1.1.0f Support openssl lib and include directories werent matching versions. fixed that, and was able to compile without issue. On Fri, Sep 22, 2017 at 11:28 AM, Jack Bates <[email protected]> wrote: > I remember backporting some fixes for building with OpenSSL 1.1 to 6.2 [1] > ... Which errors are you still getting? > > I tried just now and successfully built the 6.2 branch with OpenSSL 1.1.0f > -- I did get warnings but no errors: > > $ git clone -b 6.2.x > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_trafficserver.git&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=E1it3OczMHrBWqN3XLenMT3yqV9H6AxFkXEuxM991cw&s=8HrQiBnIzItF-b6piI1aWqjSdcpsW9r59uuAoKA_sIQ&e= > > $ cd trafficserver > $ autoreconf -i > $ ./configure > $ make > > [1] > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_trafficserver_pull_1321&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=E1it3OczMHrBWqN3XLenMT3yqV9H6AxFkXEuxM991cw&s=nr0xsT0XCw7uLhrHvYwBhmQd5BbnSAKC2RfefCTpHgI&e= > > > > On 22/09/17 06:44 AM, Jeremy Payne wrote: >> >> Did you have to make any changes to 6.2 for it to compile cleanly >> against openssl 1.1 ? >> I can only get ATS 7+ to compile without producing any errors. >> >> >> >> On Thu, Sep 21, 2017 at 10:05 PM, <[email protected]> wrote: >>> >>> I tested traffic server 6.2 + openssl 1.1 and traffic server 6.2 + >>> openssl >>> 1.0.1 + patch respectively, and they almost have the same performance. >>> >>> >>> ----- 原始邮件 ----- >>> 发件人：<[email protected]> >>> 收件人："bcall" <[email protected]> >>> 抄送人："users" <[email protected]> >>> 主题：回复：Re: Openssl 1.1.0f Support >>> 日期：2017年09月22日 10点55分 >>> >>> With the patch, the ERR_clear_error() will only be called when the error >>> occurs. In the normal situation, ERR_clear_error() will not be called, so >>> the Err_get_state() will not be called and no lock contention in openssl >>> 1.0.1 with the patch. >>> >>> >>> ----- 原始邮件 ----- >>> 发件人：Bryan Call <[email protected]> >>> 收件人：[email protected] >>> 抄送人：users <[email protected]> >>> 主题：Re: Openssl 1.1.0f Support >>> 日期：2017年09月21日 23点37分 >>> >>> This only changes the order of the calls. There is still going to be >>> lock >>> contention inside OpenSSL 1.0.1. >>> >>> -Bryan >>> >>> On Sep 20, 2017, at 11:37 PM, [email protected] wrote: >>> >>> >>> The following traffic server patch can improve openssl 1.0.1 performance >>> as >>> openssl 1.1.0: >>> >>> diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc >>> index 5c9709c..5d306a1 100644 >>> --- a/iocore/net/SSLUtils.cc >>> +++ b/iocore/net/SSLUtils.cc >>> @@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t >>> nbytes, int64_t &nwritten) >>> if (unlikely(nbytes == 0)) { >>> return SSL_ERROR_NONE; >>> } >>> - ERR_clear_error(); >>> + >>> int ret = SSL_write(ssl, buf, (int)nbytes); >>> if (ret > 0) { >>> nwritten = ret; >>> @@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t >>> nbytes, int64_t &nwritten) >>> ERR_error_string_n(e, buf, sizeof(buf)); >>> Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d, >>> ERR_get_error=%ld (%s)", ret, ssl_error, e, buf); >>> } >>> + >>> + ERR_clear_error(); >>> + >>> return ssl_error; >>> } >>> >>> @@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, >>> int64_t &nread) >>> if (unlikely(nbytes == 0)) { >>> return SSL_ERROR_NONE; >>> } >>> - ERR_clear_error(); >>> + >>> int ret = SSL_read(ssl, buf, (int)nbytes); >>> if (ret > 0) { >>> nread = ret; >>> @@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t >>> nbytes, >>> int64_t &nread) >>> Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d, >>> ERR_get_error=%ld (%s)", ret, ssl_error, e, buf); >>> } >>> >>> + ERR_clear_error(); >>> + >>> return ssl_error; >>> } >>> >>> ssl_error_t >>> SSLAccept(SSL *ssl) >>> { >>> - ERR_clear_error(); >>> int ret = SSL_accept(ssl); >>> if (ret > 0) { >>> return SSL_ERROR_NONE; >>> @@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl) >>> Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d, >>> ERR_get_error=%ld (%s)", ret, ssl_error, e, buf); >>> } >>> >>> + ERR_clear_error(); >>> + >>> return ssl_error; >>> } >>> >>> ssl_error_t >>> SSLConnect(SSL *ssl) >>> { >>> - ERR_clear_error(); >>> int ret = SSL_connect(ssl); >>> if (ret > 0) { >>> return SSL_ERROR_NONE; >>> @@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl) >>> Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d, >>> ERR_get_error=%ld (%s)", ret, ssl_error, e, buf); >>> } >>> >>> + ERR_clear_error(); >>> + >>> return ssl_error; >>> } >>> >>> >>> From: Bryan Call <[email protected]> >>> Reply-To: "[email protected]" >>> <[email protected]> >>> Date: Thursday, September 21, 2017 at 8:38 AM >>> To: "[email protected]" <[email protected]> >>> Subject: Re: Openssl 1.1.0f Support >>> >>> >>> >>> I meant to say 1.1.0. >>> >>> >>> >>> -Bryan >>> >>> >>> >>> On Sep 20, 2017, at 3:54 PM, Bryan Call <[email protected]> wrote: >>> >>> >>> >>> I was see something like 2x the performance in my benchmarks with OpenSSL >>> 1.0.1. I have been doing all my development with OpenSSL 1.0.1 ATS since >>> May, when I upgraded to Fedora 26. >>> >>> >>> >>> -Bryan >>> >>> >>> >>> On Sep 20, 2017, at 2:16 PM, Dave Thompson <[email protected]> wrote: >>> >>> >>> >>> Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by >>> now >>> at best. The gist of my recollection is that QAT is an IO based async >>> engine, which of course ATS already has done extensively. I recall the >>> under-the-hood QAT longjumping was a non-starter in an ATS framework. >>> This >>> was all static code analysis. Integration looked like a non-starter, so >>> no >>> performance test done. >>> >>> Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard >>> aes-ni >>> acceleration", Susan (?Bryan?) was just telling me today of a measured >>> order >>> of magnitude improvement over with the same using Openssl 1.0.1(x) and >>> small >>> packet sizes... Improvement attributed to lock contention issues in the >>> older OpenSSL 1.0.1(x). >>> >>> >>> >>> Dave >>> >>> >>> >>> On Wed, Sep 20, 2017 at 3:22 PM, Jeremy Payne <[email protected]> wrote: >>> >>> Dave, >>> >>> >>> Did you run any comparison performance tests using the QAT engine ? >>> >>> Specifically around these configurations(or similar) >>> >>> >>> 1. ATS + Openssl 1.1.0(x) + QAT engine(sync) >>> >>> 2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration >>> >>> >>> >>> >>> On Wed, Sep 20, 2017 at 11:26 AM, Dave Thompson <[email protected]> wrote: >>> >>>> July 2016, I was evaluating the async Quick Assist in the context of >>>> ATS, >>> >>> >>>> and came away with the opinion it's value comes into play with a much >>> >>> >>>> simpler application. It's effectively it's own async engine, long >>>> jumping >>> >>> >>>> across the stack, and doesn't play well or add value to ATS's more >>> >>> >>>> extensive model to do similar.... not to mention mutually exclusive in >>>> their >>> >>> >>>> current forms. >>> >>> >>>> >>> >>>> Dave >>> >>> >>>> >>> >>>> >>> >>>> >>> >>>> On Wed, Sep 20, 2017 at 10:08 AM, Alan Carroll >>>> <[email protected]> >>> >>> >>>> wrote: >>> >>> >>>>> >>> >>>>> Susan and Dave Thompson were working on something related to that, >>>>> "crypto >>> >>> >>>>> proxy". There's a small mention of it by Susan at the Fall 2016 summit >>>>> in >>> >>> >>>>> the TLS state slides >>> >>> >>>>> (https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_TS_Presentations-2B-2D-2B2016&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=E1it3OczMHrBWqN3XLenMT3yqV9H6AxFkXEuxM991cw&s=zhE5_TgdGKvMRHbD7pN1VeJPtK8Zzlu8mGsIu7xpNVM&e= >>>>> ). >>>>> I'd >>> >>> >>>>> start there and see if you can bug Susan or Good Dave*. Although that >>>>> work >>> >>> >>>>> was designed to use an off box crypto engine, the implementation from >>>>> the >>> >>> >>>>> ATS point of view is identical to what you're writing about. Susan will >>>>> be >>> >>> >>>>> at the Fall 2017 Summit, I'd look her up then as well. >>> >>> >>>>> >>> >>>>> * To distinguish from "Evil Dave" Carlin. >>> >>> >>>>> >>> >>>>> On Wed, Sep 20, 2017 at 9:44 AM, Jeremy Payne <[email protected]> >>>>> wrote: >>> >>> >>>>>> >>> >>>>>> Thanks guys.. Thats all I needed to know.. Now I can look closer at my >>> >>> >>>>>> end. Will let you know what I find. >>> >>> >>>>>> >>> >>>>>> Also, any plans on supporting openssl async, which then allows for >>> >>> >>>>>> taking full advantage of the Intel QAT engine? >>> >>> >>>>>> Understood patches/commits are welcome, but just figured there may be >>> >>> >>>>>> some behind the scene works already started. >>> >>> >>>>>> >>> >>>>>> Thanks! >>> >>> >>>>>> >>> >>>>>> On Tue, Sep 19, 2017 at 6:31 PM, Alan Carroll >>>>>> <[email protected]> >>> >>> >>>>>> wrote: >>> >>> >>>>>>> Susan has also run some performance tests with 7.1.x and openSSL 1.1 >>> >>> >>>>>>> vs. >>> >>> >>>>>>> openSSL 1.0.2. >>> >>> >>>>>>> >>> >>>>>>> On Tue, Sep 19, 2017 at 5:55 PM, Leif Hedstrom <[email protected]> >>> >>> >>>>>>> wrote: >>> >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> On Sep 19, 2017, at 2:20 PM, Jeremy Payne <[email protected]> >>>>>>>> wrote: >>> >>> >>>>>>>> >>> >>>>>>>> I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some >>> >>> >>>>>>>> reason I can't establish a SSL/TLS connection. Has anyone >>> >>> >>>>>>>> successfully linked ATS against openssl 1.1.0f and successfully >>>>>>>> been >>> >>> >>>>>>>> able to establish a SSL/TLS session? >>> >>> >>>>>>>> In other words, is openssl 1.1.0f supported by ATS? If not, what >>>>>>>> about >>> >>> >>>>>>>> an earlier version of 1.1.0(x)?? >>> >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> Yeh, we’re running current master with OpenSSL v1.1.0f on >>>>>>>> docs.trafficserver.apache.org. Maybe you have some mismatch / issues >>> >>> >>>>>>>> between >>> >>> >>>>>>>> headers (when compiling ATS) and runtime? >>> >>> >>>>>>>> >>> >>>>>>>> Cheers, >>> >>> >>>>>>>> >>> >>>>>>>> — Leif >>>>>>>> >>>>>>> >>>>> >>>>> >>>> >>> >>> >
