What exactly is needed to reproduce the poor performance issue with openssl 3? I was able to test 20k req/sec with it using k6 to simulate 16k users over a wan. The k6 box did have openssl1. Probably could have sustained more, but that's all I need right now. Openssl v1 tested a little faster, but within 10%. Wasn't trying to max out my tests as that should be over 4x the needed performance.
Not doing H3, and the backends are send-proxy-v2. Default libs on Alma linux on arm. # rpm -qa | grep openssl openssl-pkcs11-0.4.11-7.el9.aarch64 xmlsec1-openssl-1.2.29-9.el9.aarch64 openssl-libs-3.0.1-43.el9_0.aarch64 openssl-3.0.1-43.el9_0.aarch64 openssl-devel-3.0.1-43.el9_0.aarch64 This is the first box I setup with EL9 and thus openssl-3. Might it only be an issue when ssl is used to the backends? On Thu, Dec 15, 2022 at 11:50 PM Willy Tarreau <[email protected]> wrote: > On Thu, Dec 15, 2022 at 08:58:29PM -0700, Shawn Heisey wrote: > > I'm sure the performance issue has been brought to the attention of the > > OpenSSL project ... what did they have to say about the likelihood and > > timeline for providing a fix? > > They're still working on it for 3.1. 3.1-alpha is "less worse" than > 3.0 but still far behind 1.1.1 in our tests. > > > Is there an article or bug filed I can read for more information? > > There's this issue that centralizes the status of the most important > regression reports: > > https://github.com/openssl/openssl/issues/17627#issuecomment-1060123659 > > We've also planned to issue an article to summarize our observations > about this before users are hit too strong, but it will take some > time to collect all info and write it down. But it's definitely a big > problem for users who upgrade to latest LTS distros that shipped 3.0 > without testing it (though I can't blame distros, it's not the package > maintainers' job to run performance tests on what they maintain) :-( > > My personal feeling is that this disaster combined with the stubborn > refusal to support the QUIC crypto API that is mandatory for any > post-2021 HTTP agent basically means that OpenSSL is not part of the > future of web environments and that it's urgent to find alternatives, > just like all other projects are currently seeking. And with http-based > products forced to abandon OpenSSL, it's unlikely that their performance > issues will be relevant in the future so it should get even worse over > time by lack of testing and exposure. It's sad, because before the QUIC > drama, we hoped to spend some time helping them improve their perfomance > by reducing the locking abuse. Now the project has gone too far in the > wrong direction for anything to be doable anymore, and I doubt that > anyone has the energy to fork 1.1.1 and restart from a mostly clean > state. But anyway, a solution must be found for the next batch of LTS > distros so that users can jump from 20.x to 24.x and skip 22.x. > > There's currently a great momentum around WolfSSL that was already > adopted by Apache, Curl, and Ngtcp2 (which is the QUIC stack that > powers most HTTP/3-compatible agents). Its support on haproxy is > making fast progress thanks to the efforts on the two sides, and it's > pleasant to speak to people who care about performance. I'd bet we'll > find it packaged in a usable state long before OpenSSL finally changes > their mind on QUIC and reaches distros in a usable state. That's a > perfect (though sad) example of the impact of design by committee! > > https://www.openssl.org/policies/omc-bylaws.html#OMC > https://en.wikipedia.org/wiki/Design_by_committee > > Everything was written... > Willy > >
