Oh, wow, I think I heard my name. First, big disclaimer that I haven't really worked with any of this stuff for years, so take what I say with a big pinch of salt. I have no idea of what's the current state of things in OpenSSL-land.
As far as I can remember, the desire to do runtime discovery was for easy of packaging software using the library. For my company at the time, the main use was for Apache Spark and other software that people don't generally embed into their apps, but instead run directly from a package provided by Apache (or some distribution). Having to build multiple versions of an already large package and have to teach people to figure out what's the version that they need was just too much of a hassle. Maybe there are better ways to do that runtime discovery without the C code having to introspect into the shared libraries. Maybe that isn't even needed these days because the libssl API is more stable and the world has moved on from 1.0. Anyway, I won't make suggestions because I don't want to talk about things that I'm completely out of touch with these days, but please don't hold back on making changes because of something that, at the time, might have made sense but these days may not. On Fri, Aug 25, 2023 at 6:48 PM Alex Remily <alex.rem...@gmail.com> wrote: > <A big item to consider is if and how 1.1 vs 3.0 should be handled. Breakup > the current module into different maven modules? Not support both?> > > Agreed. Just to provide some history, when I was working on the 1.1.x > upgrade I was guided by commons committer Marcelo Vanzin. Marcelo required > a design that supported runtime discovery of the underlying openssl API. I > don't recall all of the rationale for the requirement, but he insisted that > any commons-crypto upgrade must support legacy and current versions of > openssl transparently to the calling program. The end result is what we > have now. I don't recall where in the code commons-crypto makes the > underlying version checks, at library initialization time or when a > specific function is called, but the end result is that users need only > download the latest commons-crypto release regardless of their underlying > openssl API--as long as they are running supported openssl versions. > > In my view there are a few open questions regarding the current approach as > compared to an API-specific one. One, what is the performance penalty > associated with the dynamic version checks? Two, how much complexity does > it introduce into the codebase? Finally, what was the use case that drove > the runtime checking requirement? Marcelo could answer the last question. > I don't know if he is still involved in the community (I haven't seen him > around for awhile. IIRC, he was primarily a Spark committer). > > Another consideration is the FIPS certification. I work in a heavily > regulated industry and FIPS is a real constraint. I haven't personally > encountered a requirement to deploy a FIPS compliant openssl in our > application code, but it's probably just a matter of time. In terms of > expanding our user base, it may make sense to provide that capability. It > doesn't seem to be generally available from an existing provider. > > Regarding message digests/HMAC, I question whether the performance gain > from native code would significantly outperform some of the modern JCA > providers. As Matt Sicker pointed out, there are other implementations > supported by major vendors, like AWS, that may be as fast as a JNI wrapper > on OpenSSL, or at least close enough not to bother with the added > complexity of the native stuff. The only way I know to answer that > question is to write the code and run a load test comparison. > > > https://aws.amazon.com/blogs/opensource/introducing-amazon-corretto-crypto-provider-accp/ > > In what forum should we discuss these issues? Are we limited to this > distro or do we have other options? How do we form teams? What is our > governance model? How do we make decisions? > > FYI: There's already issues on the backlog for OpenSSL 3 and HMAC: > https://issues.apache.org/jira/browse/CRYPTO-165 > https://issues.apache.org/jira/browse/CRYPTO-164 > > Alex > > > On Wed, Aug 23, 2023 at 10:21 PM Gary Gregory <garydgreg...@gmail.com> > wrote: > > > That would be great. I think this is worth the effort. A big item to > > consider is if and how 1.1 vs 3.0 should be handled. Breakup the current > > module into different maven modules? Not support both? > > > > Gary > > > > On Wed, Aug 23, 2023, 8:37 PM Alex Remily <alex.rem...@gmail.com> wrote: > > > > > <Ack about licensing. The idea wouldn't be to copy the code, but to > learn > > > how to implement message digests and HMAC on top of OpenSSL 3.0.8.> > > > > > > Implementing the OpenSSL 3 API and exposing OpenSSL HMAC functionality > in > > > commons-crypto are things I've wanted to engage on for a while now. I > > was > > > involved in the commons-crypto OpenSSL 1.1.x upgrade so I have some > > > familiarity with the code base, albeit dated. The reason that neither > > the > > > OpenSSL 3 API nor the HMAC functionality have been added to > > commons-crypto > > > yet, IMHO, is because both are non-trivial efforts and we all have day > > > jobs. Until now, there hasn't been much of an ask for either feature, > but > > > considering the recent chatter maybe there's enough interest to > generate > > > some momentum. I would be happy to collaborate on this effort if it > > gains > > > traction. > > > > > > Alex > > > > > > On Wed, Aug 23, 2023 at 5:21 PM Matt Sicker <m...@musigma.org> wrote: > > > > > >> You may find this project (also Apache licensed, so fairly safe for > > reuse > > >> with commons-crypto updates) to be helpful for that aspect of message > > >> digests, Macs, etc. > > >> > > >> [image: amazon-corretto-crypto-provider.png] > > >> > > >> corretto/amazon-corretto-crypto-provider: The Amazon Corretto Crypto > > >> Provider is a collection of high-performance cryptographic > > implementations > > >> exposed via standard JCA/JCE interfaces. > > >> <https://github.com/corretto/amazon-corretto-crypto-provider> > > >> github.com < > https://github.com/corretto/amazon-corretto-crypto-provider > > > > > >> <https://github.com/corretto/amazon-corretto-crypto-provider> > > >> > > >> > > >> On Aug 1, 2023, at 9:17 AM, Jim Showalter < > jamesleeshowal...@gmail.com> > > >> wrote: > > >> > > >> I'm still trying to come up to speed on your PR/fork. A lot to learn! > > >> > > >> Ack about licensing. The idea wouldn't be to copy the code, but to > learn > > >> how to implement message digests and HMAC on top of OpenSSL 3.0.8. > > >> > > >> On Tue, Aug 1, 2023, 5:03 AM Gary Gregory <garydgreg...@gmail.com> > > wrote: > > >> > > >> In the short, the best way to help is to provide PRs. > > >> In more detail, we should probably come up with some kind of a plan so > > >> that > > >> everyone uses their time wisely. > > >> > > >> I'll review my branch this week or next and see where that stands, but > > >> feel > > >> free to look at it, use it, PR it, as I might not actually be able to > > take > > >> the time this week. > > >> > > >> WRT openssl4j, you CANNOT bring in anything licensed under the LGPL. > > >> IANAL, > > >> but our documentation seems clear to me, please see > > >> https://www.apache.org/legal/resolved.html#category-x > > >> > > >> Hope this helps and we can make it work! > > >> Gary > > >> > > >> > > >> On Mon, Jul 31, 2023, 8:02 PM Jim Showalter < > > jamesleeshowal...@gmail.com> > > >> wrote: > > >> > > >> A split seems reasonable. We were amazed at how few changes you had to > > >> > > >> make > > >> > > >> to support OpenSSL 3.x. The EVPs are very different. But it sounds > like > > >> there's more to do. > > >> > > >> The problem with commons-codec is that it doesn't use OpenSSL or any > > >> > > >> other > > >> > > >> FIPS-certified cryptographic module. For example, HmacUtils uses Mac, > > >> > > >> which > > >> > > >> is supplied by the JRE, which isn't FIPS-certified. > > >> > > >> In order to qualify for FedRAMP High, which is table stakes for a lot > of > > >> corporate and government contracts, FIPS has to be used. It's mandated > > by > > >> statute. No wiggle room. > > >> > > >> The promise of bc-fips is that it is FIPS-certified, is a JSP, and > > >> implements the full JCE. The drawback is that the bc-fips org is > funded > > >> through donations and consulting, and is always very far behind Java > > >> releases (it's still on Java 11, for example). > > >> > > >> What we need is a full-featured JSP that is based on a FIPS-certified > > >> cryptographic module that is implemented using native code and JNI. > > >> > > >> We can get FIPS-certified digests and HMAC from openssl4j. What we > need > > >> from commons-crypto are the ciphers, and they need to be on OpenSSL > > >> > > >> 3.0.8. > > >> > > >> > > >> If there's anything we can do to help make that happen, please let us > > >> > > >> know. > > >> > > >> > > >> On Mon, Jul 31, 2023 at 2:58 PM Gary Gregory <garydgreg...@gmail.com> > > >> wrote: > > >> > > >> Hi Jim, > > >> > > >> My branch has not been merged because it does not fully work. It's > > >> challenging to update the code such that one can use either OpenSSL > > >> > > >> 1.1.1 > > >> > > >> or 3.0.0 or both. We might need the component split into more than one > > >> Maven module. > > >> > > >> The name commons-crypto might have been poorly chosen because it's > > >> > > >> current > > >> > > >> remit is an OpelSSL wrapper. That said there is room for more > features, > > >> which may mean splitting things up into more than one Maven module. > > >> > > >> Commons Code provides more convenience wrappers for JRE message > digests > > >> including HMAC: > > >> https://commons.apache.org/proper/commons-codec/apidocs/index.html > > >> > > >> Are you looking to wrap or implement HMAC and message digests > > >> > > >> differently? > > >> > > >> > > >> Gary > > >> > > >> > > >> On Mon, Jul 31, 2023, 5:04 PM Jim Showalter < > > >> > > >> jamesleeshowal...@gmail.com > > >> > > >> > > >> wrote: > > >> > > >> We are trying to replace bc-fips ( > > >> > > >> https://www.bouncycastle.org/fips-java/) > > >> > > >> with a JSP that is based on a cryptographic module that is 1) a > > >> > > >> native > > >> > > >> library and 2) is certified for FIPS 140-2 ( > > >> https://csrc.nist.gov/pubs/fips/140-2/upd2/final). > > >> > > >> A native library is faster, plus it doesn't entangle the Java > > >> > > >> classpath > > >> > > >> with restrictions on Java versions or load order the way bc-fips > > >> > > >> does. > > >> > > >> > > >> The two available native libraries we're aware of are BoringSSL and > > >> OpenSSL. > > >> > > >> For various reasons, we want to use OpenSSL. > > >> > > >> OpenSSL 1.1.1 was only FIPS-certified on RedHat (and they had to > > >> > > >> modify > > >> > > >> it > > >> > > >> to add FIPS support), and the certification expires soon. > > >> > > >> OpenSSL 1.1.1 is the version commons-crypto is currently based on. > > >> > > >> OpenSSL 3.0.8 is FIPS-certified on a variety of platforms, supports > > >> > > >> FIPS > > >> > > >> mode natively, and its successor (3.1.x) will be certified for FIPS > > >> > > >> 140-3. > > >> > > >> > > >> We're very interested in > > >> https://github.com/garydgregory/commons-crypto/tree/openssl3, which > > >> > > >> adds > > >> > > >> support for OpenSSL 3.0.8 to commons-crypto, per > > >> https://issues.apache.org/jira/browse/CRYPTO-164. > > >> > > >> But that PR was never merged, hasn't been touched since December 20, > > >> > > >> 2022, > > >> > > >> and is currently 92 commits behind the main branch. > > >> > > >> What would it take to update that PR with all of the commits since > > >> > > >> then, > > >> > > >> and get it merged? > > >> > > >> Once that's done, we'd be happy to submit a PR to add FIPS mode, per > > >> https://issues.apache.org/jira/browse/CRYPTO-136. > > >> > > >> Also, commons-crypto doesn't support message digests or HMAC. We're > > >> > > >> in > > >> > > >> the > > >> > > >> process of adding HMAC and FIPS mode to > > >> https://github.com/sfuhrm/openssl4j, > > >> which has message digests, and targets OpenSSL 3.0.8. > > >> > > >> It seems like the message digests and HMAC from openssl4j could be > > >> > > >> merged > > >> > > >> into commons-crypto, to bring it closer to being a full JCE > > >> > > >> implementation. > > >> > > >> Is there any interest in seeing that happen? > > >> > > >> > > >> > > >> > > >> > > >> > > > -- Marcelo Vanzin van...@gmail.com "Life's too short to drink cheap beer"
