Which Mac OS version did you use? Since I upgraded to BigSur (OS 11) my Commons Crypto builds fail. I think Apple moved a bunch of the developer libraries, but I haven't taken the time to troubleshoot.
On Wed, Jun 29, 2022 at 8:55 PM Gary Gregory <garydgreg...@gmail.com> wrote: > Arg, idiotic line wrapping removed: https://pastebin.com/raw/nPczrrd8 > > Gary > > On Wed, Jun 29, 2022 at 8:49 PM Gary Gregory <garydgreg...@gmail.com> > wrote: > > > > FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and ran > > 'mvn clean verify' and everything built just fine. > > > > The Maven ant-run output was: > > > > [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto --- > > [INFO] Executing tasks > > > > make: > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force > > -classpath target/classes -o > > > target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h > > org.apache.commons.crypto.random.OpenSslCryptoRandomNative > > [exec] gcc -arch x86_64 -Ilib/inc_mac > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include > > -I"src/main/native/org/apache/commons/crypto/" > > -I"/usr/local/Cellar/openjdk@8 > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin" > > -I"target/jni-classes/org/apache/commons/crypto/cipher" > > -I"target/jni-classes/org/apache/commons/crypto/random" -c > > > src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c > > -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force > > -classpath target/classes -o > > target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h > > org.apache.commons.crypto.cipher.OpenSslNative > > [exec] gcc -arch x86_64 -Ilib/inc_mac > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include > > -I"src/main/native/org/apache/commons/crypto/" > > -I"/usr/local/Cellar/openjdk@8 > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin" > > -I"target/jni-classes/org/apache/commons/crypto/cipher" > > -I"target/jni-classes/org/apache/commons/crypto/random" -c > > src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force > > -classpath target/classes -o > > target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h > > org.apache.commons.crypto.OpenSslInfoNative > > [exec] gcc -arch x86_64 -Ilib/inc_mac > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include > > -I"src/main/native/org/apache/commons/crypto/" > > -I"/usr/local/Cellar/openjdk@8 > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin" > > -I"target/jni-classes/org/apache/commons/crypto/cipher" > > -I"target/jni-classes/org/apache/commons/crypto/random" > > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"' > > -I"target/jni-classes/org/apache/commons/crypto" -c > > src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o > > [exec] gcc -arch x86_64 -Ilib/inc_mac > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include > > -I"/usr/local/Cellar/openjdk@8 > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin" > > -I"target/jni-classes/org/apache/commons/crypto/cipher" > > -I"target/jni-classes/org/apache/commons/crypto/random" -o > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o -dynamiclib > > -L/usr/local/lib > > [exec] strip -x > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib > > [exec] cp > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib > > > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib > > [exec] cp > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib > > > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib > > > > Gary > > > > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <garydgreg...@gmail.com> > wrote: > > > > > > I agree with Alex. > > > > > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and > > > tight. Last time I checked I had an antique version of LibreSSL on my > > > mac years ago, I just installed OpenSSL and never looked back. > > > > > > Gary > > > > > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <alex.rem...@gmail.com> > wrote: > > > > > > > > <The question is whether to do anything about the previously > undetected > > > > issues with JNA on macOS and Windows, and if so, whether this needs > to > > > > be done for the current or a later release. If we don't fix this > > > > release, obviously it needs some mention in the release notes.> > > > > > > > > I wouldn't characterize the issues running against LibreSSL as > > > > "undetected". I also don't see this as an issue with Mac or > Windows, but > > > > with LibreSSL. Install any supported OpenSSL library on any > supported > > > > architecture (to include Mac and Windows) and all commons crypto > > > > functionality is available. I first encountered the rand issue you > > > > describe years ago when I was becoming familiar with commons > crypto. I did > > > > a little research, discovered that I was running LibreSSL (and an old > > > > version at that), installed a supported version of OpenSSL and forgot > > > > all about it until this thread. I don't think it's unreasonable to > expect > > > > users to install a supported version of OpenSSL on a supported > architecture > > > > as a precondition of using commons crypto. I think it could become > > > > cumbersome to try and support every vendor default *SSL install. > That > > > > said, I don't have a problem committing this particular "fix" to the > > > > baseline, particularly since you have already done the work. I just > don't > > > > think that the project should obligate itself to do so or advertise > > > > LibreSSL (or any other non-OpenSSL branch) support as such. I'm > advocating > > > > a "use at your own risk" approach to anything but OpenSSL proper. I > agree > > > > that we should update the documentation to reflect whatever we move > forward > > > > with. > > > > > > > > Anyway, that's my $0.02 worth. > > > > > > > > Alex > > > > > > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <seb...@gmail.com> wrote: > > > > > > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <garydgreg...@gmail.com> > wrote: > > > > > > > > > > > > We cannot remove support for Windows and macOS, that's silly. > > > > > > > > > > AFAICT that means we must support the different set of function > names > > > > > in LibreSSL. > > > > > Note that we only currently use a small proportion of them. > > > > > > > > > > > I have not followed all the branches and commits, so I'm not > sure what > > > > > the > > > > > > current problems are, but I know I was able to release all OSs > last go > > > > > > around. I don't see why we need to rip out anything as > fundamental. > > > > > > > > > > Well, I have tried running the Crypto and OpenSslJna main classes > on > > > > > macOS and Windows, and they both fail with the 1.1.0 release. > > > > > > > > > > With current master, Crypto succeeds, but OpenSslJna fails to find > > > > > ENGINE_load_rdrand on both macOS and Windows. > > > > > (The job step succeeds, because the code catches the exception) > > > > > > > > > > It looks like the test which would have exposed at least one of the > > > > > issues was never enabled because of a failures on macOS; this hid > the > > > > > same problem on Windows. > > > > > > > > > > I am not suggesting we rip out anything at this point. > > > > > > > > > > The question is whether to do anything about the previously > undetected > > > > > issues with JNA on macOS and Windows, and if so, whether this > needs to > > > > > be done for the current or a later release. If we don't fix this > > > > > release, obviously it needs some mention in the release notes. > > > > > > > > > > Sebb > > > > > https://github.com/apache/commons-crypto/actions/runs/2586011129 > > > > > > Gary > > > > > > > > > > > > On Wed, Jun 29, 2022, 12:00 sebb <seb...@gmail.com> wrote: > > > > > > > > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily < > alex.rem...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > > I agree with Gary that we just don't support LibreSSL. To my > > > > > knowledge > > > > > > > > we've never advertised LibreSSL support, so I don't see it > as an > > > > > issue. > > > > > > > > > > > > > > In that case AFAICT we will have to drop *all* support for > macOS and > > > > > > > Windows, as they both seem to default to LibreSSL. > > > > > > > > > > > > > > Note that adding support for LibreSSL was much easier for JNI, > as it > > > > > > > uses far fewer methods. > > > > > > > Rather than checking the version, I changed the code to try > OpenSSL > > > > > > > 1.1 names first, then a fallback. > > > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL. > > > > > > > > > > > > > > If you want to try it out, compare 16345bc (old) with 3ee3f65 > (new) > > > > > > > macOS fails on 16345bc because it now uses LibreSSL which has a > > > > > > > different mix of names. > > > > > > > > > > > > > > I think it's vital we support JNI as far as possible (and the > code > > > > > > > already does with commit 3ee3f65). > > > > > > > > > > > > > > However JNA is more of a backstop, so the fact that it has > stopped > > > > > > > working for macOS and Windows is less of an issue. > > > > > > > > > > > > > > But I don't think we can say we are not supporting LibreSSL at > all. > > > > > > > > > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <seb...@gmail.com> > wrote: > > > > > > > > > > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory < > garydgreg...@gmail.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > The simple solution is to keep doing what we do now: > only support > > > > > > > OpenSSL > > > > > > > > > > and not whatever Apple does with LibreSSL which may or > may not > > > > > be up > > > > > > > to > > > > > > > > > > date. > > > > > > > > > > > > > > > > > > I think this also affects Windows. > > > > > > > > > > > > > > > > > > I don't have Windows box at present, but I have seen this > on GH > > > > > > > Actions. > > > > > > > > > > > > > > > > > > If you have a WIndows build, perhaps you could try these > tests: > > > > > > > > > > > > > > > > > > mvn -q exec:java > > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto" > > > > > > > > > mvn -q exec:java > > > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna" > > > > > > > > > > > > > > > > > > The first one should show the SSL details; on GH the output > > > > > includes: > > > > > > > > > > > > > > > > > > OpenSSL library loaded OK, version: 0x20000000 > > > > > > > > > OpenSSL library info: LibreSSL 3.0.2 > > > > > > > > > Additional OpenSSL_version(n) details: > > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl" > > > > > > > > > > > > > > > > > > The second test crashes with: > > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up function > > > > > > > > > 'ENGINE_load_rdrand': The specified procedure could not be > found. > > > > > > > > > isEnabled(): false > > > > > > > > > > > > > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError: > Error > > > > > looking > > > > > > > > > up function 'ENGINE_load_rdrand': The specified procedure > could > > > > > not be > > > > > > > > > found. > > > > > > > > > at com.sun.jna.Function.<init>(Function.java:252) > > > > > > > > > ... > > > > > > > > > > > > > > > > > > It would certainly be easier to ignore the problem for now. > > > > > > > > > > > > > > > > > > > Gary > > > > > > > > > > > > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <seb...@gmail.com> > wrote: > > > > > > > > > > > > > > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use > LibreSSL by > > > > > > > default > > > > > > > > > > > rather than OpenSSL. > > > > > > > > > > > > > > > > > > > > > > The LibreSSL API does not have the same functions as > either > > > > > 1.0.2 > > > > > > > or > > > > > > > > > > > 1.1.1, so needs its own JNA class. In particular it > looks like > > > > > > > > > > > ENGINE_load_rdrand is not present, nor is > OpenSSL_version_num; > > > > > > > 1.0.2 > > > > > > > > > > > has the former only, and 1.1.1 has the latter only. > > > > > > > > > > > > > > > > > > > > > > This makes it hard to support JNA with the current > design. > > > > > > > > > > > It would require another OpenSsl<version>NativeJna > class, and > > > > > the > > > > > > > > > > > parent class OpenSslNativeJna would need to use yet > another > > > > > > > condition > > > > > > > > > > > in each of its methods. > > > > > > > > > > > > > > > > > > > > > > This is quite tedious and error-prone. > > > > > > > > > > > > > > > > > > > > > > Seems to me it would be better to use something like a > set of > > > > > > > > > > > singleton classes that implement the required methods. > The > > > > > > > appropriate > > > > > > > > > > > one can then be initialised and used by > OpenSslNativeJna to > > > > > field > > > > > > > the > > > > > > > > > > > actual methods. i.e. replace the conditional logic > with a > > > > > static > > > > > > > > > > > reference to the correct API interface instance. This > should > > > > > only > > > > > > > > > > > affect non-public classes. > > > > > > > > > > > > > > > > > > > > > > Any other suggestions? > > > > > > > > > > > > > > > > > > > > > > Sebb > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > To unsubscribe, e-mail: > dev-unsubscr...@commons.apache.org > > > > > > > > > > > For additional commands, e-mail: > dev-h...@commons.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > > > > > > > > For additional commands, e-mail: > dev-h...@commons.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > > > > > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > > > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
