[
https://issues.apache.org/jira/browse/HADOOP-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18089330#comment-18089330
]
ASF GitHub Bot commented on HADOOP-19872:
-----------------------------------------
pan3793 commented on PR #8550:
URL: https://github.com/apache/hadoop/pull/8550#issuecomment-4716761292
@rohit-kb, this part has been broken again and again. Actually, the SSL
ENGINE API is used solely to opt into Intel's RDRAND hardware RNG as OpenSSL's
default random source. And the claim of `OpensslSecureRandom`
> it's much faster than `java.security.SecureRandom`
is dated and should be qualified. "Much faster" was a reasonable statement
circa 2014 (HADOOP-10693) but is no longer a safe blanket claim on modern JDKs.
I'd like to request a re-evaluation of the performance on the modern JDK,
and if it does not have a performance advantage over
`java.security.SecureRandom`, just drop it and switch to the JDK's
implementation.
> Build failure with OpenSSL 3 (UBI10) in native code due to removed ENGINE API
> -----------------------------------------------------------------------------
>
> Key: HADOOP-19872
> URL: https://issues.apache.org/jira/browse/HADOOP-19872
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, native
> Affects Versions: 3.5.0, 3.4.3, 3.6.0
> Environment: # OS: Red Hat UBI10 container image
> # OpenSSL: 3.x (from UBI10) (package: openssl-devel-1:3.5.1-7.el10_1.x86_64)
> # Java: 11
> Reporter: rami
> Priority: Major
> Labels: pull-request-available
>
>
> Building Hadoop Common native code against OpenSSL 3 fails on UBI10 container
> images because the code uses legacy ENGINE APIs that have been removed by
> RedHat from the openssl-devel package. See the output from the build logs
> below.
>
> I want to also mention that UBI10 doesn't provide a package for the boost
> libraries anymore (in the epel10 repositories) so if anyone looks into this
> issue, they have to also come up with a workaround/fix for this problem.
>
> This is the relevant extract from the build logs:
> ```
> 727.1 [WARNING] make[2]: Leaving directory
> '/build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/target/native'
> 727.1 [WARNING] make[2]: Leaving directory
> '/build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/target/native'
> 727.1 [WARNING] make[1]: Leaving directory
> '/build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/target/native'
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/OpensslCipher.c:184:13:
> warning: 'loadSm4Ctr' defined but not used [-Wunused-function]
> 727.1 [WARNING] 184 | static void loadSm4Ctr(JNIEnv *env)
> 727.1 [WARNING] | ^~~~~~~~~~
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/OpensslCipher.c:184:13:
> warning: 'loadSm4Ctr' defined but not used [-Wunused-function]
> 727.1 [WARNING] 184 | static void loadSm4Ctr(JNIEnv *env)
> 727.1 [WARNING] | ^~~~~~~~~~
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:
> In function 'locks_setup':
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:250:33:
> warning: implicit declaration of function 'dlsym_CRYPTO_num_locks'; did you
> mean 'dlsym_CRYPTO_malloc'? [-Wimplicit-function-declaration]
> 727.1 [WARNING] 250 | lock_cs =
> dlsym_CRYPTO_malloc(dlsym_CRYPTO_num_locks() * \
> 727.1 [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~
> 727.1 [WARNING] | dlsym_CRYPTO_malloc
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:257:3:
> warning: implicit declaration of function 'dlsym_CRYPTO_set_id_callback';
> did you mean 'CRYPTO_set_id_callback'? [-Wimplicit-function-declaration]
> 727.1 [WARNING] 257 | dlsym_CRYPTO_set_id_callback((unsigned long
> (*)())pthreads_thread_id);
> 727.1 [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 727.1 [WARNING] | CRYPTO_set_id_callback
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:258:3:
> warning: implicit declaration of function
> 'dlsym_CRYPTO_set_locking_callback'; did you mean
> 'CRYPTO_set_locking_callback'? [-Wimplicit-function-declaration]
> 727.1 [WARNING] 258 | dlsym_CRYPTO_set_locking_callback((void
> (*)())pthreads_locking_callback);
> 727.1 [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 727.1 [WARNING] | CRYPTO_set_locking_callback
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:
> In function 'openssl_rand_init':
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:329:40:
> error: 'ENGINE_METHOD_RAND' undeclared (first use in this function)
> 727.1 [WARNING] 329 | rc = dlsym_ENGINE_set_default(eng,
> ENGINE_METHOD_RAND);
> 727.1 [WARNING] |
> ^~~~~~~~~~~~~~~~~~
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:329:40:
> note: each undeclared identifier is reported only once for each function it
> appears in
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:
> At top level:
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:261:13:
> warning: 'locks_cleanup' defined but not used [-Wunused-function]
> 727.1 [WARNING] 261 | static void locks_cleanup(void)
> 727.1 [WARNING] | ^~~~~~~~~~~~~
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:247:13:
> warning: 'locks_setup' defined but not used [-Wunused-function]
> 727.1 [WARNING] 247 | static void locks_setup(void)
> 727.1 [WARNING] | ^~~~~~~~~~~
> 727.1 [WARNING] make[2]: *** [CMakeFiles/hadoop_static.dir/build.make:104:
> CMakeFiles/hadoop_static.dir/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c.o]
> Error 1
> 727.1 [WARNING] make[2]: *** Waiting for unfinished jobs....
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:
> In function 'locks_setup':
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:250:33:
> warning: implicit declaration of function 'dlsym_CRYPTO_num_locks'; did you
> mean 'dlsym_CRYPTO_malloc'? [-Wimplicit-function-declaration]
> 727.1 [WARNING] 250 | lock_cs =
> dlsym_CRYPTO_malloc(dlsym_CRYPTO_num_locks() * \
> 727.1 [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~
> 727.1 [WARNING] | dlsym_CRYPTO_malloc
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:257:3:
> warning: implicit declaration of function 'dlsym_CRYPTO_set_id_callback';
> did you mean 'CRYPTO_set_id_callback'? [-Wimplicit-function-declaration]
> 727.1 [WARNING] 257 | dlsym_CRYPTO_set_id_callback((unsigned long
> (*)())pthreads_thread_id);
> 727.1 [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 727.1 [WARNING] | CRYPTO_set_id_callback
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:258:3:
> warning: implicit declaration of function
> 'dlsym_CRYPTO_set_locking_callback'; did you mean
> 'CRYPTO_set_locking_callback'? [-Wimplicit-function-declaration]
> 727.1 [WARNING] 258 | dlsym_CRYPTO_set_locking_callback((void
> (*)())pthreads_locking_callback);
> 727.1 [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 727.1 [WARNING] | CRYPTO_set_locking_callback
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:
> In function 'openssl_rand_init':
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:329:40:
> error: 'ENGINE_METHOD_RAND' undeclared (first use in this function)
> 727.1 [WARNING] 329 | rc = dlsym_ENGINE_set_default(eng,
> ENGINE_METHOD_RAND);
> 727.1 [WARNING] |
> ^~~~~~~~~~~~~~~~~~
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:329:40:
> note: each undeclared identifier is reported only once for each function it
> appears in
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:
> At top level:
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:261:13:
> warning: 'locks_cleanup' defined but not used [-Wunused-function]
> 727.1 [WARNING] 261 | static void locks_cleanup(void)
> 727.1 [WARNING] | ^~~~~~~~~~~~~
> 727.1 [WARNING]
> /build/src/hadoop/hadoop/patchable-work/worktree/3.4.2/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:247:13:
> warning: 'locks_setup' defined but not used [-Wunused-function]
> 727.1 [WARNING] 247 | static void locks_setup(void)
> 727.1 [WARNING] | ^~~~~~~~~~~
> 727.1 [WARNING] make[2]: *** [CMakeFiles/hadoop.dir/build.make:104:
> CMakeFiles/hadoop.dir/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c.o]
> Error 1
> 727.1 [WARNING] make[2]: *** Waiting for unfinished jobs....
> 727.1 [WARNING] make[1]: *** [CMakeFiles/Makefile2:113:
> CMakeFiles/hadoop_static.dir/all] Error 2
> 727.1 [WARNING] make[1]: *** Waiting for unfinished jobs....
> 727.1 [WARNING] make[1]: *** [CMakeFiles/Makefile2:87:
> CMakeFiles/hadoop.dir/all] Error 2
> 727.1 [WARNING] make: *** [Makefile:91: all] Error 2
> 727.1 [INFO]
> ------------------------------------------------------------------------
> ```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
