Multiple things, the output of checknative only contains these stuff
only, not everything. From the code [1], So looking at your command
output everything is sorted there barring OpenSSL & PMDK which you
explicitly didn't ask for in your maven command & I believe you don't
need them either, in case you need them the instructions are there in
[2]
Looking at the trace:
> org.apache.hadoop.io.compress.Lz4Codec.getLibraryName(Lz4Codec.java:73)
at
You mentioned building ver 3.3.6, Your exception trace is calling
getLibraryName, which isn't present in the Lz4Codec.java in ver-3.3.6
[3], this method got removed as part of HADOOP-17292 [4] that is post
hadoop ver 3.3.1+, So, If you read the release notes of this ticket
you can see for Hadoop-3.3.1+ the Lz4 thing works OOTB
So, mostly it isn't a Hadoop problem.
What could be possible is the HBase version that you are using is
pulling in an older Hadoop release which is messing things up, So, I
would say try using the hadoop-3 binary of the latest version 2.6.0
[5] & see how things go, else download the source tar of their latest
release 2.6.0 and build with -Phadoop-3.0
-Dhadoop-three.version=3.3.6, Looking at their source code they still
use 3.3.5 by default.
-Ayush
[1]
https://github.com/apache/hadoop/blob/1baf0e889fec54b6560417b62cada75daf6fe312/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java#L137-L144
[2] https://github.com/apache/hadoop/blob/branch-3.3.6/BUILDING.txt
[3]
https://github.com/apache/hadoop/blob/branch-3.3.6/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/Lz4Codec.java#L73
[4] https://issues.apache.org/jira/browse/HADOOP-17292
[5]
https://www.apache.org/dyn/closer.lua/hbase/2.6.0/hbase-2.6.0-hadoop3-bin.tar.gz
On Sat, 25 May 2024 at 22:41, <fe...@posteo.nl> wrote:
>
> Hey Ayush, thanks for the advice!
>
> Building 3.3.6 from an EL9.4 machine resulted in the following:
>
> [root@localhost bin]# JAVA_HOME=/etc/alternatives/java_sdk_openjdk/
> ./hadoop checknative -a
> 2024-05-25 19:05:56,068 INFO bzip2.Bzip2Factory: Successfully loaded &
> initialized native-bzip2 library system-native
> 2024-05-25 19:05:56,071 INFO zlib.ZlibFactory: Successfully loaded &
> initialized native-zlib library
> 2024-05-25 19:05:56,097 INFO nativeio.NativeIO: The native code was
> built without PMDK support.
> Native library checking:
> hadoop: true
> /root/build/hadoop-3.3.6-src/hadoop-dist/target/hadoop-3.3.6/lib/native/libhadoop.so.1.0.0
> zlib: true /lib64/libz.so.1
> zstd : true /lib64/libzstd.so.1
> bzip2: true /lib64/libbz2.so.1
> openssl: false EVP_CIPHER_CTX_block_size
> ISA-L: true /lib64/libisal.so.2
> PMDK: false The native code was built without PMDK support.
>
> No mention of lz4, though lz4[-devel] packages were installed on the
> compiling host as per the BUILDING instructions. Is there a build option
> I'm missing? I'm using:
>
> * mvn -X package -Pdist,native -DskipTests -Dtar -Dmaven.javadoc.skip=true
>
>
> Unfortunately the hbase "org.apache.hadoop.util.NativeLibraryChecker",
> using this freshly made hadoop native library, also failed to load lz4
> in the same way as the initial message with no extra information from debug:
>
> 2024-05-25T19:03:42,320 WARN [main] lz4.Lz4Compressor:
> java.lang.UnsatisfiedLinkError: 'void
> org.apache.hadoop.io.compress.lz4.Lz4Compressor.initIDs()'
> Exception in thread "main" java.lang.UnsatisfiedLinkError:
> 'java.lang.String
> org.apache.hadoop.io.compress.lz4.Lz4Compressor.getLibraryName()'
> at
> org.apache.hadoop.io.compress.lz4.Lz4Compressor.getLibraryName(Native
> Method)
> at
> org.apache.hadoop.io.compress.Lz4Codec.getLibraryName(Lz4Codec.java:73)
> at
> org.apache.hadoop.util.NativeLibraryChecker.main(NativeLibraryChecker.java:109)
>
> Thanks for your help!
>
>
> On 5/25/24 4:16 PM, Ayush Saxena wrote:
> > above things don't work then enable debug logging &
> > then run the checknative command and capture the log & exception as
> > here [2] & they might give you an an
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org
For additional commands, e-mail: user-h...@hadoop.apache.org
