Hi,
I'm working on upgrading Hadoop/HDFS 3.1.3 + HBase 2.2.2 to the latest
stable versions, 3.3.6 and 2.5.6 (downloaded from links on
hadoop/hbase.apache.org), in the docker images I am running a test
setup on, based on Ubuntu 20.04.
The containers with the HDFS Namenode and HDFS Datanode start up fine,
but when I try starting the HBase Regionserver I get:
10:23:52.522 [main] ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer - Failed construction
RegionServer
java.io.IOException: Compression codec snappy not supported, aborting RS
construction
at
org.apache.hadoop.hbase.regionserver.HRegionServer.checkCodecs(HRegionServer.java:915)
~[hbase-server-2.5.6-hadoop3.jar:2.5.6-hadoop3]
Which is confusing to me, as I see snappy jars and libraries all over
the container (the libsnappy-java Ubuntu package is installed):
root@6e2e1aff0e54:~# locate snappy | grep jar
/opt/base/hadoop-3.3.6/share/hadoop/common/lib/snappy-java-1.1.8.2.jar
/opt/base/hadoop-3.3.6/share/hadoop/hdfs/lib/snappy-java-1.1.8.2.jar
/opt/base/hbase-2.5.6-hadoop3/lib/hbase-compression-snappy-2.5.6-hadoop3.jar
/opt/base/hbase-2.5.6-hadoop3/lib/snappy-java-1.1.10.4.jar
/usr/share/java/snappy-java-1.1.7.3.jar
/usr/share/java/snappy-java.jar
/usr/share/maven-repo/org/xerial/snappy/snappy-java/1.1.7.3/snappy-java-1.1.7.3.jar
/usr/share/maven-repo/org/xerial/snappy/snappy-java/debian/snappy-java-debian.jar
root@6e2e1aff0e54:~# locate snappy | grep so
/usr/include/snappy-sinksource.h
/usr/lib/x86_64-linux-gnu/jni/libsnappyjava.so
/usr/lib/x86_64-linux-gnu/libsnappy.so
/usr/lib/x86_64-linux-gnu/libsnappy.so.1
/usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8
root@6e2e1aff0e54:~#
If I run "DEBUG=true /opt/hbase/bin/hbase-daemon.sh start master" I
see the snappy jars "hbase-compression-snappy-2.5.6-hadoop3.jar" and
"snappy-java-1.1.10.4.jar" mentioned in the classpath output.
If I install Hadoop 3.2.4 instead of 3.3.6, it works. However Hadoop
3.2 is end of life.
Should I be running different versions, compiling myself, install some
more packages, or what am I doing wrong?
Best regards,
Adam
--
"Although, in a sense, recognizing them as ancient Adam Sjøgren
might not necessarily be wrong, it's indeed [email protected]
useless."
