-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75313/
-----------------------------------------------------------
(Updated Dec. 18, 2024, 4:12 p.m.)
Review request for atlas, Jayendra Parab, Madhan Neethiraj, and Vinayak
Marraiya.
Changes
-------
minor changes
Bugs: ATLAS-4941
https://issues.apache.org/jira/browse/ATLAS-4941
Repository: atlas
Description
-------
**Issue:** After compression type is set to SNAPPY, When Atlas is build
locally, it fails giving below console exception
**Exception:** java.lang.RuntimeException: org.apache.atlas.AtlasException:
org.apache.hadoop.hbase.DoNotRetryIOException:
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z Set
hbase.table.sanity.checks to false at conf or table descriptor if you want to
bypass sanity checks
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:318)
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:33
Caused by: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method)
at
org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)
at
org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136)
**Workaround:**
Ensure Snappy compression libraries are available.
If not already installed, download and configure the Snappy library on your
system. (below was tried on Linux OS)
1. For hbase 2.3.3 download
https://downloads.apache.org/hadoop/common/hadoop-3.2.4/hadoop-3.2.4.tar.gz
2. mv hadoop-3.2.4.tar.gz /usr/local/
3. tar -zxvf hadoop-3.2.4.tar.gz
4. sudo apt install libsnappy-dev
5. add below configs in "hbase/conf/hbase-env.sh"
export HADOOP_HOME=/usr/local/hadoop-3.2.4
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native
**Aproach:**
This patch makes GZ as a default compression algorithm, SNAPPY can be set using
below property in atlas-application.properties
atlas.graph.storage.hbase.compression-algorithm=SNAPPY
Diffs (updated)
-----
docs/src/documents/Hook/HookCouchbase.md 53b9e9cb3
docs/src/documents/Setup/InstallationInstruction.md de432e034
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
e0ecacc6d
Diff: https://reviews.apache.org/r/75313/diff/2/
Changes: https://reviews.apache.org/r/75313/diff/1-2/
Testing
-------
Precommit :
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1759
Thanks,
Pinal Shah
