[
https://issues.apache.org/jira/browse/ATLAS-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pinal Shah updated ATLAS-4941:
------------------------------
Description:
*Issue:* After compression type is set to SNAPPY, When Atlas is build locally,
it fails giving below console exception
*Exception:*
{code:java}
java.lang.RuntimeException: org.apache.atlas.AtlasException:
org.apache.hadoop.hbase.DoNotRetryIOException:
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z Set
hbase.table.sanity.checks to false at conf or table descriptor if you want to
bypass sanity checks
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:318)
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:33
Caused by: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method)
at
org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)
at
org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136)
{code}
*Workaround:*
Ensure Snappy compression libraries are available.
If not already installed, download and configure the Snappy library on your
system. (below was tried on Linux OS)
1. For hbase 2.3.3 download
[https://downloads.apache.org/hadoop/common/hadoop-3.2.4/hadoop-3.2.4.tar.gz]
2. mv hadoop-3.2.4.tar.gz /usr/local/
3. tar -zxvf hadoop-3.2.4.tar.gz
4. sudo apt install libsnappy-dev
5. add below configs in "hbase/conf/hbase-env.sh"
export HADOOP_HOME=/usr/local/hadoop-3.2.4
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native
*Aproach:*
This patch makes GZ as a default compression algorithm, SNAPPY can be set using
below property in atlas-application.properties
{code:java}
atlas.graph.storage.hbase.compression-algorithm=SNAPPY{code}
was:
*Issue:* After compression type is set to SNAPPY, When Atlas is build locally,
it fails giving below console exception
*Exception:*
{code:java}
java.lang.RuntimeException: org.apache.atlas.AtlasException:
org.apache.hadoop.hbase.DoNotRetryIOException:
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z Set
hbase.table.sanity.checks to false at conf or table descriptor if you want to
bypass sanity checks
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:318)
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:33
Caused by: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method)
at
org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)
at
org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136)
{code}
*Workaround:*
Ensure Snappy compression libraries are available.
If not already installed, download and configure the Snappy library on your
system. (below was tried on Linux OS)
1. For hbase 2.3.3 download
[https://downloads.apache.org/hadoop/common/hadoop-3.2.4/hadoop-3.2.4.tar.gz]
2. mv hadoop-3.2.4.tar.gz /usr/local/
3. tar -zxvf hadoop-3.2.4.tar.gz
4. sudo apt install libsnappy-dev
5. add below configs in "hbase/conf/hbase-env.sh"
export HADOOP_HOME=/usr/local/hadoop-3.2.4
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native
*Aproach:*
This patch makes GZ as a default compression algorithm, SNAPPY can be set using
below property in atlas-application.properties
atlas.graph.storage.hbase.compression-algorithm=SNAPPY
> Atlas fails to startup locally after compression type has changed to SNAPPY
> ----------------------------------------------------------------------------
>
> Key: ATLAS-4941
> URL: https://issues.apache.org/jira/browse/ATLAS-4941
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
> Reporter: Pinal Shah
> Assignee: Pinal Shah
> Priority: Major
> Fix For: 3.0.0, 2.4.0
>
>
> *Issue:* After compression type is set to SNAPPY, When Atlas is build
> locally, it fails giving below console exception
> *Exception:*
> {code:java}
> java.lang.RuntimeException: org.apache.atlas.AtlasException:
> org.apache.hadoop.hbase.DoNotRetryIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z Set
> hbase.table.sanity.checks to false at conf or table descriptor if you want to
> bypass sanity checks
> at
> org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:318)
> at
> org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:33
>
> Caused by: java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at
> org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method)
> at
> org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)
> at
> org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136)
> {code}
> *Workaround:*
> Ensure Snappy compression libraries are available.
> If not already installed, download and configure the Snappy library on your
> system. (below was tried on Linux OS)
> 1. For hbase 2.3.3 download
> [https://downloads.apache.org/hadoop/common/hadoop-3.2.4/hadoop-3.2.4.tar.gz]
> 2. mv hadoop-3.2.4.tar.gz /usr/local/
> 3. tar -zxvf hadoop-3.2.4.tar.gz
> 4. sudo apt install libsnappy-dev
> 5. add below configs in "hbase/conf/hbase-env.sh"
> export HADOOP_HOME=/usr/local/hadoop-3.2.4
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native
> *Aproach:*
> This patch makes GZ as a default compression algorithm, SNAPPY can be set
> using below property in atlas-application.properties
> {code:java}
> atlas.graph.storage.hbase.compression-algorithm=SNAPPY{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
