[
https://issues.apache.org/jira/browse/ATLAS-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nixon Rodrigues updated ATLAS-5325:
-----------------------------------
Description:
Problem Statement
Apache Atlas packaged using the embedded-hbase-solr profile fails to start
during initialization with the following exception:
java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKClientConfig(
Lorg/apache/hadoop/conf/Configuration;
)Lorg/apache/zookeeper/client/ZKClientConfig;
As a result, Atlas is unable to establish connectivity with HBase and startup
fails.
Environment
* Apache Atlas built using embedded-hbase-solr profile
* Embedded HBase
* Embedded Solr
* HBase 2.6.4 based runtime
Root Cause Analysis (RCA)
Investigation revealed a classpath conflict involving HBase shaded dependencies
packaged in the Atlas runtime distribution.
The runtime classpath contains:
hbase-shaded-mapreduce-2.6.4-hadoop3.jar
This artifact includes HBase classes whose method signatures differ from those
expected by the HBase client libraries used by Atlas.
Specifically, during initialization of HBase's ZKConnectionRegistry, Atlas
invokes:
org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKClientConfig(Configuration)
However, the version of ZKConfig resolved at runtime exposes an incompatible
method signature, resulting in:
java.lang.NoSuchMethodError
The exception occurs during creation of ReadOnlyZKClient and prevents Atlas
from completing startup.
Evidence
Bytecode inspection confirmed that the ZKConfig class packaged through the
shaded MapReduce dependency exposes a different API signature than the one
expected by Atlas runtime components.
Removing the conflicting dependency eliminates the NoSuchMethodError and allows
Atlas startup to proceed successfully.
Workaround
Remove the following jar from the Atlas runtime classpath:
hbase-shaded-mapreduce-2.6.4-hadoop3.jar
Example:
mv server/webapp/atlas/WEB-INF/lib/hbase-shaded-mapreduce-2.6.4-hadoop3.jar
/tmp/
After removing this dependency and restarting Atlas:
* Embedded HBase starts successfully
* Embedded Solr starts successfully
* Atlas initializes successfully
* Atlas UI becomes accessible
*Resolution*
Atlas startup failure was caused by a runtime classpath conflict introduced by
hbase-shaded-mapreduce-2.6.4-hadoop3.jar packaged in the embedded-hbase-solr
distribution. Removing the jar resolves the issue and allows Atlas to start
successfully.
was:
Problem Statement
Apache Atlas packaged using the embedded-hbase-solr profile fails to start
during initialization with the following exception:
java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKClientConfig(
Lorg/apache/hadoop/conf/Configuration;
)Lorg/apache/zookeeper/client/ZKClientConfig;
As a result, Atlas is unable to establish connectivity with HBase and startup
fails.
Environment
* Apache Atlas built using embedded-hbase-solr profile
* Embedded HBase
* Embedded Solr
* HBase 2.6.4 based runtime
Root Cause Analysis (RCA)
Investigation revealed a classpath conflict involving HBase shaded dependencies
packaged in the Atlas runtime distribution.
The runtime classpath contains:
hbase-shaded-mapreduce-2.6.4-hadoop3.jar
This artifact includes HBase classes whose method signatures differ from those
expected by the HBase client libraries used by Atlas.
Specifically, during initialization of HBase's ZKConnectionRegistry, Atlas
invokes:
org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKClientConfig(Configuration)
However, the version of ZKConfig resolved at runtime exposes an incompatible
method signature, resulting in:
java.lang.NoSuchMethodError
The exception occurs during creation of ReadOnlyZKClient and prevents Atlas
from completing startup.
Evidence
Bytecode inspection confirmed that the ZKConfig class packaged through the
shaded MapReduce dependency exposes a different API signature than the one
expected by Atlas runtime components.
Removing the conflicting dependency eliminates the NoSuchMethodError and allows
Atlas startup to proceed successfully.
Workaround
Remove the following jar from the Atlas runtime classpath:
hbase-shaded-mapreduce-2.6.4-hadoop3.jar
Example:
mv server/webapp/atlas/WEB-INF/lib/hbase-shaded-mapreduce-2.6.4-hadoop3.jar
/tmp/
After removing this dependency and restarting Atlas:
* Embedded HBase starts successfully
* Embedded Solr starts successfully
* Atlas initializes successfully
* Atlas UI becomes accessible
Possible Fixes
Option 1 (Recommended)
Exclude hbase-shaded-mapreduce-2.6.4-hadoop3.jar from the Atlas distribution
generated by the embedded-hbase-solr profile.
Pros:
* Minimal change
* Resolves startup failure
* Avoids runtime classpath conflicts
Option 2
Review the dependency tree for the embedded-hbase-solr profile and eliminate
duplicate or conflicting HBase shaded artifacts during packaging.
Pros:
* Prevents similar runtime linkage errors
* Produces a cleaner runtime classpath
Option 3
Introduce dependency management rules or exclusions to ensure a single
compatible implementation of HBase Zookeeper classes is present in the final
Atlas distribution.
Pros:
* Long-term maintainable solution
* Reduces risk of future dependency regressions
Resolution
Atlas startup failure was caused by a runtime classpath conflict introduced by
hbase-shaded-mapreduce-2.6.4-hadoop3.jar packaged in the embedded-hbase-solr
distribution. Removing the jar resolves the issue and allows Atlas to start
successfully.
> Atlas startup failure in embedded-hbase-solr profile due to conflicting HBase
> shaded MapReduce dependency
> ---------------------------------------------------------------------------------------------------------
>
> Key: ATLAS-5325
> URL: https://issues.apache.org/jira/browse/ATLAS-5325
> Project: Atlas
> Issue Type: Bug
> Affects Versions: trunk
> Reporter: Nixon Rodrigues
> Priority: Major
>
> Problem Statement
> Apache Atlas packaged using the embedded-hbase-solr profile fails to start
> during initialization with the following exception:
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKClientConfig(
> Lorg/apache/hadoop/conf/Configuration;
> )Lorg/apache/zookeeper/client/ZKClientConfig;
> As a result, Atlas is unable to establish connectivity with HBase and startup
> fails.
> Environment
> * Apache Atlas built using embedded-hbase-solr profile
> * Embedded HBase
> * Embedded Solr
> * HBase 2.6.4 based runtime
> Root Cause Analysis (RCA)
> Investigation revealed a classpath conflict involving HBase shaded
> dependencies packaged in the Atlas runtime distribution.
> The runtime classpath contains:
> hbase-shaded-mapreduce-2.6.4-hadoop3.jar
> This artifact includes HBase classes whose method signatures differ from
> those expected by the HBase client libraries used by Atlas.
> Specifically, during initialization of HBase's ZKConnectionRegistry, Atlas
> invokes:
> org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKClientConfig(Configuration)
> However, the version of ZKConfig resolved at runtime exposes an incompatible
> method signature, resulting in:
> java.lang.NoSuchMethodError
> The exception occurs during creation of ReadOnlyZKClient and prevents Atlas
> from completing startup.
> Evidence
> Bytecode inspection confirmed that the ZKConfig class packaged through the
> shaded MapReduce dependency exposes a different API signature than the one
> expected by Atlas runtime components.
> Removing the conflicting dependency eliminates the NoSuchMethodError and
> allows Atlas startup to proceed successfully.
> Workaround
> Remove the following jar from the Atlas runtime classpath:
> hbase-shaded-mapreduce-2.6.4-hadoop3.jar
> Example:
> mv server/webapp/atlas/WEB-INF/lib/hbase-shaded-mapreduce-2.6.4-hadoop3.jar
> /tmp/
> After removing this dependency and restarting Atlas:
> * Embedded HBase starts successfully
> * Embedded Solr starts successfully
> * Atlas initializes successfully
> * Atlas UI becomes accessible
>
> *Resolution*
> Atlas startup failure was caused by a runtime classpath conflict introduced
> by hbase-shaded-mapreduce-2.6.4-hadoop3.jar packaged in the
> embedded-hbase-solr distribution. Removing the jar resolves the issue and
> allows Atlas to start successfully.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
