[jira] [Commented] (SPARK-24843) Spark2 job (in cluster mode) is unable to execute steps in HBase (error# java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/CompatibilityFactory)
[
https://issues.apache.org/jira/browse/SPARK-24843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547774#comment-16547774
]
Manish commented on SPARK-24843:
Thanks Wang.
I am setting it using export command before firing spark2-submit. It works
perfectly fine with client mode but not working in cluster mode. Any leads will
be very helpful to me.
{color:#205081}export
HADOOP_CONF_DIR=/etc/hadoop/conf:/etc/hive/conf:/etc/hbase/conf:/opt/cloudera/parcels/CDH/lib/hbase/lib/hbase-common-1.2.0-cdh5.11.1.jar:/home/svc-cop-realtime-d/scala1/jar/lib/hbase-rdd_2.11-0.8.0.jar:/opt/cloudera/parcels/CDH/lib/hbase/lib/hbase-hadoop2-compat-1.2.0-cdh5.11.1.jar{color}
> Spark2 job (in cluster mode) is unable to execute steps in HBase (error#
> java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/CompatibilityFactory)
> --
>
> Key: SPARK-24843
> URL: https://issues.apache.org/jira/browse/SPARK-24843
> Project: Spark
> Issue Type: Bug
> Components: Build, Java API
>Affects Versions: 2.1.0
>Reporter: Manish
>Priority: Major
>
> I am running Spark2 streaming job to do processing in HBase. It wokrs
> perfectly fine with client deploy mode but don't work with deploy mode as
> cluster . Below is the error message:
> |{color:#ff}_User class threw exception: java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/CompatibilityFactory_{color}|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24843) Spark2 job (in cluster mode) is unable to execute steps in HBase (error# java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/CompatibilityFactory)
[
https://issues.apache.org/jira/browse/SPARK-24843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547674#comment-16547674
]
Yuming Wang commented on SPARK-24843:
-
It's not a Spark bug. Please make sure you classpath contains
{{hbase-hadoop-compat*.jar}}
> Spark2 job (in cluster mode) is unable to execute steps in HBase (error#
> java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/CompatibilityFactory)
> --
>
> Key: SPARK-24843
> URL: https://issues.apache.org/jira/browse/SPARK-24843
> Project: Spark
> Issue Type: Bug
> Components: Build, Java API
>Affects Versions: 2.1.0
>Reporter: Manish
>Priority: Major
>
> I am running Spark2 streaming job to do processing in HBase. It wokrs
> perfectly fine with client deploy mode but don't work with deploy mode as
> cluster . Below is the error message:
> |{color:#ff}_User class threw exception: java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/CompatibilityFactory_{color}|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
