[
https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod KC updated SPARK-35747:
-----------------------------
Summary: Avoid printing full Exception stack trace, if HBase/Kafka/Hive
services are not running in a secure cluster (was: Avoid printing full
Exception stack trace, if HBase service is not running in a secure cluster )
> Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are
> not running in a secure cluster
> ------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-35747
> URL: https://issues.apache.org/jira/browse/SPARK-35747
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.0.0, 3.1.0, 3.1.2
> Reporter: Vinod KC
> Priority: Minor
>
> In a secure Yarn cluster where HBase service is down, even if the spark
> application is not using HBase, during the application submit stage,
> HBaseDelegationTokenProvider prints full Exception Stack trace and it causes
> a noisy warning.
>
> {code:java}
> WARN security.HBaseDelegationTokenProvider: Failed to get token from service
> hbase
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.security.HBaseDelegationTokenProvider.obtainDelegationTokensWithHBaseConn(HBaseDelegationT
> okenProvider.scala:93)
> more than 100 line exception stack trace{code}
> Also, Application submission taking more time as
> `HBaseDelegationTokenProvider.obtainDelegationTokensWithHBaseConn` retries
> the connection to HBase master multiple times before it gives up. This slows
> down the application submission steps. Hence, if HBase is not used in the
> user Application, it is better to suggest user to disable HBase Delegation
> Token generation.
> ie, spark.security.credentials.hbase.enabled=false
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
