[
https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-35747:
-------------------------------------
Assignee: Vinod KC
> Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are
> not running in a secure cluster
> ------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-35747
> URL: https://issues.apache.org/jira/browse/SPARK-35747
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.0.0, 3.1.0, 3.1.2
> Reporter: Vinod KC
> Assignee: Vinod KC
> Priority: Minor
>
> In a secure Yarn cluster, even though HBase or Kafka, or Hive services are
> not used in the user application, yarn client unnecessarily trying to
> generate Delegations token from these services. This will add additional
> delays while submitting spark application in a yarn cluster
> Also during HBase delegation, token generation step in the application
> submit stage, HBaseDelegationTokenProvider prints a full Exception Stack
> trace and it causes a noisy warning.
> {code:java}
> WARN security.HBaseDelegationTokenProvider: Failed to get token from service
> hbase
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.security.HBaseDelegationTokenProvider.obtainDelegationTokensWithHBaseConn(HBaseDelegationT
> okenProvider.scala:93)
> more than 100+ line exception stack trace{code}
> Hence, if these services are not used in the user Application, it is better
> add WARN message to disable Delegation Token generation for those services.
> ie, spark.security.credentials.hbase.enabled=false ,
> spark.security.credentials.hive.enabled=false ,
> spark.security.credentials.kafka.enabled=false
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
