Vinod KC created SPARK-35747: -------------------------------- Summary: Avoid printing full Exception stack trace, if HBase service is not running in a secure cluster Key: SPARK-35747 URL: https://issues.apache.org/jira/browse/SPARK-35747 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 3.1.2, 3.1.0, 3.0.0 Reporter: Vinod KC
In a secure Yarn cluster where HBase service is down, even if the spark application is not using HBase, during the application submit stage, HBaseDelegationTokenProvider prints full Exception Stack trace and it causes a noisy warning. {code:java} WARN security.HBaseDelegationTokenProvider: Failed to get token from service hbase java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.security.HBaseDelegationTokenProvider.obtainDelegationTokensWithHBaseConn(HBaseDelegationT okenProvider.scala:93) {code} Also, Application submission taking more time as `HBaseDelegationTokenProvider.obtainDelegationTokensWithHBaseConn` retries the connection to HBase master multiple times before it gives up. This slows down the application submission steps. Hence, if HBase is not used in the user Application, it is better to suggest user to disable HBase Delegation Token generation. ie, spark.security.credentials.hbase.enabled=false -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org