YulongZ created SPARK-35875:
-------------------------------
Summary: SparkThriftServer can not start when HMS use DBTokenStore
Key: SPARK-35875
URL: https://issues.apache.org/jira/browse/SPARK-35875
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.1
Environment: Spark 3.1.1
Hive metastore 3.1.2
Reporter: YulongZ
CMD:
./sbin/start-thriftserver.sh --master=local --conf
spark.sql.hive.metastore.version=3.1.2 --conf
spark.sql.hive.metastore.jars=path --conf
spark.sql.hive.metastore.jars.path="file:///tmp/metastorejars/*.jar"
ERROR LOG:
21/06/24 16:43:09 ERROR HiveThriftServer2: Error starting HiveThriftServer2
org.apache.hive.service.ServiceException: Failed to Start HiveServer2
at
org.apache.hive.service.CompositeService.start(CompositeService.java:80)
at
org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:105)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.start(HiveThriftServer2.scala:154)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.hive.service.ServiceException: Error initializing
ThriftBinaryCLIService
at
org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.initializeServer(ThriftBinaryCLIService.java:113)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.start(ThriftCLIService.java:177)
at
org.apache.hive.service.CompositeService.start(CompositeService.java:70)
... 17 more
Caused by: org.apache.thrift.transport.TTransportException: Failed to start
token manager
at
org.apache.hive.service.auth.HiveAuthFactory.<init>(HiveAuthFactory.java:167)
at
org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.initializeServer(ThriftBinaryCLIService.java:66)
... 19 more
Caused by: java.io.IOException: Failed to initialize master key
at
org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager.startThreads(TokenStoreDelegationTokenSecretManager.java:213)
at
org.apache.hadoop.hive.thrift.HiveDelegationTokenManager.startDelegationTokenSecretManager(HiveDelegationTokenManager.java:97)
at
org.apache.hive.service.auth.HiveAuthFactory.<init>(HiveAuthFactory.java:162)
... 20 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager.startThreads(TokenStoreDelegationTokenSecretManager.java:211)
... 22 more
Caused by: java.lang.ClassCastException: com.sun.proxy.$Proxy45 cannot be cast
to java.lang.Class
at
org.apache.hadoop.hive.thrift.DBTokenStore.invokeOnTokenStore(DBTokenStore.java:154)
at
org.apache.hadoop.hive.thrift.DBTokenStore.addMasterKey(DBTokenStore.java:43)
at
org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager.logUpdateMasterKey(TokenStoreDelegationTokenSecretManager.java:193)
at
org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.updateCurrentKey(AbstractDelegationTokenSecretManager.java:364)
... 27 more
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
