[
https://issues.apache.org/jira/browse/SPARK-3919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Armbrust resolved SPARK-3919.
-------------------------------------
Resolution: Not A Problem
Closing as not a problem. Configure Spark SQL to talk to a different metastore
version if desired.
> HiveThriftServer2 fails to start because of Hive 0.12 metastore schema
> verification failure
> -------------------------------------------------------------------------------------------
>
> Key: SPARK-3919
> URL: https://issues.apache.org/jira/browse/SPARK-3919
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.1.0
> Reporter: Cheng Lian
>
> When using MySQL backed Metastore with {{hive.metastore.schema.verification}}
> set to {{true}}, HiveThriftServer2 fails to start:
> {code}
> 14/10/12 17:05:01 ERROR HiveThriftServer2: Error starting HiveThriftServer2
> org.apache.hive.service.ServiceException: Failed to Start HiveServer2
> at
> org.apache.hive.service.CompositeService.start(CompositeService.java:80)
> at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:73)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:84)
> at
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:335)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> Caused by: org.apache.hive.service.ServiceException: Unable to connect to
> MetaStore!
> at org.apache.hive.service.cli.CLIService.start(CLIService.java:85)
> at
> org.apache.hive.service.CompositeService.start(CompositeService.java:70)
> ... 10 more
> Caused by: MetaException(message:Hive Schema version 0.12.0-protobuf-2.5 does
> not match metastore's schema version 0.12.0 Metastore is not upgraded or
> corrupt)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:5651)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:5622)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:124)
> at com.sun.proxy.$Proxy11.verifySchema(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:403)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:441)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:326)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:286)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:54)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4060)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:121)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:104)
> at org.apache.hive.service.cli.CLIService.start(CLIService.java:82)
> ... 11 more
> {code}
> Seems that recent Akka/Protobuf dependency changes are related to this.
> A valid workaround is to set {{hive.metastore.schema.verification}} to
> {{false}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
