Hi Roc,
Spark SQL 1.2.0 can only work with Hive 0.12.0 or Hive 0.13.1
(controlled by compilation flags), versions prior 1.2.0 only works with
Hive 0.12.0. So Hive 0.15.0-SNAPSHOT is not an option.
Would like to add that this is due to backwards compatibility issue of
Hive metastore, AFAIK there's no popular system works with multiple
versions of Hive metastore simultaneously.
Cheng
On 12/24/14 6:31 PM, Roc Chu wrote:
this is my problem. I use mysql to store hive meta data. and i can get
what i want when I exec show tables in hive shell. but in the same
machine. I use spark-sql to execute same command (show tables), I got
errors.
I look at the log of hive metastore find this errors
2014-12-24 05:04:59,874 ERROR [pool-3-thread-2]:
server.TThreadPoolServer (TThreadPoolServer.java:run(294)) - Thrift
error occurred during processing of message.
org.apache.thrift.protocol.TProtocolException: Expected protocol id
ffffff82 but got ffffff80
at
org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:503)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:75)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-12-24 05:05:01,883 ERROR [pool-3-thread-3]:
server.TThreadPoolServer (TThreadPoolServer.java:run(294)) - Thrift
error occurred during processing of message.
org.apache.thrift.protocol.TProtocolException: Expected protocol id
ffffff82 but got ffffff80
at
org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:503)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:75)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
I compiled e spark from soucecode of spark-1.1.1
and my hive was compiled from apache-hive-0.15.0
I put the same hive-site.xml both hive conf and apark conf.
waiting for help
thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]