Re: Spark SQL is not able to connect to hive metastore
Hi Ayan, Thanks for your response. In my case the constraint is I have to use Hive 0.14 for some other usecases. I believe the incompatibility is at the thrift server level (the hiveserver 2 which comes with hive). If I use Hive 0.13 hiverserver 2 in the same node as of spark master should that help ? Regards, Sourav On Sat, May 16, 2015 at 8:48 AM, ayan guha guha.a...@gmail.com wrote: Hi Try with Hive 0.13. If I am not wrong, Hive 0.14 is not supported yet, definitely not with 1.2.1 :) On Sun, May 17, 2015 at 1:14 AM, smazumder sourav.mazumde...@gmail.com wrote: HI, I'm trying to execute simple sql statement from spark-shell val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc) - This one executes properly. Next I'm trying - sqlContext.sql(CREATE TABLE IF NOT EXISTS src (key INT, value STRING)) This keeps on trying to connect Metastore but could nener. The error I'm getting is - Trying to connect to metastore with URI thrift://host-name.com:9083 set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it. org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset Any idea ? I'm using Spark 1.2.1 and Hive is 0.14. Regards, Sourav -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-is-not-able-to-connect-to-hive-metastore-tp22916.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org -- Best Regards, Ayan Guha
Spark SQL is not able to connect to hive metastore
HI, I'm trying to execute simple sql statement from spark-shell val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc) - This one executes properly. Next I'm trying - sqlContext.sql(CREATE TABLE IF NOT EXISTS src (key INT, value STRING)) This keeps on trying to connect Metastore but could nener. The error I'm getting is - Trying to connect to metastore with URI thrift://host-name.com:9083 set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it. org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset Any idea ? I'm using Spark 1.2.1 and Hive is 0.14. Regards, Sourav -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-is-not-able-to-connect-to-hive-metastore-tp22916.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Spark SQL is not able to connect to hive metastore
Hi Try with Hive 0.13. If I am not wrong, Hive 0.14 is not supported yet, definitely not with 1.2.1 :) On Sun, May 17, 2015 at 1:14 AM, smazumder sourav.mazumde...@gmail.com wrote: HI, I'm trying to execute simple sql statement from spark-shell val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc) - This one executes properly. Next I'm trying - sqlContext.sql(CREATE TABLE IF NOT EXISTS src (key INT, value STRING)) This keeps on trying to connect Metastore but could nener. The error I'm getting is - Trying to connect to metastore with URI thrift://host-name.com:9083 set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it. org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset Any idea ? I'm using Spark 1.2.1 and Hive is 0.14. Regards, Sourav -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-is-not-able-to-connect-to-hive-metastore-tp22916.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org -- Best Regards, Ayan Guha
Re: Spark SQL is not able to connect to hive metastore
Here is from documentation: Spark SQL is designed to be compatible with the Hive Metastore, SerDes and UDFs. Currently Spark SQL is based on Hive 0.12.0 and 0.13.1. On Sun, May 17, 2015 at 1:48 AM, ayan guha guha.a...@gmail.com wrote: Hi Try with Hive 0.13. If I am not wrong, Hive 0.14 is not supported yet, definitely not with 1.2.1 :) On Sun, May 17, 2015 at 1:14 AM, smazumder sourav.mazumde...@gmail.com wrote: HI, I'm trying to execute simple sql statement from spark-shell val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc) - This one executes properly. Next I'm trying - sqlContext.sql(CREATE TABLE IF NOT EXISTS src (key INT, value STRING)) This keeps on trying to connect Metastore but could nener. The error I'm getting is - Trying to connect to metastore with URI thrift://host-name.com:9083 set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it. org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset Any idea ? I'm using Spark 1.2.1 and Hive is 0.14. Regards, Sourav -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-is-not-able-to-connect-to-hive-metastore-tp22916.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org -- Best Regards, Ayan Guha -- Best Regards, Ayan Guha
