Re: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
which location should i need to specify the classpath exactly . Thanks, On Mon, Mar 16, 2015 at 12:52 PM, Cheng, Hao hao.ch...@intel.com wrote: It doesn’t take effect if just putting jar files under the lib-managed/jars folder, you need to put that under class path explicitly. *From:* sandeep vura [mailto:sandeepv...@gmail.com] *Sent:* Monday, March 16, 2015 2:21 PM *To:* Cheng, Hao *Cc:* fightf...@163.com; Ted Yu; user *Subject:* Re: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient I have already added mysql-connector-xx.jar file in spark/lib-managed/jars directory. Regards, Sandeep.v On Mon, Mar 16, 2015 at 11:48 AM, Cheng, Hao hao.ch...@intel.com wrote: Or you need to specify the jars either in configuration or bin/spark-sql --jars mysql-connector-xx.jar *From:* fightf...@163.com [mailto:fightf...@163.com] *Sent:* Monday, March 16, 2015 2:04 PM *To:* sandeep vura; Ted Yu *Cc:* user *Subject:* Re: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient Hi, Sandeep From your error log I can see that jdbc driver not found in your classpath. Did you had your mysql jdbc jar correctly configured in the specific classpath? Can you establish a hive jdbc connection using the url : jdbc:hive2://localhost:1 ? Thanks, Sun. -- fightf...@163.com *From:* sandeep vura sandeepv...@gmail.com *Date:* 2015-03-16 14:13 *To:* Ted Yu yuzhih...@gmail.com *CC:* user@spark.apache.org *Subject:* Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient Hi Ted, Did you find any solution. Thanks Sandeep On Mon, Mar 16, 2015 at 10:44 AM, sandeep vura sandeepv...@gmail.com wrote: Hi Ted, I am using Spark -1.2.1 and hive -0.13.1 you can check my configuration files attached below. ERROR IN SPARK n: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav a:346) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkS QLCLIDriver.scala:101) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQ LCLIDriver.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:622) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.h ive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStore Utils.java:1412) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.init(Retry ingMetaStoreClient.java:62) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(Ret ryingMetaStoreClient.java:72) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.ja va:2453) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav a:340) ... 9 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct orAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC onstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:534) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStore Utils.java:1410) ... 14 more Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional con nection factory NestedThrowables: java.lang.reflect.InvocationTargetException at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusExc eption(NucleusJDOHelper.java:587) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfigurat ion(JDOPersistenceManagerFactory.java:788) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenc eManagerFactory(JDOPersistenceManagerFactory.java:333
Re: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
Hi Fightfate, I have attached my hive-site.xml file in the previous mail.Please check the configuration once. In hive i am able to create tables and also able to load data into hive table. Please find the attached file. Regards, Sandeep.v On Mon, Mar 16, 2015 at 11:34 AM, fightf...@163.com fightf...@163.com wrote: Hi, Sandeep From your error log I can see that jdbc driver not found in your classpath. Did you had your mysql jdbc jar correctly configured in the specific classpath? Can you establish a hive jdbc connection using the url : jdbc:hive2://localhost:1 ? Thanks, Sun. -- fightf...@163.com *From:* sandeep vura sandeepv...@gmail.com *Date:* 2015-03-16 14:13 *To:* Ted Yu yuzhih...@gmail.com *CC:* user@spark.apache.org *Subject:* Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient Hi Ted, Did you find any solution. Thanks Sandeep On Mon, Mar 16, 2015 at 10:44 AM, sandeep vura sandeepv...@gmail.com wrote: Hi Ted, I am using Spark -1.2.1 and hive -0.13.1 you can check my configuration files attached below. ERROR IN SPARK n: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav a:346) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkS QLCLIDriver.scala:101) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQ LCLIDriver.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:622) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.h ive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStore Utils.java:1412) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.init(Retry ingMetaStoreClient.java:62) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(Ret ryingMetaStoreClient.java:72) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.ja va:2453) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav a:340) ... 9 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct orAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC onstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:534) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStore Utils.java:1410) ... 14 more Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional con nection factory NestedThrowables: java.lang.reflect.InvocationTargetException at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusExc eption(NucleusJDOHelper.java:587) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfigurat ion(JDOPersistenceManagerFactory.java:788) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenc eManagerFactory(JDOPersistenceManagerFactory.java:333) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceMa nagerFactory(JDOPersistenceManagerFactory.java:202) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:622) at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965) at java.security.AccessController.doPrivileged(Native Method) at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960) at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementatio n(JDOHelper.java:1166) at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java
RE: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
Or you need to specify the jars either in configuration or bin/spark-sql --jars mysql-connector-xx.jar From: fightf...@163.com [mailto:fightf...@163.com] Sent: Monday, March 16, 2015 2:04 PM To: sandeep vura; Ted Yu Cc: user Subject: Re: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient Hi, Sandeep From your error log I can see that jdbc driver not found in your classpath. Did you had your mysql jdbc jar correctly configured in the specific classpath? Can you establish a hive jdbc connection using the url : jdbc:hive2://localhost:1 ? Thanks, Sun. fightf...@163.commailto:fightf...@163.com From: sandeep vuramailto:sandeepv...@gmail.com Date: 2015-03-16 14:13 To: Ted Yumailto:yuzhih...@gmail.com CC: user@spark.apache.orgmailto:user@spark.apache.org Subject: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient Hi Ted, Did you find any solution. Thanks Sandeep On Mon, Mar 16, 2015 at 10:44 AM, sandeep vura sandeepv...@gmail.commailto:sandeepv...@gmail.com wrote: Hi Ted, I am using Spark -1.2.1 and hive -0.13.1 you can check my configuration files attached below. ERROR IN SPARK n: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav a:346) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkS QLCLIDriver.scala:101) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQ LCLIDriver.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:622) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.h ive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStore Utils.java:1412) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.init(Retry ingMetaStoreClient.java:62) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(Ret ryingMetaStoreClient.java:72) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.ja va:2453) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav a:340) ... 9 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct orAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC onstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:534) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStore Utils.java:1410) ... 14 more Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional con nection factory NestedThrowables: java.lang.reflect.InvocationTargetException
Re: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
I have already added mysql-connector-xx.jar file in spark/lib-managed/jars directory. Regards, Sandeep.v On Mon, Mar 16, 2015 at 11:48 AM, Cheng, Hao hao.ch...@intel.com wrote: Or you need to specify the jars either in configuration or bin/spark-sql --jars mysql-connector-xx.jar *From:* fightf...@163.com [mailto:fightf...@163.com] *Sent:* Monday, March 16, 2015 2:04 PM *To:* sandeep vura; Ted Yu *Cc:* user *Subject:* Re: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient Hi, Sandeep From your error log I can see that jdbc driver not found in your classpath. Did you had your mysql jdbc jar correctly configured in the specific classpath? Can you establish a hive jdbc connection using the url : jdbc:hive2://localhost:1 ? Thanks, Sun. -- fightf...@163.com *From:* sandeep vura sandeepv...@gmail.com *Date:* 2015-03-16 14:13 *To:* Ted Yu yuzhih...@gmail.com *CC:* user@spark.apache.org *Subject:* Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient Hi Ted, Did you find any solution. Thanks Sandeep On Mon, Mar 16, 2015 at 10:44 AM, sandeep vura sandeepv...@gmail.com wrote: Hi Ted, I am using Spark -1.2.1 and hive -0.13.1 you can check my configuration files attached below. ERROR IN SPARK n: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav a:346) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkS QLCLIDriver.scala:101) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQ LCLIDriver.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:622) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.h ive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStore Utils.java:1412) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.init(Retry ingMetaStoreClient.java:62) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(Ret ryingMetaStoreClient.java:72) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.ja va:2453) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav a:340) ... 9 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct orAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC onstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:534) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStore Utils.java:1410) ... 14 more Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional con nection factory NestedThrowables: java.lang.reflect.InvocationTargetException at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusExc eption(NucleusJDOHelper.java:587) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfigurat ion(JDOPersistenceManagerFactory.java:788) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenc eManagerFactory(JDOPersistenceManagerFactory.java:333) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceMa nagerFactory(JDOPersistenceManagerFactory.java:202) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:622) at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965
RE: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
It doesn’t take effect if just putting jar files under the lib-managed/jars folder, you need to put that under class path explicitly. From: sandeep vura [mailto:sandeepv...@gmail.com] Sent: Monday, March 16, 2015 2:21 PM To: Cheng, Hao Cc: fightf...@163.com; Ted Yu; user Subject: Re: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient I have already added mysql-connector-xx.jar file in spark/lib-managed/jars directory. Regards, Sandeep.v On Mon, Mar 16, 2015 at 11:48 AM, Cheng, Hao hao.ch...@intel.commailto:hao.ch...@intel.com wrote: Or you need to specify the jars either in configuration or bin/spark-sql --jars mysql-connector-xx.jar From: fightf...@163.commailto:fightf...@163.com [mailto:fightf...@163.commailto:fightf...@163.com] Sent: Monday, March 16, 2015 2:04 PM To: sandeep vura; Ted Yu Cc: user Subject: Re: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient Hi, Sandeep From your error log I can see that jdbc driver not found in your classpath. Did you had your mysql jdbc jar correctly configured in the specific classpath? Can you establish a hive jdbc connection using the url : jdbc:hive2://localhost:1 ? Thanks, Sun. fightf...@163.commailto:fightf...@163.com From: sandeep vuramailto:sandeepv...@gmail.com Date: 2015-03-16 14:13 To: Ted Yumailto:yuzhih...@gmail.com CC: user@spark.apache.orgmailto:user@spark.apache.org Subject: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient Hi Ted, Did you find any solution. Thanks Sandeep On Mon, Mar 16, 2015 at 10:44 AM, sandeep vura sandeepv...@gmail.commailto:sandeepv...@gmail.com wrote: Hi Ted, I am using Spark -1.2.1 and hive -0.13.1 you can check my configuration files attached below. ERROR IN SPARK n: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav a:346) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkS QLCLIDriver.scala:101) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQ LCLIDriver.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:622) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.h ive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStore Utils.java:1412) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.init(Retry ingMetaStoreClient.java:62) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(Ret ryingMetaStoreClient.java:72) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.ja va:2453) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav a:340) ... 9 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct orAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC
