I downloaded different versions of the jars and it worked. Thanks!
On Tue, Mar 3, 2015 at 4:45 PM, Cheng, Hao <hao.ch...@intel.com> wrote: > Which version / distribution are you using? Please references this blog > that Felix C posted if you’re running on CDH. > > > http://eradiating.wordpress.com/2015/02/22/getting-hivecontext-to-work-in-cdh/ > > > > Or you may also need to download the datanucleus*.jar files try to add the > option of “--jars” while starting the spark shell. > > > > *From:* Anusha Shamanur [mailto:anushas...@gmail.com] > *Sent:* Wednesday, March 4, 2015 5:07 AM > *To:* Cheng, Hao > *Subject:* Re: Spark SQL Thrift Server start exception : > java.lang.ClassNotFoundException: > org.datanucleus.api.jdo.JDOPersistenceManagerFactory > > > > Hi, > > > > I am getting the same error. There is no lib folder in my $SPARK_HOME. But > I included these jars while calling spark-shell. > > > > Now, I get this: > > Caused by: org.datanucleus.exceptions.ClassNotResolvedException: Class > "org.datanucleus.store.rdbms.RDBMSStoreManager" was not found in the > CLASSPATH. Please check your specification and your CLASSPATH. > > at > org.datanucleus.ClassLoaderResolverImpl.classForName(ClassLoaderResolverImpl.java:218) > > > > How do I solve this? > > > > On Mon, Mar 2, 2015 at 11:04 PM, Cheng, Hao <hao.ch...@intel.com> wrote: > > Copy those jars into the $SPARK_HOME/lib/ > > datanucleus-api-jdo-3.2.6.jar > datanucleus-core-3.2.10.jar > datanucleus-rdbms-3.2.9.jar > > see > https://github.com/apache/spark/blob/master/bin/compute-classpath.sh#L120 > > > > -----Original Message----- > From: fanooos [mailto:dev.fano...@gmail.com] > Sent: Tuesday, March 3, 2015 2:50 PM > To: user@spark.apache.org > Subject: Spark SQL Thrift Server start exception : > java.lang.ClassNotFoundException: > org.datanucleus.api.jdo.JDOPersistenceManagerFactory > > I have installed a hadoop cluster (version : 2.6.0), apache spark (version > : > 1.2.1 preBuilt for hadoop 2.4 and later), and hive (version 1.0.0). > > When I try to start the spark sql thrift server I am getting the following > exception. > > Exception in thread "main" java.lang.RuntimeException: > java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.metastore.HiveMetaStoreClient > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346) > at > > org.apache.spark.sql.hive.HiveContext$$anonfun$4.apply(HiveContext.scala:235) > at > > org.apache.spark.sql.hive.HiveContext$$anonfun$4.apply(HiveContext.scala:231) > at scala.Option.orElse(Option.scala:257) > at > org.apache.spark.sql.hive.HiveContext.x$3$lzycompute(HiveContext.scala:231) > at org.apache.spark.sql.hive.HiveContext.x$3(HiveContext.scala:229) > at > > org.apache.spark.sql.hive.HiveContext.hiveconf$lzycompute(HiveContext.scala:229) > at > org.apache.spark.sql.hive.HiveContext.hiveconf(HiveContext.scala:229) > at > org.apache.spark.sql.hive.HiveContext.runHive(HiveContext.scala:292) > at > org.apache.spark.sql.hive.HiveContext.runSqlHive(HiveContext.scala:276) > at > org.apache.spark.sql.hive.HiveContext.setConf(HiveContext.scala:248) > at > org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:91) > at > org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:90) > at > > scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) > at > scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108) > at org.apache.spark.sql.SQLContext.<init>(SQLContext.scala:90) > at > org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:72) > at > > org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:51) > at > > org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:56) > at > > org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.metastore.HiveMetaStoreClient > at > > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1412) > at > > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62) > at > > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72) > at > > org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453) > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340) > ... 26 more > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1410) > ... 31 more > Caused by: javax.jdo.JDOFatalUserException: Class > org.datanucleus.api.jdo.JDOPersistenceManagerFactory was not found. > NestedThrowables: > java.lang.ClassNotFoundException: > org.datanucleus.api.jdo.JDOPersistenceManagerFactory > at > > javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1175) > at > javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808) > at > javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701) > at > org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:310) > at > > org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:339) > at > > org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:248) > at > org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:223) > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) > at > > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) > at > > org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) > at > > org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) > at > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:497) > at > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:475) > at > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:523) > at > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:397) > at > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:356) > at > > org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:54) > at > > org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59) > at > > org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4944) > at > > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:171) > ... 36 more > Caused by: java.lang.ClassNotFoundException: > org.datanucleus.api.jdo.JDOPersistenceManagerFactory > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:274) > at javax.jdo.JDOHelper$18.run(JDOHelper.java:2018) > at javax.jdo.JDOHelper$18.run(JDOHelper.java:2016) > at java.security.AccessController.doPrivileged(Native Method) > at javax.jdo.JDOHelper.forName(JDOHelper.java:2015) > at > > javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1162) > ... 55 more > > As I understand from the exception, the datanucleus jars is not in the > ClassPath so I added the following line to my .profile > > export $SPARK_OPTS="-Djava.library.path=$SPARK_HOME/lib" > > but the same exception still thrown and the server does not run. > > How can I solve this issue ? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Thrift-Server-start-exception-java-lang-ClassNotFoundException-org-datanucleus-api-jdo-JDOy-tp21887.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional > commands, e-mail: user-h...@spark.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > > > > > > -- > > Regards, > > Anusha > -- Regards, Anusha