That worked perfectly...thanks so much!
On Fri, Feb 20, 2015 at 3:49 PM, Sourigna Phetsarath <
gna.phetsar...@teamaol.com> wrote:
> Correction,
>
> should be HADOOP_CONF_DIR="/etc/hive/conf" spark-shell --driver-class-path
> '/data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/*'
> --driver-java-options '-Dspark.executor.extraClassPath=/opt/cloudera/
> parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/*'
>
> On Fri, Feb 20, 2015 at 3:48 PM, Sourigna Phetsarath <
> gna.phetsar...@teamaol.com> wrote:
>
>> Correction,
>>
>> should be HADOOP_CONF_DIR="/etc/hive/conf" --driver-class-path
>> '/data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/*'
>> --driver-java-options '-Dspark.executor.extraClassPath=/opt/cloudera/
>> parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/*'
>>
>> On Fri, Feb 20, 2015 at 3:43 PM, Sourigna Phetsarath <
>> gna.phetsar...@teamaol.com> wrote:
>>
>>> Also, you might want to add the hadoop configs:
>>>
>>> HADOOP_CONF_DIR="/etc/hadoop/conf:/etc/hive/conf" --driver-class-path
>>> '/data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/*'
>>> --driver-java-options '-Dspark.executor.extraClassPath=/opt/cloudera/
>>> parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/*'
>>>
>>> Ok. It needs the CDH configs for hive and hadoop. Hopefully this works
>>> for you.
>>>
>>>
>>>
>>> On Fri, Feb 20, 2015 at 3:41 PM, chirag lakhani <
>>> chirag.lakh...@gmail.com> wrote:
>>>
>>>> Thanks! I am able to login to Spark now but I am still getting the
>>>> same error
>>>>
>>>> scala> sqlContext.sql("FROM analytics.trainingdatafinal SELECT
>>>> *").collect().foreach(println)
>>>> 15/02/20 14:40:22 INFO ParseDriver: Parsing command: FROM
>>>> analytics.trainingdatafinal SELECT *
>>>> 15/02/20 14:40:22 INFO ParseDriver: Parse Completed
>>>> 15/02/20 14:40:23 INFO HiveMetaStore: 0: Opening raw store with
>>>> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
>>>> 15/02/20 14:40:23 INFO ObjectStore: ObjectStore, initialize called
>>>> 15/02/20 14:40:23 WARN General: Plugin (Bundle)
>>>> "org.datanucleus.api.jdo" is already registered. Ensure you dont have
>>>> multiple JAR versions of the same plugin in the classpath. The URL
>>>> "file:/data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/jars/datanucleus-api-jdo-3.2.6.jar"
>>>> is already registered, and you are trying to register an identical plugin
>>>> located at URL
>>>> "file:/data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/datanucleus-api-jdo-3.2.6.jar."
>>>> 15/02/20 14:40:23 WARN General: Plugin (Bundle)
>>>> "org.datanucleus.store.rdbms" is already registered. Ensure you dont have
>>>> multiple JAR versions of the same plugin in the classpath. The URL
>>>> "file:/data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/jars/datanucleus-rdbms-3.2.9.jar"
>>>> is already registered, and you are trying to register an identical plugin
>>>> located at URL
>>>> "file:/data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/datanucleus-rdbms-3.2.9.jar."
>>>> 15/02/20 14:40:23 WARN General: Plugin (Bundle) "org.datanucleus" is
>>>> already registered. Ensure you dont have multiple JAR versions of the same
>>>> plugin in the classpath. The URL
>>>> "file:/data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/jars/datanucleus-core-3.2.10.jar"
>>>> is already registered, and you are trying to register an identical plugin
>>>> located at URL
>>>> "file:/data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/datanucleus-core-3.2.10.jar."
>>>> 15/02/20 14:40:23 INFO Persistence: Property datanucleus.cache.level2
>>>> unknown - will be ignored
>>>> 15/02/20 14:40:23 INFO Persistence: Property
>>>> hive.metastore.integral.jdo.pushdown unknown - will be ignored
>>>> 15/02/20 14:40:25 INFO ObjectStore: Setting MetaStore object pin
>>>> classes with
>>>> hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
>>>> 15/02/20 14:40:25 INFO MetaStoreDirectSql: MySQL check failed, assuming
>>>> we are not on mysql: Lexical error at line 1, column 5. Encountered: "@"
>>>> (64), after : "".
>>>> 15/02/20 14:40:27 INFO Datastore: The class
>>>> "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
>>>> "embedded-only" so does not have its own datastore table.
>>>> 15/02/20 14:40:27 INFO Datastore: The class
>>>> "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as
>>>> "embedded-only" so does not have its own datastore table.
>>>> 15/02/20 14:40:28 INFO Datastore: The class
>>>> "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
>>>> "embedded-only" so does not have its own datastore table.
>>>> 15/02/20 14:40:28 INFO Datastore: The class
>>>> "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as
>>>> "embedded-only" so does not have its own datastore table.
>>>> 15/02/20 14:40:28 INFO Query: Reading in results for query
>>>> "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection
>>>> used is closing
>>>> 15/02/20 14:40:28 INFO ObjectStore: Initialized ObjectStore
>>>> 15/02/20 14:40:28 INFO HiveMetaStore: Added admin role in metastore
>>>> 15/02/20 14:40:28 INFO HiveMetaStore: Added public role in metastore
>>>> 15/02/20 14:40:29 INFO HiveMetaStore: No user is added in admin role,
>>>> since config is empty
>>>> 15/02/20 14:40:29 INFO SessionState: No Tez session required at this
>>>> point. hive.execution.engine=mr.
>>>> 15/02/20 14:40:29 INFO HiveMetaStore: 0: get_table : db=analytics
>>>> tbl=trainingdatafinal
>>>> 15/02/20 14:40:29 INFO audit: ugi=hdfs ip=unknown-ip-addr cmd=get_table
>>>> : db=analytics tbl=trainingdatafinal
>>>> 15/02/20 14:40:29 ERROR Hive:
>>>> NoSuchObjectException(message:analytics.trainingdatafinal table not found)
>>>> at
>>>> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1569)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>> at
>>>> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:106)
>>>> at com.sun.proxy.$Proxy24.get_table(Unknown Source)
>>>> at
>>>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1008)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>> at
>>>> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
>>>> at com.sun.proxy.$Proxy25.getTable(Unknown Source)
>>>> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1000)
>>>> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:974)
>>>> at
>>>> org.apache.spark.sql.hive.HiveMetastoreCatalog.lookupRelation(HiveMetastoreCatalog.scala:70)
>>>> at org.apache.spark.sql.hive.HiveContext$$anon$2.org
>>>> $apache$spark$sql$catalyst$analysis$OverrideCatalog$$super$lookupRelation(HiveContext.scala:253)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:141)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:141)
>>>> at scala.Option.getOrElse(Option.scala:120)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$class.lookupRelation(Catalog.scala:141)
>>>> at
>>>> org.apache.spark.sql.hive.HiveContext$$anon$2.lookupRelation(HiveContext.scala:253)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$5.applyOrElse(Analyzer.scala:143)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$5.applyOrElse(Analyzer.scala:138)
>>>> at
>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:144)
>>>> at
>>>> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:162)
>>>> at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
>>>> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>>>> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
>>>> at
>>>> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
>>>> at
>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
>>>> at
>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
>>>> at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
>>>> at scala.collection.AbstractIterator.to(Iterator.scala:1157)
>>>> at
>>>> scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
>>>> at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
>>>> at
>>>> scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
>>>> at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
>>>> at
>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformChildrenDown(TreeNode.scala:191)
>>>> at
>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:147)
>>>> at
>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transform(TreeNode.scala:135)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:138)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:137)
>>>> at
>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1$$anonfun$apply$2.apply(RuleExecutor.scala:61)
>>>> at
>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1$$anonfun$apply$2.apply(RuleExecutor.scala:59)
>>>> at
>>>> scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:111)
>>>> at scala.collection.immutable.List.foldLeft(List.scala:84)
>>>> at
>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1.apply(RuleExecutor.scala:59)
>>>> at
>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1.apply(RuleExecutor.scala:51)
>>>> at scala.collection.immutable.List.foreach(List.scala:318)
>>>> at
>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor.apply(RuleExecutor.scala:51)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.analyzed$lzycompute(SQLContext.scala:411)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.analyzed(SQLContext.scala:411)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.withCachedData$lzycompute(SQLContext.scala:412)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.withCachedData(SQLContext.scala:412)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.optimizedPlan$lzycompute(SQLContext.scala:413)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.optimizedPlan(SQLContext.scala:413)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.sparkPlan$lzycompute(SQLContext.scala:418)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.sparkPlan(SQLContext.scala:416)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.executedPlan$lzycompute(SQLContext.scala:422)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.executedPlan(SQLContext.scala:422)
>>>> at org.apache.spark.sql.SchemaRDD.collect(SchemaRDD.scala:444)
>>>> at $line9.$read$$iwC$$iwC$$iwC$$iwC.<init>(<console>:15)
>>>> at $line9.$read$$iwC$$iwC$$iwC.<init>(<console>:20)
>>>> at $line9.$read$$iwC$$iwC.<init>(<console>:22)
>>>> at $line9.$read$$iwC.<init>(<console>:24)
>>>> at $line9.$read.<init>(<console>:26)
>>>> at $line9.$read$.<init>(<console>:30)
>>>> at $line9.$read$.<clinit>(<console>)
>>>> at $line9.$eval$.<init>(<console>:7)
>>>> at $line9.$eval$.<clinit>(<console>)
>>>> at $line9.$eval.$print(<console>)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>> at
>>>> org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:852)
>>>> at
>>>> org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1125)
>>>> at
>>>> org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:674)
>>>> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:705)
>>>> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:669)
>>>> at
>>>> org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:828)
>>>> at
>>>> org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:873)
>>>> at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:785)
>>>> at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:628)
>>>> at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:636)
>>>> at org.apache.spark.repl.SparkILoop.loop(SparkILoop.scala:641)
>>>> at
>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:968)
>>>> at
>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916)
>>>> at
>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916)
>>>> at
>>>> scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
>>>> at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:916)
>>>> at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1011)
>>>> at org.apache.spark.repl.Main$.main(Main.scala:31)
>>>> at org.apache.spark.repl.Main.main(Main.scala)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>> at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358)
>>>> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
>>>> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>>>>
>>>> org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not
>>>> found trainingdatafinal
>>>> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1004)
>>>> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:974)
>>>> at
>>>> org.apache.spark.sql.hive.HiveMetastoreCatalog.lookupRelation(HiveMetastoreCatalog.scala:70)
>>>> at org.apache.spark.sql.hive.HiveContext$$anon$2.org
>>>> $apache$spark$sql$catalyst$analysis$OverrideCatalog$$super$lookupRelation(HiveContext.scala:253)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:141)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:141)
>>>> at scala.Option.getOrElse(Option.scala:120)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$class.lookupRelation(Catalog.scala:141)
>>>> at
>>>> org.apache.spark.sql.hive.HiveContext$$anon$2.lookupRelation(HiveContext.scala:253)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$5.applyOrElse(Analyzer.scala:143)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$5.applyOrElse(Analyzer.scala:138)
>>>> at
>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:144)
>>>> at
>>>> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:162)
>>>> at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
>>>> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>>>> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
>>>> at
>>>> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
>>>> at
>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
>>>> at
>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
>>>> at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
>>>> at scala.collection.AbstractIterator.to(Iterator.scala:1157)
>>>> at
>>>> scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
>>>> at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
>>>> at
>>>> scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
>>>> at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
>>>> at
>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformChildrenDown(TreeNode.scala:191)
>>>> at
>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:147)
>>>> at
>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transform(TreeNode.scala:135)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:138)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:137)
>>>> at
>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1$$anonfun$apply$2.apply(RuleExecutor.scala:61)
>>>> at
>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1$$anonfun$apply$2.apply(RuleExecutor.scala:59)
>>>> at
>>>> scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:111)
>>>> at scala.collection.immutable.List.foldLeft(List.scala:84)
>>>> at
>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1.apply(RuleExecutor.scala:59)
>>>> at
>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1.apply(RuleExecutor.scala:51)
>>>> at scala.collection.immutable.List.foreach(List.scala:318)
>>>> at
>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor.apply(RuleExecutor.scala:51)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.analyzed$lzycompute(SQLContext.scala:411)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.analyzed(SQLContext.scala:411)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.withCachedData$lzycompute(SQLContext.scala:412)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.withCachedData(SQLContext.scala:412)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.optimizedPlan$lzycompute(SQLContext.scala:413)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.optimizedPlan(SQLContext.scala:413)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.sparkPlan$lzycompute(SQLContext.scala:418)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.sparkPlan(SQLContext.scala:416)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.executedPlan$lzycompute(SQLContext.scala:422)
>>>> at
>>>> org.apache.spark.sql.SQLContext$QueryExecution.executedPlan(SQLContext.scala:422)
>>>> at org.apache.spark.sql.SchemaRDD.collect(SchemaRDD.scala:444)
>>>> at $iwC$$iwC$$iwC$$iwC.<init>(<console>:15)
>>>> at $iwC$$iwC$$iwC.<init>(<console>:20)
>>>> at $iwC$$iwC.<init>(<console>:22)
>>>> at $iwC.<init>(<console>:24)
>>>> at <init>(<console>:26)
>>>> at .<init>(<console>:30)
>>>> at .<clinit>(<console>)
>>>> at .<init>(<console>:7)
>>>> at .<clinit>(<console>)
>>>> at $print(<console>)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>> at
>>>> org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:852)
>>>> at
>>>> org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1125)
>>>> at
>>>> org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:674)
>>>> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:705)
>>>> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:669)
>>>> at
>>>> org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:828)
>>>> at
>>>> org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:873)
>>>> at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:785)
>>>> at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:628)
>>>> at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:636)
>>>> at org.apache.spark.repl.SparkILoop.loop(SparkILoop.scala:641)
>>>> at
>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:968)
>>>> at
>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916)
>>>> at
>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916)
>>>> at
>>>> scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
>>>> at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:916)
>>>> at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1011)
>>>> at org.apache.spark.repl.Main$.main(Main.scala:31)
>>>> at org.apache.spark.repl.Main.main(Main.scala)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>> at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358)
>>>> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
>>>> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>>>>
>>>>
>>>> On Fri, Feb 20, 2015 at 3:28 PM, Sourigna Phetsarath <
>>>> gna.phetsar...@teamaol.com> wrote:
>>>>
>>>>> Try it without
>>>>>
>>>>> "--master yarn-cluster"
>>>>>
>>>>> if you are trying to run a spark-shell. :)
>>>>>
>>>>> On Fri, Feb 20, 2015 at 3:18 PM, chirag lakhani <
>>>>> chirag.lakh...@gmail.com> wrote:
>>>>>
>>>>>> I tried
>>>>>>
>>>>>> spark-shell --master yarn-cluster --driver-class-path
>>>>>> '/data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/*'
>>>>>> --driver-java-options
>>>>>> '-Dspark.executor.extraClassPath=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/*'
>>>>>>
>>>>>> and I get the following error
>>>>>>
>>>>>> Error: Cluster deploy mode is not applicable to Spark shells.
>>>>>> Run with --help for usage help or --verbose for debug output
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Feb 20, 2015 at 2:52 PM, Sourigna Phetsarath <
>>>>>> gna.phetsar...@teamaol.com> wrote:
>>>>>>
>>>>>>> Chirag,
>>>>>>>
>>>>>>> This worked for us:
>>>>>>>
>>>>>>> spark-submit --master yarn-cluster --driver-class-path
>>>>>>> '/opt/cloudera/parcels/CDH/lib/hive/lib/*' --driver-java-options
>>>>>>> '-Dspark.executor.extraClassPath=/opt/cloudera/parcels/CDH/lib/hive/lib/*'
>>>>>>> ...
>>>>>>>
>>>>>>> Let me know, if you have any issues.
>>>>>>>
>>>>>>> On Fri, Feb 20, 2015 at 2:43 PM, chirag lakhani <
>>>>>>> chirag.lakh...@gmail.com> wrote:
>>>>>>>
>>>>>>>> I am trying to access a hive table using spark sql but I am having
>>>>>>>> trouble. I followed the instructions in a cloudera community board
>>>>>>>> which
>>>>>>>> stated
>>>>>>>>
>>>>>>>> 1) Import hive jars into the class path
>>>>>>>>
>>>>>>>> export SPARK_CLASSPATH=$(find
>>>>>>>> /data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/
>>>>>>>> -name
>>>>>>>> '*.jar' -print0 | sed 's/\x0/:/g')
>>>>>>>>
>>>>>>>> 2) start the spark shell
>>>>>>>>
>>>>>>>> spark-shell
>>>>>>>>
>>>>>>>> 3) created a hive context
>>>>>>>>
>>>>>>>> val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
>>>>>>>>
>>>>>>>> 4) then run query
>>>>>>>>
>>>>>>>> sqlContext.sql("FROM analytics.trainingdatafinal SELECT
>>>>>>>> *").collect().foreach(println)
>>>>>>>>
>>>>>>>>
>>>>>>>> When I do this it seems that it cannot find the table in the hive
>>>>>>>> metastore, I have put all of my cloudera parcels in the partition
>>>>>>>> starting
>>>>>>>> with /data as opposed to the default location used by cloudera. Any
>>>>>>>> suggestions on what can be done? I am putting the error below
>>>>>>>>
>>>>>>>>
>>>>>>>> 15/02/20 13:43:01 ERROR Hive:
>>>>>>>> NoSuchObjectException(message:analytics.trainingdatafinal table not
>>>>>>>> found)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1569)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at
>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>>>> at
>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:106)
>>>>>>>> at com.sun.proxy.$Proxy24.get_table(Unknown Source)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1008)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at
>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>>>> at
>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
>>>>>>>> at com.sun.proxy.$Proxy25.getTable(Unknown Source)
>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1000)
>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:974)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.hive.HiveMetastoreCatalog.lookupRelation(HiveMetastoreCatalog.scala:70)
>>>>>>>> at org.apache.spark.sql.hive.HiveContext$$anon$2.org
>>>>>>>> $apache$spark$sql$catalyst$analysis$OverrideCatalog$$super$lookupRelation(HiveContext.scala:253)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:141)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:141)
>>>>>>>> at scala.Option.getOrElse(Option.scala:120)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$class.lookupRelation(Catalog.scala:141)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.hive.HiveContext$$anon$2.lookupRelation(HiveContext.scala:253)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$5.applyOrElse(Analyzer.scala:143)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$5.applyOrElse(Analyzer.scala:138)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:144)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:162)
>>>>>>>> at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
>>>>>>>> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>>>>>>>> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
>>>>>>>> at
>>>>>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
>>>>>>>> at
>>>>>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
>>>>>>>> at scala.collection.TraversableOnce$class.to
>>>>>>>> (TraversableOnce.scala:273)
>>>>>>>> at scala.collection.AbstractIterator.to(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
>>>>>>>> at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
>>>>>>>> at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformChildrenDown(TreeNode.scala:191)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:147)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:162)
>>>>>>>> at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
>>>>>>>> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>>>>>>>> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
>>>>>>>> at
>>>>>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
>>>>>>>> at
>>>>>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
>>>>>>>> at scala.collection.TraversableOnce$class.to
>>>>>>>> (TraversableOnce.scala:273)
>>>>>>>> at scala.collection.AbstractIterator.to(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
>>>>>>>> at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
>>>>>>>> at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformChildrenDown(TreeNode.scala:191)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:147)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transform(TreeNode.scala:135)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:138)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:137)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1$$anonfun$apply$2.apply(RuleExecutor.scala:61)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1$$anonfun$apply$2.apply(RuleExecutor.scala:59)
>>>>>>>> at
>>>>>>>> scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:111)
>>>>>>>> at scala.collection.immutable.List.foldLeft(List.scala:84)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1.apply(RuleExecutor.scala:59)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1.apply(RuleExecutor.scala:51)
>>>>>>>> at scala.collection.immutable.List.foreach(List.scala:318)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor.apply(RuleExecutor.scala:51)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.analyzed$lzycompute(SQLContext.scala:411)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.analyzed(SQLContext.scala:411)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.withCachedData$lzycompute(SQLContext.scala:412)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.withCachedData(SQLContext.scala:412)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.optimizedPlan$lzycompute(SQLContext.scala:413)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.optimizedPlan(SQLContext.scala:413)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.sparkPlan$lzycompute(SQLContext.scala:418)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.sparkPlan(SQLContext.scala:416)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.executedPlan$lzycompute(SQLContext.scala:422)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.executedPlan(SQLContext.scala:422)
>>>>>>>> at org.apache.spark.sql.SchemaRDD.collect(SchemaRDD.scala:444)
>>>>>>>> at org.apache.spark.sql.SchemaRDD.take(SchemaRDD.scala:446)
>>>>>>>> at $line9.$read$$iwC$$iwC$$iwC$$iwC.<init>(<console>:15)
>>>>>>>> at $line9.$read$$iwC$$iwC$$iwC.<init>(<console>:20)
>>>>>>>> at $line9.$read$$iwC$$iwC.<init>(<console>:22)
>>>>>>>> at $line9.$read$$iwC.<init>(<console>:24)
>>>>>>>> at $line9.$read.<init>(<console>:26)
>>>>>>>> at $line9.$read$.<init>(<console>:30)
>>>>>>>> at $line9.$read$.<clinit>(<console>)
>>>>>>>> at $line9.$eval$.<init>(<console>:7)
>>>>>>>> at $line9.$eval$.<clinit>(<console>)
>>>>>>>> at $line9.$eval.$print(<console>)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at
>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>>>> at
>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:852)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1125)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:674)
>>>>>>>> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:705)
>>>>>>>> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:669)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:828)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:873)
>>>>>>>> at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:785)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:628)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:636)
>>>>>>>> at org.apache.spark.repl.SparkILoop.loop(SparkILoop.scala:641)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:968)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916)
>>>>>>>> at
>>>>>>>> scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
>>>>>>>> at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:916)
>>>>>>>> at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1011)
>>>>>>>> at org.apache.spark.repl.Main$.main(Main.scala:31)
>>>>>>>> at org.apache.spark.repl.Main.main(Main.scala)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at
>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>>>> at
>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>> at
>>>>>>>> org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358)
>>>>>>>> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
>>>>>>>> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>>>>>>>>
>>>>>>>> org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not
>>>>>>>> found trainingdatafinal
>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1004)
>>>>>>>> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:974)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.hive.HiveMetastoreCatalog.lookupRelation(HiveMetastoreCatalog.scala:70)
>>>>>>>> at org.apache.spark.sql.hive.HiveContext$$anon$2.org
>>>>>>>> $apache$spark$sql$catalyst$analysis$OverrideCatalog$$super$lookupRelation(HiveContext.scala:253)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:141)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:141)
>>>>>>>> at scala.Option.getOrElse(Option.scala:120)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.OverrideCatalog$class.lookupRelation(Catalog.scala:141)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.hive.HiveContext$$anon$2.lookupRelation(HiveContext.scala:253)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$5.applyOrElse(Analyzer.scala:143)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$5.applyOrElse(Analyzer.scala:138)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:144)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:162)
>>>>>>>> at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
>>>>>>>> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>>>>>>>> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
>>>>>>>> at
>>>>>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
>>>>>>>> at
>>>>>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
>>>>>>>> at scala.collection.TraversableOnce$class.to
>>>>>>>> (TraversableOnce.scala:273)
>>>>>>>> at scala.collection.AbstractIterator.to(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
>>>>>>>> at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
>>>>>>>> at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformChildrenDown(TreeNode.scala:191)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:147)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:162)
>>>>>>>> at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
>>>>>>>> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>>>>>>>> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
>>>>>>>> at
>>>>>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
>>>>>>>> at
>>>>>>>> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
>>>>>>>> at scala.collection.TraversableOnce$class.to
>>>>>>>> (TraversableOnce.scala:273)
>>>>>>>> at scala.collection.AbstractIterator.to(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
>>>>>>>> at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
>>>>>>>> at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformChildrenDown(TreeNode.scala:191)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:147)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.trees.TreeNode.transform(TreeNode.scala:135)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:138)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:137)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1$$anonfun$apply$2.apply(RuleExecutor.scala:61)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1$$anonfun$apply$2.apply(RuleExecutor.scala:59)
>>>>>>>> at
>>>>>>>> scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:111)
>>>>>>>> at scala.collection.immutable.List.foldLeft(List.scala:84)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1.apply(RuleExecutor.scala:59)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1.apply(RuleExecutor.scala:51)
>>>>>>>> at scala.collection.immutable.List.foreach(List.scala:318)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.catalyst.rules.RuleExecutor.apply(RuleExecutor.scala:51)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.analyzed$lzycompute(SQLContext.scala:411)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.analyzed(SQLContext.scala:411)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.withCachedData$lzycompute(SQLContext.scala:412)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.withCachedData(SQLContext.scala:412)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.optimizedPlan$lzycompute(SQLContext.scala:413)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.optimizedPlan(SQLContext.scala:413)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.sparkPlan$lzycompute(SQLContext.scala:418)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.sparkPlan(SQLContext.scala:416)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.executedPlan$lzycompute(SQLContext.scala:422)
>>>>>>>> at
>>>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.executedPlan(SQLContext.scala:422)
>>>>>>>> at org.apache.spark.sql.SchemaRDD.collect(SchemaRDD.scala:444)
>>>>>>>> at org.apache.spark.sql.SchemaRDD.take(SchemaRDD.scala:446)
>>>>>>>> at $iwC$$iwC$$iwC$$iwC.<init>(<console>:15)
>>>>>>>> at $iwC$$iwC$$iwC.<init>(<console>:20)
>>>>>>>> at $iwC$$iwC.<init>(<console>:22)
>>>>>>>> at $iwC.<init>(<console>:24)
>>>>>>>> at <init>(<console>:26)
>>>>>>>> at .<init>(<console>:30)
>>>>>>>> at .<clinit>(<console>)
>>>>>>>> at .<init>(<console>:7)
>>>>>>>> at .<clinit>(<console>)
>>>>>>>> at $print(<console>)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at
>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>>>> at
>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:852)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1125)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:674)
>>>>>>>> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:705)
>>>>>>>> at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:669)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:828)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:873)
>>>>>>>> at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:785)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:628)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:636)
>>>>>>>> at org.apache.spark.repl.SparkILoop.loop(SparkILoop.scala:641)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:968)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916)
>>>>>>>> at
>>>>>>>> org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:916)
>>>>>>>> at
>>>>>>>> scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
>>>>>>>> at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:916)
>>>>>>>> at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1011)
>>>>>>>> at org.apache.spark.repl.Main$.main(Main.scala:31)
>>>>>>>> at org.apache.spark.repl.Main.main(Main.scala)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at
>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>>>> at
>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>> at
>>>>>>>> org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358)
>>>>>>>> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
>>>>>>>> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>>
>>>>>>> *Gna Phetsarath*C: +1 917.373.7363
>>>>>>> AIM: sphetsarath20 VVMR: 8890237
>>>>>>> Address | 54 West 40th Street, New York, NY 10018
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>>
>>>>> *Gna Phetsarath*C: +1 917.373.7363
>>>>> AIM: sphetsarath20 VVMR: 8890237
>>>>> Address | 54 West 40th Street, New York, NY 10018
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>>
>>> *Gna Phetsarath*C: +1 917.373.7363
>>> AIM: sphetsarath20 VVMR: 8890237
>>> Address | 54 West 40th Street, New York, NY 10018
>>>
>>
>>
>>
>> --
>>
>>
>> *Gna Phetsarath*C: +1 917.373.7363
>> AIM: sphetsarath20 VVMR: 8890237
>> Address | 54 West 40th Street, New York, NY 10018
>>
>
>
>
> --
>
>
> *Gna Phetsarath*C: +1 917.373.7363
> AIM: sphetsarath20 VVMR: 8890237
> Address | 54 West 40th Street, New York, NY 10018
>
