Hi All, I wanted Spark on Yarn to up and running.
I did "*SPARK_HADOOP_VERSION=2.3.0 SPARK_YARN=true ./sbt/sbt assembly*" Then i ran "*SPARK_JAR=./assembly/target/scala-2.9.3/spark-assembly-0.8.1-incubating-hadoop2.3.0.jar SPARK_YARN_APP_JAR=examples/target/scala-2.9.3/spark-examples_2.9.3-0.8.1-incubating.jar MASTER=yarn-client ./spark-shell*" I have SPARK_HOME, YARN_CONF_DIR/HADOOP_CONF_DIR set. Still i get the following error. Any clues ?? *ERROR:* *...........................................Using Scala version 2.9.3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_51)Initializing interpreter...Creating SparkContext...java.lang.NullPointerException at scala.collection.mutable.ArrayOps$ofRef.length(ArrayOps.scala:115) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:38) at org.apache.spark.deploy.yarn.Client$.populateHadoopClasspath(Client.scala:489) at org.apache.spark.deploy.yarn.Client$.populateClasspath(Client.scala:510) at org.apache.spark.deploy.yarn.Client.setupLaunchEnv(Client.scala:327) at org.apache.spark.deploy.yarn.Client.runApp(Client.scala:90) at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:71) at org.apache.spark.scheduler.cluster.ClusterScheduler.start(ClusterScheduler.scala:119) at org.apache.spark.SparkContext.<init>(SparkContext.scala:273) at org.apache.spark.repl.SparkILoop.createSparkContext(SparkILoop.scala:862) at <init>(<console>:10) at <init>(<console>:22) at <init>(<console>:24) at .<init>(<console>:28) at .<clinit>(<console>) at .<init>(<console>:7) at .<clinit>(<console>) at $export(<console>) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:629) at org.apache.spark.repl.SparkIMain$Request$$anonfun$10.apply(SparkIMain.scala:897) at scala.tools.nsc.interpreter.Line$$anonfun$1.apply$mcV$sp(Line.scala:43) at scala.tools.nsc.io.package$$anon$2.run(package.scala:25) at java.lang.Thread.run(Thread.java:744)* *.....................................................................*
