Hi Kal El, to run you an application outside the spark-shell you need to write a *build file* (example.sbt) parallel to your source code. step1: *example.sbt*
name := "Simple Project" version := "1.0" scalaVersion := "2.9.3" libraryDependencies += "org.apache.spark" %% "spark-streaming" % "0.9.0-incubating" resolvers ++= Seq( "Akka Repository" at "http://repo.akka.io/releases/","Spray Repository" at "http://repo.spray.cc/") step2: download sbt from http://scalasbt.artifactoryonline.com/scalasbt/sbt-native-packages/org/scala-sbt/sbt/0.12.2/sbt.tgz unzip it and set the path. (up to sbt/bin) step3: parallel to your .sbt file type command $sbt package $sbt run [?] On Tue, Feb 11, 2014 at 2:48 PM, Kal El <pinu.datri...@yahoo.com> wrote: > I am trying to run this code on spark 0.9.0 and I am getting this error: > "java.lang.ClassNotFoundException: > org.apache.spark.streaming.StreamingContext" > > My CLASSPATH variable contains the following > path"....spark-0.9.0/assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-hadoop1.0.4.jar" > what else do I need to add here ? > > The code works fine in the spark-shell. > > "import org.apache.spark.streaming.{Time, Seconds, StreamingContext} > import org.apache.spark.streaming.StreamingContext._ > > object HdfsWordCount { > def main(args: Array[String]) { > if (args.length < 2) { > System.err.println("Usage: HdfsWordCount <master> <directory>") > System.exit(1) > } > > // StreamingExamples.setStreamingLogLevels() > > // Create the context > // val ssc = new StreamingContext(args(0), "HdfsWordCount", Seconds(2), > System.getenv("SPARK_HOME"), Seq(System.getenv("SPARK_HOME"))) > val ssc = new StreamingContext(args(0), "HdfsWordCount", Seconds(2)) > // Create the FileInputDStream on the directory and use the > // stream to count words in new files created > val lines = ssc.textFileStream(args(1)) > val words = lines.flatMap(_.split(" ")) > val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _) > wordCounts.print() > ssc.start() > ssc.awaitTermination() > } > }" > > Thanks > > >
<<328.png>>