Hi Jacek, Maven build output *mvn clean install*
[INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 30:12 min [INFO] Finished at: 2016-06-17T15:15:46+02:00 [INFO] Final Memory: 82M/1253M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project spark-core_2.11: There are test failures -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :spark-core_2.11 and the error - handles standalone cluster mode *** FAILED *** Map("spark.driver.memory" -> "4g", "SPARK_SUBMIT" -> "true", "spark.driver.cores" -> "5", "spark.ui.enabled" -> "false", "spark.driver.supervise" -> "true", "spark.app.name" -> "org.SomeClass", "spark.jars" -> "file:/Users/krishna/Experiment/spark/core/thejar.jar", "spark.submit.deployMode" -> "cluster", "spark.executor.extraClassPath" -> "~/mysql-connector-java-5.1.12.jar", "spark.master" -> "spark://h:p", "spark.driver.extraClassPath" -> "~/mysql-connector-java-5.1.12.jar") had size 11 instead of expected size 9 (SparkSubmitSuite.scala:294) - handles legacy standalone cluster mode *** FAILED *** Map("spark.driver.memory" -> "4g", "SPARK_SUBMIT" -> "true", "spark.driver.cores" -> "5", "spark.ui.enabled" -> "false", "spark.driver.supervise" -> "true", "spark.app.name" -> "org.SomeClass", "spark.jars" -> "file:/Users/krishna/Experiment/spark/core/thejar.jar", "spark.submit.deployMode" -> "cluster", "spark.executor.extraClassPath" -> "~/mysql-connector-java-5.1.12.jar", "spark.master" -> "spark://h:p", "spark.driver.extraClassPath" -> "~/mysql-connector-java-5.1.12.jar") had size 11 instead of expected size 9 (SparkSubmitSuite.scala:294) On Thu, Jun 16, 2016 at 1:57 PM, Jacek Laskowski <ja...@japila.pl> wrote: > Hi, > > Before you try to do it inside another environment like an IDE, could > you build Spark using mvn or sbt and only when successful try to run > SparkPi using spark-submit run-example. With that, you could try to > have a complete environment inside your beloved IDE (and I'm very glad > to hear it's IDEA :)) > > Pozdrawiam, > Jacek Laskowski > ---- > https://medium.com/@jaceklaskowski/ > Mastering Apache Spark http://bit.ly/mastering-apache-spark > Follow me at https://twitter.com/jaceklaskowski > > > On Thu, Jun 16, 2016 at 1:37 AM, Krishna Kalyan > <krishnakaly...@gmail.com> wrote: > > Hello, > > I am faced with problems when I try to run SparkPi.scala. > > I took the following steps below: > > a) git pull https://github.com/apache/spark > > b) Import the project in Intellij as a maven project > > c) Run 'SparkPi' > > > > Error Below: > > Information:16/06/16 01:34 - Compilation completed with 10 errors and 5 > > warnings in 5s 843ms > > Warning:scalac: Class org.jboss.netty.channel.ChannelFactory not found - > > continuing with a stub. > > Warning:scalac: Class org.jboss.netty.channel.ChannelPipelineFactory not > > found - continuing with a stub. > > Warning:scalac: Class org.jboss.netty.handler.execution.ExecutionHandler > not > > found - continuing with a stub. > > Warning:scalac: Class org.jboss.netty.channel.group.ChannelGroup not > found - > > continuing with a stub. > > Warning:scalac: Class com.google.common.collect.ImmutableMap not found - > > continuing with a stub. > > > /Users/krishna/Experiment/spark/external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkAvroCallbackHandler.scala > > Error:(45, 66) not found: type SparkFlumeProtocol > > val transactionTimeout: Int, val backOffInterval: Int) extends > > SparkFlumeProtocol with Logging { > > ^ > > Error:(70, 39) not found: type EventBatch > > override def getEventBatch(n: Int): EventBatch = { > > ^ > > Error:(85, 13) not found: type EventBatch > > new EventBatch("Spark sink has been stopped!", "", > > java.util.Collections.emptyList()) > > ^ > > > /Users/krishna/Experiment/spark/external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/TransactionProcessor.scala > > Error:(80, 22) not found: type EventBatch > > def getEventBatch: EventBatch = { > > ^ > > Error:(48, 37) not found: type EventBatch > > @volatile private var eventBatch: EventBatch = new EventBatch("Unknown > > Error", "", > > ^ > > Error:(48, 54) not found: type EventBatch > > @volatile private var eventBatch: EventBatch = new EventBatch("Unknown > > Error", "", > > ^ > > Error:(115, 41) not found: type SparkSinkEvent > > val events = new util.ArrayList[SparkSinkEvent](maxBatchSize) > > ^ > > Error:(146, 28) not found: type EventBatch > > eventBatch = new EventBatch("", seqNum, events) > > ^ > > > /Users/krishna/Experiment/spark/external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkSinkUtils.scala > > Error:(25, 27) not found: type EventBatch > > def isErrorBatch(batch: EventBatch): Boolean = { > > ^ > > > /Users/krishna/Experiment/spark/external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkSink.scala > > Error:(86, 51) not found: type SparkFlumeProtocol > > val responder = new SpecificResponder(classOf[SparkFlumeProtocol], > > handler.get) > > > > Thanks, > > Krishan >
