I am synced up to the Spark master branch as of commit 23468e7e96. I have Maven 3.0.5, Scala 2.10.3, and SBT 0.13.1. I’ve built the master branch successfully previously and am trying to rebuild again to take advantage of the new Hive 0.13.1 profile. I execute the following command:
$ mvn -DskipTests -Phive-0.13-1 -Phadoop-2.4 -Pyarn clean package The build fails at the following stage: INFO] Using incremental compilation [INFO] compiler plugin: BasicArtifact(org.scalamacros,paradise_2.10.4,2.0.1,null) [INFO] Compiling 5 Scala sources to /home/terrys/Applications/spark/yarn/stable/target/scala-2.10/test-classes... [ERROR] /home/terrys/Applications/spark/yarn/common/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala:20: object MemLimitLogger is not a member of package org.apache.spark.deploy.yarn [ERROR] import org.apache.spark.deploy.yarn.MemLimitLogger._ [ERROR] ^ [ERROR] /home/terrys/Applications/spark/yarn/common/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala:29: not found: value memLimitExceededLogMessage [ERROR] val vmemMsg = memLimitExceededLogMessage(diagnostics, VMEM_EXCEEDED_PATTERN) [ERROR] ^ [ERROR] /home/terrys/Applications/spark/yarn/common/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala:30: not found: value memLimitExceededLogMessage [ERROR] val pmemMsg = memLimitExceededLogMessage(diagnostics, PMEM_EXCEEDED_PATTERN) [ERROR] ^ [ERROR] three errors found [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Spark Project Parent POM .......................... SUCCESS [2.758s] [INFO] Spark Project Common Network Code ................. SUCCESS [6.716s] [INFO] Spark Project Core ................................ SUCCESS [2:46.610s] [INFO] Spark Project Bagel ............................... SUCCESS [16.776s] [INFO] Spark Project GraphX .............................. SUCCESS [52.159s] [INFO] Spark Project Streaming ........................... SUCCESS [1:09.883s] [INFO] Spark Project ML Library .......................... SUCCESS [1:18.932s] [INFO] Spark Project Tools ............................... SUCCESS [10.210s] [INFO] Spark Project Catalyst ............................ SUCCESS [1:12.499s] [INFO] Spark Project SQL ................................. SUCCESS [1:10.561s] [INFO] Spark Project Hive ................................ SUCCESS [1:08.571s] [INFO] Spark Project REPL ................................ SUCCESS [32.377s] [INFO] Spark Project YARN Parent POM ..................... SUCCESS [1.317s] [INFO] Spark Project YARN Stable API ..................... FAILURE [25.918s] [INFO] Spark Project Assembly ............................ SKIPPED [INFO] Spark Project External Twitter .................... SKIPPED [INFO] Spark Project External Kafka ...................... SKIPPED [INFO] Spark Project External Flume Sink ................. SKIPPED [INFO] Spark Project External Flume ...................... SKIPPED [INFO] Spark Project External ZeroMQ ..................... SKIPPED [INFO] Spark Project External MQTT ....................... SKIPPED [INFO] Spark Project Examples ............................ SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 11:15.889s [INFO] Finished at: Fri Oct 31 12:08:55 PDT 2014 [INFO] Final Memory: 73M/829M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.0:testCompile (scala-test-compile-first) on project spark-yarn_2.10: Execution scala-test-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.0:testCompile failed. CompileFailed -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :spark-yarn_2.10 I could not find MemLimitLogger anywhere in the Spark code. Anybody else seen/encounter this? Thanks, -Terry
