I am seeing what look like environmental errors when I try to run a test on a clean local branch which has been sync'd to the head of the development trunk. I would appreciate advice about how to debug or hack around this problem. For the record, the test ran cleanly last week. This is the experiment I am running:
# build mvn -Pyarn -Phadoop-2.3 -DskipTests -Phive -Phive-thriftserver clean package # run one suite mvn -Dhadoop.version=2.4.0 -DwildcardSuites=JDBCSuite The test bombs out before getting to JDBCSuite. I see this summary at the end... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Spark Project Parent POM ........................... SUCCESS [ 2.023 s] [INFO] Spark Project Test Tags ............................ SUCCESS [ 1.924 s] [INFO] Spark Project Launcher ............................. SUCCESS [ 5.837 s] [INFO] Spark Project Networking ........................... SUCCESS [ 12.498 s] [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [01:28 min] [INFO] Spark Project Unsafe ............................... SUCCESS [01:09 min] [INFO] Spark Project Core ................................. SUCCESS [02:45 min] [INFO] Spark Project Bagel ................................ SUCCESS [ 30.182 s] [INFO] Spark Project GraphX ............................... SUCCESS [ 59.002 s] [INFO] Spark Project Streaming ............................ FAILURE [06:21 min] [INFO] Spark Project Catalyst ............................. SKIPPED [INFO] Spark Project SQL .................................. SKIPPED [INFO] Spark Project ML Library ........................... SKIPPED [INFO] Spark Project Tools ................................ SKIPPED [INFO] Spark Project Hive ................................. SKIPPED [INFO] Spark Project REPL ................................. SKIPPED [INFO] Spark Project Assembly ............................. SKIPPED [INFO] Spark Project External Twitter ..................... SKIPPED [INFO] Spark Project External Flume Sink .................. SKIPPED [INFO] Spark Project External Flume ....................... SKIPPED [INFO] Spark Project External Flume Assembly .............. SKIPPED [INFO] Spark Project External MQTT ........................ SKIPPED [INFO] Spark Project External MQTT Assembly ............... SKIPPED [INFO] Spark Project External ZeroMQ ...................... SKIPPED [INFO] Spark Project External Kafka ....................... SKIPPED [INFO] Spark Project Examples ............................. SKIPPED [INFO] Spark Project External Kafka Assembly .............. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 13:37 min [INFO] Finished at: 2015-10-15T09:03:06-07:00 [INFO] Final Memory: 69M/793M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project spark-streaming_2.10: There are test failures. [ERROR] [ERROR] Please refer to /Users/rhillegas/spark/spark/streaming/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :spark-streaming_2.10 >From the logs in streaming/target/surefire-reports, it appears that the following tests failed... org.apache.spark.streaming.JavaAPISuite.txt org.apache.spark.streaming.JavaReceiverAPISuite.txt ...with this error: java.net.BindException: Failed to bind to: /9.52.158.156:0: Service 'sparkDriver' failed after 100 retries! at org.jboss.netty.bootstrap.ServerBootstrap.bind (ServerBootstrap.java:272) at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply (NettyTransport.scala:393) at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply (NettyTransport.scala:389) at scala.util.Success$$anonfun$map$1.apply(Try.scala:206) at scala.util.Try$.apply(Try.scala:161) at scala.util.Success.map(Try.scala:206) at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235) at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch (BatchingExecutor.scala:55) at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply $mcV$sp(BatchingExecutor.scala:91) at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply (BatchingExecutor.scala:91) at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply (BatchingExecutor.scala:91) at scala.concurrent.BlockContext$.withBlockContext (BlockContext.scala:72) at akka.dispatch.BatchingExecutor$BlockableBatch.run (BatchingExecutor.scala:90) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec (AbstractDispatcher.scala:397) at scala.concurrent.forkjoin.ForkJoinTask.doExec (ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask (ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker (ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run (ForkJoinWorkerThread.java:107) It is suggested that there might be a problem with my /etc/hosts, according to http://stackoverflow.com/questions/29906686/failed-to-bind-to-spark-master-using-a-remote-cluster-with-two-workers . But /etc/hosts looks fine to me: bash-3.2$ cat /etc/hosts ## # Host Database # # localhost is used to configure the loopback interface # when the system is booting. Do not change this entry. ## 127.0.0.1 localhost 255.255.255.255 broadcasthost ::1 localhost Is there some environmental variable, config file setting, or JVM system property which will hack around this problem? Any advice would be appreciated. Thanks, -Rick
