[ https://issues.apache.org/jira/browse/SPARK-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-19770. ------------------------------- Resolution: Invalid Questions should go to the mailing list but this sounds like some kind of cluster problem. > Running Example SparkPi Job on Mesos > ------------------------------------ > > Key: SPARK-19770 > URL: https://issues.apache.org/jira/browse/SPARK-19770 > Project: Spark > Issue Type: Question > Components: Mesos, Spark Core, Spark Submit > Affects Versions: 2.1.0 > Environment: spark-2.1.0-bin-hadoop2.3, mesos-1-1 > Reporter: Joe Olson > > I am trying to submit the example SparkPi job to a 9 node Spark cluster > running on Mesos. My spark-submit statement: > {quote} > ./bin/spark-submit \ > --name "Test01:" \ > --class org.apache.spark.examples.SparkPi \ > --master mesos://<IP Address>:7078 \ > --deploy-mode cluster \ > --executor-memory 16G \ > --executor-cores 1 \ > --driver-cores 10 \ > --driver-memory 48G \ > --num-executors 1 \ > file://mnt/ocarchive1/sstables/jars/spark-examples_2.11-2.1.0.jar \ > 1000 > {quote} > When I do this, Mesos reports the job completes successfully. I can go into > the stdout file on the driver machine, and see the "Pi is roughly...." > output. > However, on most of the slave machines, If I go into the stderr file for that > same job, I see the following exceptions: > {quote} > 17/02/28 10:20:59 ERROR RetryingBlockFetcher: Exception while beginning fetch > of 1 outstanding blocks > java.io.IOException: Failed to connect to <machine > name>/fe80:0:0:0:ec4:7aff:fea4:82e1%5:44791 > {quote} > There appears to be a connectivity issue between some of the 9 nodes, some of > the time. It does not seem to be consistent on the routes between machines. > (Sometimes #2 and #7 talk, sometimes they do not). > How can I troubleshoot this? Network behavior seems normal otherwise. > ALSO - sometimes in the logs I'll see > {quote} > 17/02/28 10:20:54 WARN TaskSchedulerImpl: Initial job has not accepted any > resources; check your cluster UI to ensure that workers are registered and > have sufficient resources > {quote} > However, my resource count in Mesos (via the UI) is accurate. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org