[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14340968#comment-14340968 ]
Thomas Graves commented on SPARK-6050: -------------------------------------- Note that there is one other problem Mridul reported in this jira and perhaps we should split this into another jira unless [~vanzin] thinks the pr fixes it also: So the scenario is like this. Suppose we asked for 10 workers, RM gave us say 12 - but we decided to use only 6 (in this example, we used 0 due to vCores constraints). Now, we still need 4 workers more - but we will never ask for it again from RM. > Spark on YARN does not work --executor-cores is specified > --------------------------------------------------------- > > Key: SPARK-6050 > URL: https://issues.apache.org/jira/browse/SPARK-6050 > Project: Spark > Issue Type: Bug > Components: YARN > Affects Versions: 1.3.0 > Environment: 2.5 based YARN cluster. > Reporter: Mridul Muralidharan > Priority: Blocker > > There are multiple issues here (which I will detail as comments), but to > reproduce running the following ALWAYS hangs in our cluster with the 1.3 RC > ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master > yarn-cluster --executor-cores 8 --num-executors 15 --driver-memory 4g > --executor-memory 2g --queue webmap lib/spark-examples*.jar > 10 -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org