[ https://issues.apache.org/jira/browse/SPARK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves resolved SPARK-1453. ---------------------------------- Resolution: Duplicate duplicate of https://issues.apache.org/jira/browse/SPARK-1946 > Improve the way Spark on Yarn waits for executors before starting > ----------------------------------------------------------------- > > Key: SPARK-1453 > URL: https://issues.apache.org/jira/browse/SPARK-1453 > Project: Spark > Issue Type: Improvement > Components: YARN > Affects Versions: 1.0.0 > Reporter: Thomas Graves > Assignee: Thomas Graves > > Currently Spark on Yarn just delays a few seconds between when the spark > context is initialized and when it allows the job to start. If you are on a > busy hadoop cluster is might take longer to get the number of executors. > In the very least we could make this timeout a configurable value. Its > currently hardcoded to 3 seconds. > Better yet would be to allow user to give a minimum number of executors it > wants to wait for, but that looks much more complex. -- This message was sent by Atlassian JIRA (v6.2#6252)