[ https://issues.apache.org/jira/browse/SPARK-16019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Vanzin resolved SPARK-16019. ------------------------------------ Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.3.0 > Eliminate unexpected delay during spark on yarn job launch > ---------------------------------------------------------- > > Key: SPARK-16019 > URL: https://issues.apache.org/jira/browse/SPARK-16019 > Project: Spark > Issue Type: Improvement > Components: YARN > Reporter: Olasoji > Assignee: Marcelo Vanzin > Priority: Minor > Fix For: 2.3.0 > > > Currently when launching a job in yarn mode, there will be an added delay of > about "spark.yarn.report.interval" seconds before launch. By default this > parameter is set to 1 second, however if a user increases this for whatever > reason, an unexpected startup delay is introduced. > The "waitForApplication" function called during job submission and launch > will eventually call monitorApplication and so sleep for the configured > interval time before checking the state of the job and proceeding with job > startup. > One solution would be to add an "interval" argument to the monitorApplication > function. This added flexibility would allow callers who don't need to wait > to set a lower wait interval or none. > Patch to follow soon -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org