[ https://issues.apache.org/jira/browse/SPARK-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen updated SPARK-16138: ------------------------------ Summary: YarnAllocator tries to cancel executor requests when we have none (was: YarnAllocator tries to cancel executor requests when we have non) Is it that some method shouldn't even be called if there are 0 executors to request, thus avoiding the warning? Like if numToCancel on line 356 is 0? > YarnAllocator tries to cancel executor requests when we have none > ----------------------------------------------------------------- > > Key: SPARK-16138 > URL: https://issues.apache.org/jira/browse/SPARK-16138 > Project: Spark > Issue Type: Bug > Reporter: Peter Ableda > Priority: Minor > > Easy to reproduce: > $ spark-shell > > sc.parallelize(1 to 10000).count() > After the computation finishes we have 0 pending tasks so we will need 0 > executors. > {code} > 16/06/21 04:07:41 INFO yarn.YarnAllocator: Driver requested a total number of > 0 executor(s). > 16/06/21 04:07:41 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:07:41 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > ... > 16/06/21 04:08:41 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:08:41 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > 16/06/21 04:08:41 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to > kill executor(s) 1. > 16/06/21 04:08:48 INFO yarn.YarnAllocator: Driver requested a total number of > 0 executor(s). > 16/06/21 04:08:48 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:08:48 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > 16/06/21 04:08:48 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to > kill executor(s) 2. > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org