[jira] [Updated] (SPARK-16138) YarnAllocator tries to cancel executor requests when we have none
[ https://issues.apache.org/jira/browse/SPARK-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16138: -- Component/s: YARN > YarnAllocator tries to cancel executor requests when we have none > - > > Key: SPARK-16138 > URL: https://issues.apache.org/jira/browse/SPARK-16138 > Project: Spark > Issue Type: Bug > Components: YARN >Reporter: Peter Ableda >Assignee: Peter Ableda >Priority: Minor > Fix For: 2.1.0 > > > Easy to reproduce: > $ spark-shell > > sc.parallelize(1 to 1).count() > After the computation finishes we have 0 pending tasks so we will need 0 > executors. > {code} > 16/06/21 04:07:41 INFO yarn.YarnAllocator: Driver requested a total number of > 0 executor(s). > 16/06/21 04:07:41 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:07:41 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > ... > 16/06/21 04:08:41 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:08:41 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > 16/06/21 04:08:41 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to > kill executor(s) 1. > 16/06/21 04:08:48 INFO yarn.YarnAllocator: Driver requested a total number of > 0 executor(s). > 16/06/21 04:08:48 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:08:48 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > 16/06/21 04:08:48 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to > kill executor(s) 2. > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-16138) YarnAllocator tries to cancel executor requests when we have none
[ https://issues.apache.org/jira/browse/SPARK-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-16138: -- Assignee: Peter Ableda (was: Apache Spark) > YarnAllocator tries to cancel executor requests when we have none > - > > Key: SPARK-16138 > URL: https://issues.apache.org/jira/browse/SPARK-16138 > Project: Spark > Issue Type: Bug >Reporter: Peter Ableda >Assignee: Peter Ableda >Priority: Minor > Fix For: 2.1.0 > > > Easy to reproduce: > $ spark-shell > > sc.parallelize(1 to 1).count() > After the computation finishes we have 0 pending tasks so we will need 0 > executors. > {code} > 16/06/21 04:07:41 INFO yarn.YarnAllocator: Driver requested a total number of > 0 executor(s). > 16/06/21 04:07:41 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:07:41 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > ... > 16/06/21 04:08:41 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:08:41 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > 16/06/21 04:08:41 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to > kill executor(s) 1. > 16/06/21 04:08:48 INFO yarn.YarnAllocator: Driver requested a total number of > 0 executor(s). > 16/06/21 04:08:48 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:08:48 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > 16/06/21 04:08:48 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to > kill executor(s) 2. > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-16138) YarnAllocator tries to cancel executor requests when we have none
[ https://issues.apache.org/jira/browse/SPARK-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16138: -- Summary: YarnAllocator tries to cancel executor requests when we have none (was: YarnAllocator tries to cancel executor requests when we have non) Is it that some method shouldn't even be called if there are 0 executors to request, thus avoiding the warning? Like if numToCancel on line 356 is 0? > YarnAllocator tries to cancel executor requests when we have none > - > > Key: SPARK-16138 > URL: https://issues.apache.org/jira/browse/SPARK-16138 > Project: Spark > Issue Type: Bug >Reporter: Peter Ableda >Priority: Minor > > Easy to reproduce: > $ spark-shell > > sc.parallelize(1 to 1).count() > After the computation finishes we have 0 pending tasks so we will need 0 > executors. > {code} > 16/06/21 04:07:41 INFO yarn.YarnAllocator: Driver requested a total number of > 0 executor(s). > 16/06/21 04:07:41 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:07:41 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > ... > 16/06/21 04:08:41 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:08:41 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > 16/06/21 04:08:41 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to > kill executor(s) 1. > 16/06/21 04:08:48 INFO yarn.YarnAllocator: Driver requested a total number of > 0 executor(s). > 16/06/21 04:08:48 INFO yarn.YarnAllocator: Canceling requests for 0 executor > containers > 16/06/21 04:08:48 WARN yarn.YarnAllocator: Expected to find pending requests, > but found none. > 16/06/21 04:08:48 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to > kill executor(s) 2. > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org