[ https://issues.apache.org/jira/browse/SPARK-10644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14900277#comment-14900277 ]
Madhusudanan Kandasamy commented on SPARK-10644: ------------------------------------------------ Standalone cluster manager launches executors during application's registration. So when you say Number of executors: 63, do you mean total number of cores available across all the workers? Can you give the values for spark.executor.cores and spark.cores.max. > Applications wait even if free executors are available > ------------------------------------------------------ > > Key: SPARK-10644 > URL: https://issues.apache.org/jira/browse/SPARK-10644 > Project: Spark > Issue Type: Bug > Components: Scheduler > Affects Versions: 1.5.0 > Environment: RHEL 6.5 64 bit > Reporter: Balagopal Nair > > Number of workers: 21 > Number of executors: 63 > Steps to reproduce: > 1. Run 4 jobs each with max cores set to 10 > 2. The first 3 jobs run with 10 each. (30 executors consumed so far) > 3. The 4 th job waits even though there are 33 idle executors. > The reason is that a job will not get executors unless > the total number of EXECUTORS in use < the number of WORKERS > If there are executors available, resources should be allocated to the > pending job. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org