Saisai Shao created SPARK-14788: ----------------------------------- Summary: Initial number of executors should honor min number if streaming dynamic allocation is enabled Key: SPARK-14788 URL: https://issues.apache.org/jira/browse/SPARK-14788 Project: Spark Issue Type: Bug Components: Streaming Affects Versions: 2.0.0 Reporter: Saisai Shao
Streaming dynamic allocation has lower bound {{spark.streaming.dynamicAllocation.minExecutors}} and upper bound {{spark.streaming.dynamicAllocation.maxExecutors}} of executor number. But seems it is not used when starting on yarn or other cluster manager, I think we should honor {{minExecutors}} as an initial number of executors for starting if streaming dynamic allocation is enabled, like what we did in Spark dynamic allocation. >From my understanding this is an issue should be fixed, but I'm not sure it is >a by-design choice, what is your opinion [~andrewor14] and [~tdas] ? -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org