[
https://issues.apache.org/jira/browse/SPARK-14723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-14723:
------------------------------
Target Version/s: (was: 2.1.0)
Labels: (was: features)
Fix Version/s: (was: 2.1.0)
I'm not sure I agree that the goal of a streaming job is to make the processing
time exactly fit the batch interval. Finishing sooner is generally the goal,
and taking nowhere near the batch interval to finish for safety. That is, I
think the current behavior -- adding executors when there's work to do -- is
more ideal than hoping to guess the right number of executors based on previous
runtimes.
> A new way to support dynamic allocation in Spark Streaming
> ----------------------------------------------------------
>
> Key: SPARK-14723
> URL: https://issues.apache.org/jira/browse/SPARK-14723
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core, Streaming
> Reporter: WilliamZhu
> Attachments: spark-streaming-dynamic-allocation-desigh.pdf
>
>
> Provide a more powerful Algorithm to support dynamic allocation in spark
> streaming.
> more details: http://www.jianshu.com/p/ae7fdd4746f6
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
