Hi Richard, Thanks for the confirmation.
However, I believe you must be facing issue as in JIRA 22008. Regards, Sourav Sent from my iPhone > On Dec 3, 2017, at 9:39 AM, Qiao, Richard <richard.q...@capitalone.com> wrote: > > Sourav: > I’m using spark streaming 2.1.0 and can confirm > spark.dynamicAllocation.enabled is enough. > > Best Regards > Richard > > From: Sourav Mazumder <sourav.mazumde...@gmail.com> > Date: Sunday, December 3, 2017 at 12:31 PM > To: user <user@spark.apache.org> > Subject: Dynamic Resource allocation in Spark Streaming > > Hi, > > I see the following jira is resolved in Spark 2.0 > https://issues.apache.org/jira/browse/SPARK-12133 which is supposed to > support Dynamic Resource Allocation in Spark Streaming. > > I also see the JiRA https://issues.apache.org/jira/browse/SPARK-22008 which > is about fixing numer of executor related issue in dynamic allocation in > Spark Streaming > > But when I check http://spark.apache.org/docs/2.1.1/configuration.html (or > teh same for 2.2) I don;t see the configuration parameter > park.streaming.dynamicAllocation.enabled. > > Is this feature there at all in Spark 2.0 ? Or setting > spark.dynamicAllocation.enabled is good enough ? > > Regards, > Sourav > > The information contained in this e-mail is confidential and/or proprietary > to Capital One and/or its affiliates and may only be used solely in > performance of work or services for Capital One. The information transmitted > herewith is intended only for use by the individual or entity to which it is > addressed. If the reader of this message is not the intended recipient, you > are hereby notified that any review, retransmission, dissemination, > distribution, copying or other use of, or taking of any action in reliance > upon this information is strictly prohibited. If you have received this > communication in error, please contact the sender and delete the material > from your computer.
