Re: Support for Auto scaling

2017-02-02 Thread Till Rohrmann
Hi Sandeep, it's also not yet supported to dynamically start new TaskManagers when needed. However, it can be done manually via the taskmanager.sh script. We're working to enable this feature for 1.3 if everything goes according to plan. Cheers, Till On Thu, Feb 2, 2017 at 6:49 AM, Tzu-Li

Re: Support for Auto scaling

2017-02-01 Thread Tzu-Li (Gordon) Tai
Hi Sandeep! While auto scaling jobs in Flink still isn’t possible, in Flink 1.2 you will be able to rescale jobs by stopping and restarting. This works by taking a savepoint of the job before stopping the job, and then redeploy the job with a higher / lower parallelism using the savepoint. Upon

Support for Auto scaling

2017-02-01 Thread Meghashyam Sandeep V
Hi Guys, I currently run flink 1.1.4 streaming jobs in EMR in AWS with yarn. I understand that EMR supports auto scaling but Flink doesn't. Is there a plan for this support in 1.2. Thanks, Sandeep