Re: Changing parallelism

2017-01-10 Thread Fabian Hueske
en or otherwise (from > an > operator). > > Are there APIs to inspect the savepoints - using offline programs? > > -Abhishek- > > > > -- > View this message in context: http://apache-flink-user- > mailing-list-archive.2336050.n4.nabble.com/Changing- > parallelism-tp49

Re: Changing parallelism

2017-01-08 Thread abhishekrs
- using offline programs? -Abhishek- -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Changing-parallelism-tp4967p10911.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Changing parallelism

2016-02-18 Thread Zach Cox
Hi Ufuk - thanks for the 2016 roadmap - glad to see changing parallelism is the first bullet :) Mesos support also sounds great, we're currently running job and task managers on Mesos statically via Marathon. Hi Stephan - thanks, that trick sounds pretty clever, I will try wrapping my head

Re: Changing parallelism

2016-02-18 Thread Stephan Ewen
Hi Zach! Yes, changing parallelism is pretty high up the priority list. The good news is that "scaling in" is the simpler part of changing the parallelism and we are pushing to get that in soon. Until then, there is only a pretty ugly trick that you can do right now to "rescale'

Re: Changing parallelism

2016-02-18 Thread Ufuk Celebi
Hey Zach! Sounds like a great use case. On Wed, Feb 17, 2016 at 3:16 PM, Zach Cox wrote: > However, the savepoint docs state that the job parallelism cannot be changed > over time [1]. Does this mean we need to use the same, fixed parallelism=n > during reprocessing and going

Changing parallelism

2016-02-17 Thread Zach Cox
Hi - we are building a stateful Flink streaming job that will run indefinitely. One part of the job builds up state per key in a global window that will need to exist for a very long time. We will definitely be using the savepoints to restore job state after new code deploys. We were planning to