Re: Restart from checkpoint - kinesis consumer

2022-02-23 Thread Vijayendra Yadav
Thanks Danny, Let me comeback with results. > > On Feb 23, 2022, at 3:41 AM, Danny Cranmer wrote: > >  > Hello Vijay, > > > Once i do that my flink consumer need to be restarted with changed > > parallelism. > Why is this? The Flink consumer continuously scans for new shards, and will > aut

Re: Restart from checkpoint - kinesis consumer

2022-02-23 Thread Danny Cranmer
Hello Vijay, > Once i do that my flink consumer need to be restarted with changed parallelism. Why is this? The Flink consumer continuously scans for new shards, and will auto scale up/down the number of shard consumer threads to accommodate Kinesis resharding. Flink job/operator parallelism does

Restart from checkpoint - kinesis consumer

2022-02-23 Thread Vijayendra Yadav
Hi Team, I am running flink 1.11 kinesis consumer with say N kinesis shards, but i want to increase/decrease shards to N+M or N-M. Once i do that my flink consumer need to be restarted with changed parallelism. But i am unable to restart from existing checkpoint because of change in number of s