Hi community,
I have a question is that I cancel the flink task and retain the
checkpoint dir, then restore from the checkpoint dir ,can I change the
flink operator's parallelism,in my thoughts, I think I can't change the
flink operator's parallelism,but I am not sure.
Thanks to your rep
I think you can modify the operator’s parallelism. It is only if you have
set maxParallelism, and while restoring from a checkpoint, you shouldn’t
modify the maxParallelism. Otherwise, I believe the state will be lost.
-
Sivaprasanna
On Fri, 13 Mar 2020 at 9:01 AM, LakeShen wrote:
> Hi communit
Hi,
Yes, you can change the parallelism. One thing that you can not change is “max
parallelism”.
Piotrek
> On 13 Mar 2020, at 04:34, Sivaprasanna wrote:
>
> I think you can modify the operator’s parallelism. It is only if you have set
> maxParallelism, and while restoring from a checkpoint,
Hi Piotr,
Does this also apply to savepoint? (meaning the max parallelism should not
change for job resume from savepoint?)
Thanks a lot!
Eleanore
On Fri, Mar 13, 2020 at 6:33 AM Piotr Nowojski wrote:
> Hi,
>
> Yes, you can change the parallelism. One thing that you can not change is
> “max par
Hi Eleanore , if you resume from savepoint , you can't change the flink
operator's max parallelism .
Eleanore Jin 于2020年3月14日周六 上午12:51写道：
> Hi Piotr,
> Does this also apply to savepoint? (meaning the max parallelism should not
> change for job resume from savepoint?)
>
> Thanks a lot!
> Eleanor
If you want to change the max parallelism then you need to take a savepoint
and use Flink's state processor API [1] to rewrite the max parallelism by
creating a new savepoint from the old one.
[1]
https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/state_processor_api.html
Cheers,
Til
