Re: Cancel the flink task and restore from checkpoint ,can I change the flink operator's parallelism

[LakeShen]

Thank you, I will do that.

jinhai wang  于2020年3月17日周二 下午5:58写道：

> Hi LakeShen
>
> You also must assign IDs to all operators of an application. Otherwise,
> you may not be able to recover from checkpoint
>
> Doc:
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/upgrading.html#matching-operator-state
>
>
> Best Regards
>
> jinhai...@gmail.com
>
> 2020年3月17日 下午5:51，Till Rohrmann  写道：
>
> Let us know if you should run into any problems. The state processor API
> is still young and could benefit from as much feedback as possible.
>
> Cheers,
> Till
>
> On Tue, Mar 17, 2020 at 2:57 AM LakeShen 
> wrote:
>
>> Wow,this feature is look so good,I will see this feature.
>> Thank you to reply me , Till😀.
>>
>> Best Regards,
>> LakeShen
>>
>> Till Rohrmann  于2020年3月17日周二 上午4:17写道：
>>
>>> If you want to change the max parallelism then you need to take a
>>> savepoint and use Flink's state processor API [1] to rewrite the max
>>> parallelism by creating a new savepoint from the old one.
>>>
>>> [1]
>>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/state_processor_api.html
>>>
>>> Cheers,
>>> Till
>>>
>>> On Sat, Mar 14, 2020 at 4:07 AM LakeShen 
>>> wrote:
>>>
 Hi Eleanore , if you resume from savepoint , you can't change the flink
 operator's max parallelism .

 Eleanore Jin  于2020年3月14日周六 上午12:51写道：

 > Hi Piotr,
 > Does this also apply to savepoint? (meaning the max parallelism
 should not
 > change for job resume from savepoint?)
 >
 > Thanks a lot!
 > Eleanore
 >
 > On Fri, Mar 13, 2020 at 6:33 AM Piotr Nowojski 
 > wrote:
 >
 > > Hi,
 > >
 > > Yes, you can change the parallelism. One thing that you can not
 change is
 > > “max parallelism”.
 > >
 > > Piotrek
 > >
 > > > On 13 Mar 2020, at 04:34, Sivaprasanna >>> >
 > > wrote:
 > > >
 > > > I think you can modify the operator’s parallelism. It is only if
 you
 > > have set maxParallelism, and while restoring from a checkpoint, you
 > > shouldn’t modify the maxParallelism. Otherwise, I believe the state
 will
 > be
 > > lost.
 > > >
 > > > -
 > > > Sivaprasanna
 > > >
 > > > On Fri, 13 Mar 2020 at 9:01 AM, LakeShen <
 shenleifight...@gmail.com
 > > > wrote:
 > > > Hi community,
 > > >   I have a question is that I cancel the flink task and
 retain the
 > > checkpoint dir, then restore from the checkpoint dir ,can I change
 the
 > > flink operator's parallelism,in my thoughts, I think I can't change
 the
 > > flink operator's parallelism,but I am not sure.
 > > >  Thanks to your reply.
 > > >
 > > > Best wishes,
 > > > LakeShen
 > >
 > >
 >

>>>
>

Re: Cancel the flink task and restore from checkpoint ,can I change the flink operator's parallelism

[jinhai wang]

Hi LakeShen

You also must assign IDs to all operators of an application. Otherwise, you may 
not be able to recover from checkpoint

Doc: 
https://ci.apache.org/projects/flink/flink-docs-stable/ops/upgrading.html#matching-operator-state
 


Best Regards

jinhai...@gmail.com

> 2020年3月17日 下午5:51，Till Rohrmann  写道：
> 
> Let us know if you should run into any problems. The state processor API is 
> still young and could benefit from as much feedback as possible.
> 
> Cheers,
> Till
> 
> On Tue, Mar 17, 2020 at 2:57 AM LakeShen  > wrote:
> Wow,this feature is look so good,I will see this feature.
> Thank you to reply me , Till😀.
> 
> Best Regards,
> LakeShen
> 
> Till Rohrmann mailto:trohrm...@apache.org>> 
> 于2020年3月17日周二 上午4:17写道：
> If you want to change the max parallelism then you need to take a savepoint 
> and use Flink's state processor API [1] to rewrite the max parallelism by 
> creating a new savepoint from the old one.
> 
> [1] 
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/state_processor_api.html
>  
> 
> 
> Cheers,
> Till
> 
> On Sat, Mar 14, 2020 at 4:07 AM LakeShen  > wrote:
> Hi Eleanore , if you resume from savepoint , you can't change the flink
> operator's max parallelism .
> 
> Eleanore Jin mailto:eleanore@gmail.com>> 
> 于2020年3月14日周六 上午12:51写道：
> 
> > Hi Piotr,
> > Does this also apply to savepoint? (meaning the max parallelism should not
> > change for job resume from savepoint?)
> >
> > Thanks a lot!
> > Eleanore
> >
> > On Fri, Mar 13, 2020 at 6:33 AM Piotr Nowojski  > >
> > wrote:
> >
> > > Hi,
> > >
> > > Yes, you can change the parallelism. One thing that you can not change is
> > > “max parallelism”.
> > >
> > > Piotrek
> > >
> > > > On 13 Mar 2020, at 04:34, Sivaprasanna  > > > >
> > > wrote:
> > > >
> > > > I think you can modify the operator’s parallelism. It is only if you
> > > have set maxParallelism, and while restoring from a checkpoint, you
> > > shouldn’t modify the maxParallelism. Otherwise, I believe the state will
> > be
> > > lost.
> > > >
> > > > -
> > > > Sivaprasanna
> > > >
> > > > On Fri, 13 Mar 2020 at 9:01 AM, LakeShen  > > > 
> > > >> 
> > > wrote:
> > > > Hi community,
> > > >   I have a question is that I cancel the flink task and retain the
> > > checkpoint dir, then restore from the checkpoint dir ,can I change the
> > > flink operator's parallelism,in my thoughts, I think I can't change the
> > > flink operator's parallelism,but I am not sure.
> > > >  Thanks to your reply.
> > > >
> > > > Best wishes,
> > > > LakeShen
> > >
> > >
> >

Re: Cancel the flink task and restore from checkpoint ,can I change the flink operator's parallelism

[Till Rohrmann]

Let us know if you should run into any problems. The state processor API is
still young and could benefit from as much feedback as possible.

Cheers,
Till

On Tue, Mar 17, 2020 at 2:57 AM LakeShen  wrote:

> Wow,this feature is look so good,I will see this feature.
> Thank you to reply me , Till😀.
>
> Best Regards,
> LakeShen
>
> Till Rohrmann  于2020年3月17日周二 上午4:17写道：
>
>> If you want to change the max parallelism then you need to take a
>> savepoint and use Flink's state processor API [1] to rewrite the max
>> parallelism by creating a new savepoint from the old one.
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/state_processor_api.html
>>
>> Cheers,
>> Till
>>
>> On Sat, Mar 14, 2020 at 4:07 AM LakeShen 
>> wrote:
>>
>>> Hi Eleanore , if you resume from savepoint , you can't change the flink
>>> operator's max parallelism .
>>>
>>> Eleanore Jin  于2020年3月14日周六 上午12:51写道：
>>>
>>> > Hi Piotr,
>>> > Does this also apply to savepoint? (meaning the max parallelism should
>>> not
>>> > change for job resume from savepoint?)
>>> >
>>> > Thanks a lot!
>>> > Eleanore
>>> >
>>> > On Fri, Mar 13, 2020 at 6:33 AM Piotr Nowojski 
>>> > wrote:
>>> >
>>> > > Hi,
>>> > >
>>> > > Yes, you can change the parallelism. One thing that you can not
>>> change is
>>> > > “max parallelism”.
>>> > >
>>> > > Piotrek
>>> > >
>>> > > > On 13 Mar 2020, at 04:34, Sivaprasanna 
>>> > > wrote:
>>> > > >
>>> > > > I think you can modify the operator’s parallelism. It is only if
>>> you
>>> > > have set maxParallelism, and while restoring from a checkpoint, you
>>> > > shouldn’t modify the maxParallelism. Otherwise, I believe the state
>>> will
>>> > be
>>> > > lost.
>>> > > >
>>> > > > -
>>> > > > Sivaprasanna
>>> > > >
>>> > > > On Fri, 13 Mar 2020 at 9:01 AM, LakeShen <
>>> shenleifight...@gmail.com
>>> > > > wrote:
>>> > > > Hi community,
>>> > > >   I have a question is that I cancel the flink task and retain
>>> the
>>> > > checkpoint dir, then restore from the checkpoint dir ,can I change
>>> the
>>> > > flink operator's parallelism,in my thoughts, I think I can't change
>>> the
>>> > > flink operator's parallelism,but I am not sure.
>>> > > >  Thanks to your reply.
>>> > > >
>>> > > > Best wishes,
>>> > > > LakeShen
>>> > >
>>> > >
>>> >
>>>
>>

Re: Cancel the flink task and restore from checkpoint ,can I change the flink operator's parallelism

[Till Rohrmann]

If you want to change the max parallelism then you need to take a savepoint
and use Flink's state processor API [1] to rewrite the max parallelism by
creating a new savepoint from the old one.

[1]
https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/state_processor_api.html

Cheers,
Till

On Sat, Mar 14, 2020 at 4:07 AM LakeShen  wrote:

> Hi Eleanore , if you resume from savepoint , you can't change the flink
> operator's max parallelism .
>
> Eleanore Jin  于2020年3月14日周六 上午12:51写道：
>
> > Hi Piotr,
> > Does this also apply to savepoint? (meaning the max parallelism should
> not
> > change for job resume from savepoint?)
> >
> > Thanks a lot!
> > Eleanore
> >
> > On Fri, Mar 13, 2020 at 6:33 AM Piotr Nowojski 
> > wrote:
> >
> > > Hi,
> > >
> > > Yes, you can change the parallelism. One thing that you can not change
> is
> > > “max parallelism”.
> > >
> > > Piotrek
> > >
> > > > On 13 Mar 2020, at 04:34, Sivaprasanna 
> > > wrote:
> > > >
> > > > I think you can modify the operator’s parallelism. It is only if you
> > > have set maxParallelism, and while restoring from a checkpoint, you
> > > shouldn’t modify the maxParallelism. Otherwise, I believe the state
> will
> > be
> > > lost.
> > > >
> > > > -
> > > > Sivaprasanna
> > > >
> > > > On Fri, 13 Mar 2020 at 9:01 AM, LakeShen  > > > wrote:
> > > > Hi community,
> > > >   I have a question is that I cancel the flink task and retain
> the
> > > checkpoint dir, then restore from the checkpoint dir ,can I change the
> > > flink operator's parallelism,in my thoughts, I think I can't change the
> > > flink operator's parallelism,but I am not sure.
> > > >  Thanks to your reply.
> > > >
> > > > Best wishes,
> > > > LakeShen
> > >
> > >
> >
>

Re: Cancel the flink task and restore from checkpoint ,can I change the flink operator's parallelism

[Piotr Nowojski]

Hi,

Yes, you can change the parallelism. One thing that you can not change is “max 
parallelism”.

Piotrek

> On 13 Mar 2020, at 04:34, Sivaprasanna  wrote:
> 
> I think you can modify the operator’s parallelism. It is only if you have set 
> maxParallelism, and while restoring from a checkpoint, you shouldn’t modify 
> the maxParallelism. Otherwise, I believe the state will be lost.
> 
> -
> Sivaprasanna 
> 
> On Fri, 13 Mar 2020 at 9:01 AM, LakeShen  > wrote:
> Hi community,
>   I have a question is that I cancel the flink task and retain the 
> checkpoint dir, then restore from the checkpoint dir ,can I change the flink 
> operator's parallelism,in my thoughts, I think I can't change the flink 
> operator's parallelism,but I am not sure.
>  Thanks to your reply.
> 
> Best wishes,
> LakeShen

Re: Cancel the flink task and restore from checkpoint ,can I change the flink operator's parallelism

[Sivaprasanna]

I think you can modify the operator’s parallelism. It is only if you have
set maxParallelism, and while restoring from a checkpoint, you shouldn’t
modify the maxParallelism. Otherwise, I believe the state will be lost.

-
Sivaprasanna

On Fri, 13 Mar 2020 at 9:01 AM, LakeShen  wrote:

> Hi community,
>   I have a question is that I cancel the flink task and retain the
> checkpoint dir, then restore from the checkpoint dir ,can I change the
> flink operator's parallelism,in my thoughts, I think I can't change the
> flink operator's parallelism,but I am not sure.
>  Thanks to your reply.
>
> Best wishes,
> LakeShen
>

Cancel the flink task and restore from checkpoint ,can I change the flink operator's parallelism

[LakeShen]

Hi community,
  I have a question is that I cancel the flink task and retain the
checkpoint dir, then restore from the checkpoint dir ,can I change the
flink operator's parallelism,in my thoughts, I think I can't change the
flink operator's parallelism,but I am not sure.
 Thanks to your reply.

Best wishes,
LakeShen