Thanks for sharing your feedback about FLIP-461 [1]. I addressed the
comments (also renaming the CheckpointLifecycleListener into
CheckpointStatsListener to have a less generic name for this internal
interface).
I will go ahead and start a vote on this proposal.
Matthias
[1]
Hi Zakelly,
good point. I updated the FLIP to use "scale-on-failed-checkpoints-count"
and "max-delay-for-scale-trigger".
On Fri, Jun 7, 2024 at 12:18 PM Zakelly Lan wrote:
> Hi Matthias,
>
> Thanks for your reply!
>
> That's something that could be considered as another optimization. I would
>
Hi Matthias,
Thanks for your reply!
That's something that could be considered as another optimization. I would
> consider this as a possible follow-up. My concern here is that we'd make
> the rescaling configuration even more complicated by introducing yet
> another parameter.
I'd be fine with
Thanks for the FLIP Matthias, I think it looks pretty solid!
I also don't see a relation to unaligned checkpoints. From the AS
perspective, the checkpoint time doesn't matter.
Is it possible a change event observed right after a complete checkpoint
> (or within a specific short time after a
Hi Zakelly,
thanks for your reply. See my inlined responses below:
On Wed, Jun 5, 2024 at 10:26 AM Zakelly Lan wrote:
> Hi Matthias,
>
> Thanks for your proposal! I have a few questions:
>
> 1. Is it possible a change event observed right after a complete checkpoint
> (or within a specific
Thanks Rui for your reply. Find my answers inlined below:
[1]
https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#jobmanager-adaptive-scheduler-resource-stabilization-timeout
On Wed, Jun 5, 2024 at 10:16 AM Rui Fan <1996fan...@gmail.com> wrote:
> Thanks Matthias for
Hi Matthias,
Thanks for your proposal! I have a few questions:
1. Is it possible a change event observed right after a complete checkpoint
(or within a specific short time after a checkpoint) that triggers a
rescale immediately? Sometimes the checkpoint interval is huge and it is
better to
Thanks Matthias for driving this proposal!
This proposal can reduce the amount of data that is processed repeatedly
after rescaling, so this proposal makes sense to me.
I have some questions:
1. The public change only includes the "New Configuration Parameters" part,
right?
2.
Hi ConradJam,
thanks for your response.
The CheckpointStatsTracker gets notified about the checkpoint completion
after the checkpoint is finalized, i.e. all its data is persisted and the
metadata is written to the CompletedCheckpointStore. At this moment, the
checkpoint is considered for
I have a few questions:
Unaligned checkpoints Do we need to enable this feature? Whether this
feature should be disabled for checkpoints that do not check it
Matthias Pohl 于2024年6月4日周二 18:03写道：
> Hi everyone,
> I'd like to discuss FLIP-461 [1]. The FLIP proposes the synchronization of
>
Hi everyone,
I'd like to discuss FLIP-461 [1]. The FLIP proposes the synchronization of
rescaling and the completion of checkpoints. The idea is to reduce the
amount of data that needs to be processed after rescaling happened. A more
detailed motivation can be found in FLIP-461.
I'm looking
11 matches
Mail list logo
