Thank You Andrey.
Regards,
Vijay
> On Aug 29, 2020, at 3:38 AM, Andrey Zagrebin wrote:
>
>
> Hi Vijay,
>
> I would apply the same judgement. It is latency vs throughput vs spent
> resources vs practical need.
>
> The more concurrent checkpoints your system is capable of handling, the
>
Hi Vijay,
I would apply the same judgement. It is latency vs throughput vs spent
resources vs practical need.
The more concurrent checkpoints your system is capable of handling, the
better end-to-end result latency you will observe and see computation
results more frequently.
On the other hand
Hi Andrey,
Thanks,
what is recommendation for : env.getCheckpointConfig.
*setMaxConcurrentCheckpoints*(concurrentchckpt) ?
1 or higher based on what factor.
Regards,
Vijay
On Tue, Aug 25, 2020 at 8:55 AM Andrey Zagrebin
wrote:
> Hi Vijay,
>
> I think it depends on your job requirements,
Hi Vijay,
I think it depends on your job requirements, in particular how many records
are processed per second and how much resources you have to process them.
If the checkpointing interval is short then the checkpointing overhead can
be too high and you need more resources to efficiently keep
Hi Team,
Bulk Formats can only have `OnCheckpointRollingPolicy`, which rolls (ONLY)
on every checkpoint.
*.withRollingPolicy(OnCheckpointRollingPolicy.build())*
Question: What are recommended values related to checkpointing to fsstate,
should it be more frequent checkpoints, or longer
