Hi
Maybe there contains some ByteStreamStateHandle in the checkpoint, if you
want to verify this, maybe you can configure
`state.backend.fs.memory-threshold` to verify it. Please be careful to set
this config, because it may produce many files with small size.
Best,
Congxian
Arvid Heise
Hi Yu,
are you using incremental checkpoints [1]? If so, then the smaller
checkpoints would be the deltas and the larger the complete state.
[1]
https://flink.apache.org/features/2018/01/30/incremental-checkpointing.html
On Wed, Mar 4, 2020 at 6:41 PM Yu Yang wrote:
> Hi all,
>
> We have a
Hi all,
We have a flink job that does check-pointing per 10 minutes. We noticed
that for the check-points of this job, the _metadata file size can vary a
lot. In some checkpoint, we observe that _metadata file size was >900MB,
while in some other check-points of the same job, the _metadata file
