[
https://issues.apache.org/jira/browse/FLINK-25470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17524033#comment-17524033
]
Yun Tang commented on FLINK-25470:
----------------------------------
I think the incremental size of materialization part is something easy to mix
with incremental checkpoint. Actually, the time to execute materialization has
no direct relationship with checkpoint. And I think the full size of
materialization part within checkpoint is enough.
We can introduce the incremental materialization size within the changelog
part. Apart from the checkpoint related metrics, we can also have changelog
related metrics. The changelog related metrics only compare with each other, in
other words, the x-axis of timestamp is only related with time to execute
materalization on task side.
> Add/Expose/Differentiate metrics of checkpoint size between changelog size vs
> materialization size
> --------------------------------------------------------------------------------------------------
>
> Key: FLINK-25470
> URL: https://issues.apache.org/jira/browse/FLINK-25470
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics, Runtime / State Backends
> Reporter: Yuan Mei
> Priority: Major
> Fix For: 1.16.0
>
> Attachments: Screen Shot 2021-12-29 at 1.09.48 PM.png
>
>
> FLINK-25557 only resolves part of the problems.
> Eventually, we should answer questions:
> * How much Data Size increases/exploding
> * When a checkpoint includes a new Materialization
> * Materialization size
> * changelog sizes from the last complete checkpoint (that can roughly infer
> restore time)
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
