rkhachatryan commented on code in PR #540: URL: https://github.com/apache/flink-web/pull/540#discussion_r879903747
########## _posts/2022-05-20-changelog-state-backend.md: ########## @@ -0,0 +1,353 @@ +--- +layout: post +title: "Improving speed and stability of checkpointing with generic log-based incremental checkpoints" +date: 2022-05-20T00:00:00.000Z +authors: +- Roman Khachatryan: + name: "Roman Khachatryan" +- Yuan Mei: + name: "Yuan Mei" +excerpt: This post describes the mechanism introduced in Flink 1.15 that continuously uploads state changes to a durable storage while performing materialization in the background + +--- + +# Introduction + +One of the most important characteristics of stream processing systems is end-to-end latency, i.e. the time it takes for the results of processing an input record to reach the outputs. In the case of Flink, end-to-end latency mostly depends on the checkpointing mechanism, because processing results should only become visible after the state of the stream is persisted to non-volatile storage (this is assuming exactly-once mode; in other modes, results can be published immediately). + +Furthermore, сheckpoint duration also defines the reasonable interval with which checkpoints are made. A shorter interval provides the following advantages: + +* Lower latency for transactional sinks: Transactional sinks commit on checkpoints, so faster checkpoints mean more frequent commits. +* More predictable checkpoint intervals: Currently the length of the checkpoint depends on the size of the artifacts that need to be persisted in the checkpoint storage. Review Comment: Agreed, replaced "length" with "duration" (uploading is mentioned right after it). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
