[ https://issues.apache.org/jira/browse/FLINK-35129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837942#comment-17837942 ]
Hongshun Wang commented on FLINK-35129: --------------------------------------- [~m.orazow] , would you like to take it? > Postgres source commits the offset after every multiple checkpoint cycles. > -------------------------------------------------------------------------- > > Key: FLINK-35129 > URL: https://issues.apache.org/jira/browse/FLINK-35129 > Project: Flink > Issue Type: Improvement > Components: Flink CDC > Reporter: Hongshun Wang > Priority: Major > > After entering the Stream phase, the offset consumed by the global slot is > committed upon the completion of each checkpoint, preventing log files from > being unable to be recycled continuously, which could lead to insufficient > disk space. > However, the job can only restart from the latest checkpoint or savepoint. if > restored from an earlier state, WAL may already have been recycled. > > The way to solve it is to commit the offset after every multiple checkpoint > cycles. The number of checkpoint cycles is determine by connector option, and > the default value is 3. -- This message was sent by Atlassian Jira (v8.20.10#820010)