Hongshun Wang created FLINK-35129:
-------------------------------------
Summary: Postgres source commits the offset after every multiple
checkpoint cycles.
Key: FLINK-35129
URL: https://issues.apache.org/jira/browse/FLINK-35129
Project: Flink
Issue Type: Improvement
Components: Flink CDC
Reporter: Hongshun Wang
After entering the Stream phase, the offset consumed by the global slot is
committed upon the completion of each checkpoint, preventing log files from
being unable to be recycled continuously, which could lead to insufficient disk
space.
However, the job can only restart from the latest checkpoint or savepoint. if
restored from an earlier state, WAL may already have been recycled.
The way to solve it is to commit the offset after every multiple checkpoint
cycles. The number of checkpoint cycles is determine by connector option, and
the default value is 3.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
