Hi Lihe, What happens if we enable incremental checkpoints by default while the used memory backend is HashMapStateBackend, which doesn't support incremental checkpoints?
Best regards, Martijn Op ma 13 jun. 2022 om 11:59 schreef Lihe Ma <ma_l...@163.com>: > Hi, Everyone, > > I would like to open a discussion on setting incremental checkpoint as > default behavior. > > Currently, the configuration of state.backend.incremental is set as false > by default. Incremental checkpoint has been adopted widely in industry > community for many years , and it is also well-tested from the feedback in > the community discussion. Incremental checkpointing is more light-weighted: > shorter checkpoint duration, less uploaded data and less resource > consumption. > > In terms of backward compatibility, enable incremental checkpointing would > not make any data loss no matter restoring from a full checkpoint/savepoint > or an incremental checkpoint. > > FLIP-193 (Snapshot ownership)[1] has been released in 1.15, incremental > checkpoint no longer depends on a previous restored checkpoint in default > NO_CLAIM mode, which makes the checkpoint lineage much cleaner, it is a > good chance to change the configuration state.backend.incremental to true > as default. > > Thus, based on the above discussion, I suggest to make > state.backend.incremental as true by default. What do you think of this > proposal? > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-193%3A+Snapshots+ownership > > Best regards, > Lihe Ma >
