Yun Tang created FLINK-25524: -------------------------------- Summary: If enabled changelog, RocksDB incremental checkpoint would always be full Key: FLINK-25524 URL: https://issues.apache.org/jira/browse/FLINK-25524 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing, Runtime / State Backends Reporter: Yun Tang
Once changelog is enabled, RocksDB incremental checkpoint would only be executed during materialization. During this phase, it will leverage the {{materization id}} as the checkpoint id for RocksDB state backend's snapshot method. However, current incremental checkpoint mechanism heavily depends on the checkpoint id. And {{SortedMap<Long, Set<StateHandleID>> uploadedStateIDs}} with checkpoint id as the key within {{RocksIncrementalSnapshotStrategy}} is the kernel for incremental checkpoint. Once we notify checkpoint complete of previous checkpoint, it will then remove the uploaded stateIds of that checkpoint, leading to we cannot get proper checkpoint information on the next RocksDBKeyedStateBackend#snapshot. That is to say, we will always upload all RocksDB artifacts. -- This message was sent by Atlassian Jira (v8.20.1#820001)