[ https://issues.apache.org/jira/browse/FLINK-27557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jane Chan updated FLINK-27557: ------------------------------ Summary: Submit compaction when prepareCommit is triggered (was: Trigger compaction when prepareCommit is triggered) > Submit compaction when prepareCommit is triggered > ------------------------------------------------- > > Key: FLINK-27557 > URL: https://issues.apache.org/jira/browse/FLINK-27557 > Project: Flink > Issue Type: Sub-task > Components: Table Store > Affects Versions: table-store-0.2.0 > Reporter: Jane Chan > Priority: Major > Fix For: table-store-0.2.0 > > Attachments: normal-compaction.png, optimized-normal-compaction.png, > rescale-bucket-compaction.png > > > Currently, FileStoreWrite will scan and plan files when creating a non-empty > writer. We should also create a non-empty writer for non-rescale compaction > cases but instead use the pre-planned manifest entries. > The sink records should be dumped, and when prepareCommit is invoked, it > should trigger submitCompaction. This will generate two snapshots, the first > commit kind is "ADD" with empty new files, and the second commit kind is > "COMPACT" with specified manifest entries marked as deleted and compacted > files marked as added. -- This message was sent by Atlassian Jira (v8.20.7#820007)