JingsongLi opened a new pull request, #10686: URL: https://github.com/apache/paimon/pull/10686
### Purpose Data-evolution compaction and clustering currently put their change preparation and index-remapping logic directly in `FileStoreCommitImpl`. Extract that logic into `DataEvolutionCommitChangesRewriter`, injected by `AbstractFileStore` through a small `CommitChangesRewriter` interface. Ordinary tables use the identity implementation. Keep conflict detection, final RowID assignment, manifest preparation, optimistic retry, publication, and file cleanup in the existing committer. Reuse one changes provider across retries while creating fresh preparation state for each attempt. Remap indexes after the final RowIDs are assigned, and register new files before writing so pre-publication failures clean them up without deleting files after an uncertain publication result. ### Tests - Add eight commit-extension cases covering concurrent append and optimistic retry, final RowIDs and index visibility, read-only inputs, rewrite/manifest/pre-callback failure cleanup, and successful publication with a lost false/exception response. - Run the complete `FileStoreCommitTest`, `DataEvolutionClusterTest`, `DataEvolutionDeletionVectorTest`, `DataEvolutionRowIdReassignerTest`, `DataEvolutionNormalCompactTaskTest`, `DataEvolutionClusterCompactionTest`, and `RowIdRangeConflictCheckerTest` suites alongside the new tests (299 cases), including concurrent index creation and repeated retries. - Run 12 Flink integration cases covering ordinary DV clustering, DE scalar-index remapping with local/global sort, repeated column updates, combined clustering/compaction, and unselected partitions. - Run eight Spark 3.5 integration cases covering ordinary clustering, DV materialization across planner batches, DE compact rebase/retry and failure cleanup, and DE clustering with local/global sort. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
