Zerolzj commented on code in PR #10582:
URL: https://github.com/apache/paimon/pull/10582#discussion_r4291905309
##########
paimon-core/src/main/java/org/apache/paimon/operation/FileStoreCommitImpl.java:
##########
@@ -725,7 +726,14 @@ public void abort(List<CommitMessage> commitMessages) {
List<DataFileMeta> dataFilesToDelete = new ArrayList<>();
dataFilesToDelete.addAll(commitMessage.newFilesIncrement().newFiles());
dataFilesToDelete.addAll(commitMessage.newFilesIncrement().changelogFiles());
-
dataFilesToDelete.addAll(commitMessage.compactIncrement().compactAfter());
+ // Level upgrades reuse files which are still referenced by the
previous snapshot.
+ Set<Path> compactInputs =
+ commitMessage.compactIncrement().compactBefore().stream()
+ .map(dataPathFactory::toPath)
+ .collect(Collectors.toSet());
+ commitMessage.compactIncrement().compactAfter().stream()
+ .filter(file ->
!compactInputs.contains(dataPathFactory.toPath(file)))
Review Comment:
Updated in `3d1ed204d`.
Abort cleanup now protects all physical paths from `compactBefore`,
including existing sidecars, and deletes only output paths outside that set.
The regression now covers a reused main file with a newly created lookup
SST, for both internal and external paths, including repeated abort. Both cases
failed on the previous head and pass with this fix.
All 117 related tests passed locally on both JDK 8 and JDK 17, with
Checkstyle and Spotless enabled.
Please take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
