Zerolzj commented on code in PR #10582:
URL: https://github.com/apache/paimon/pull/10582#discussion_r4303805996
##########
paimon-core/src/main/java/org/apache/paimon/operation/FileStoreCommitImpl.java:
##########
@@ -728,8 +729,16 @@ public void abort(List<CommitMessage> commitMessages) {
dataFilesToDelete.addAll(commitMessage.compactIncrement().compactAfter());
dataFilesToDelete.addAll(commitMessage.compactIncrement().changelogFiles());
+ // A level upgrade may reuse a main file while adding new
auxiliary files.
+ Set<Path> compactInputPaths =
+ commitMessage.compactIncrement().compactBefore().stream()
+ .flatMap(file ->
file.collectFiles(dataPathFactory).stream())
+ .collect(Collectors.toSet());
+
for (DataFileMeta file : dataFilesToDelete) {
-
file.collectFiles(dataPathFactory).forEach(fileIO::deleteQuietly);
+ file.collectFiles(dataPathFactory).stream()
+ .filter(path -> !compactInputPaths.contains(path))
Review Comment:
Updated in `bdf1bce66`.
Abort now cleans newly written files and changelog files independently. Only
compaction-output cleanup is filtered against the physical paths in
`compactBefore`, so fresh inputs remain eligible for deletion while committed
reused files and their existing sidecars are preserved.
Added the real TestFileStore writer sequence with two flushes of the same
key, full compaction, prepare and repeated abort, for NONE, INPUT and
FULL_COMPACTION changelog modes. All three cases fail on the previous head
because the inputs survive, and pass with the fix. The assertions run before
writer close and verify that no snapshot was committed. The internal/external
reused-file and new-sidecar regressions remain in place.
All 120 related commit/writer tests passed locally on both JDK 8 and JDK 17,
with Checkstyle and Spotless enabled. The JDK 8 run used a clean build. A
broader JDK 17 reactor test attempt was stopped in the unchanged Common module
after Mockito/Byte Buddy reported `Unsupported class file major version 61`; I
am not claiming a full-suite pass.
Please take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
