Stephen0421 commented on code in PR #10114:
URL: https://github.com/apache/paimon/pull/10114#discussion_r4256078369
##########
paimon-python/pypaimon/write/file_store_commit.py:
##########
@@ -1023,22 +1039,36 @@ def abort(self, commit_messages: List[CommitMessage]):
paths must preserve prepared files instead of calling this method.
"""
for message in commit_messages:
+ preserve_blob_files = message.preserve_blob_files_on_abort
for file in (list(message.new_files) +
list(message.changelog_files)
+ list(message.compact_after)
+ list(message.compact_changelog_files)):
+ paths = []
path = None
+ data_path = None
try:
if not file.physical_path():
file.file_path = canonical_data_file_path(
self.table, tuple(message.partition),
message.bucket, file.file_name)
+ data_path = file.physical_path()
for path in file.collect_files():
- self.table.file_io.delete_quietly(path)
+ if preserve_blob_files and
_is_preserved_blob_pack(path):
+ continue
Review Comment:
Explicit `commit.abort(messages)` now deletes every newly written file in
the messages, including `.blob` packs, matching `FileStoreCommitImpl.abort`.
Writer `abort()` still leaves a pack in place when a `BlobConsumer` has already
been given it.
`preserve_blob_files_on_abort` is gone, and those messages are no longer
excluded from native abort. Cleanup no longer depends on a Python-only flag
that a Java v14 round trip drops.
`test_flush_only_consumer_commit_abort_deletes_new_files` covers a
flush-only consumer through `prepare_commit()` and then
`commit.abort(messages)`, and checks that the new data and BLOB files are gone.
Writer abort still keeps descriptors readable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
