rangareddy commented on issue #10812: URL: https://github.com/apache/hudi/issues/10812#issuecomment-4850553867
This is expected behavior, not a bug — `delete_partition` and the cleaner are two separate steps. `delete_partition` only writes a **replacecommit** marking the partition's file groups as *replaced* (a logical delete); no files are removed yet. The **cleaner** does the physical deletion, and the timing depends on the policy. `KEEP_LATEST_FILE_VERSIONS` retains the latest N versions of *live* file groups — a replaced group isn't a live version of anything, so it's removed immediately in the clean cycle triggered by the delete_partition commit, regardless of `CLEANER_FILE_VERSIONS_RETAINED` (that count only applies to live groups). `KEEP_LATEST_COMMITS` instead keeps files needed for the last N commits (for rollback/time-travel/incremental), so the replaced files are retained until the replacecommit ages out of that window — which is why you needed an extra commit to trigger the eviction. To your question: there's no per-file `to_be_deleted` flag; the replaced file groups are recorded in the replacecommit metadata and the cleaner reads the timeline to find and remove them per policy. If you want immediate reclamation, `FILE_VERSIONS` is the right choice, just note it's effectively irreversible (no time-travel/rollback to before the delete); use `KEEP_LATEST_COMMITS` or a savepoint if you need recoverability. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
