lw309637554 commented on pull request #1756: URL: https://github.com/apache/hudi/pull/1756#issuecomment-652546754
> > for rollback successful commit, in HoodieWriteClient.java i remove the deleteMarkerDir() in postcommit when is in usingmarkers mode. But it will double the file numbers in dfs. > > I think delaying marker deletion till cleaning is probably ok. but the reconcilation with data files i.e the deletion of extraeneous data files written due to spark stage retries must be handled pre-commit.. > > > if the markers file retain, if we should clean it when the datafile is cleaned, also if we should archive the markers file when archiveCommitsWith > > there is no need to archive teh marker files in my opinion.. the contract in Hudi is that once an instant leaves the active timeline, its effects are permanent on the table ... so if a rollback needs to happen based on marker files, then it needs to be within the retained commits for active timeline.. I think this is a practical approach.. > > think of active timeline as the transaction log with pending actions/inflight/completed actions.. thanks, agree with you . I will update the PR. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
