satishkotha edited a comment on pull request #3509: URL: https://github.com/apache/hudi/pull/3509#issuecomment-917339044
@zhangyue19921010 Would this solve the problem completely? IIUC, I think there are still possible race conditions between when replacecommit.rejected file is created and when clustering validation check is performed. Anyway, my high level thought is that it is better to find a way to integrate this with Multi-writer conflict resolution mentioned here https://cwiki.apache.org/confluence/display/HUDI/RFC+-+22+%3A+Snapshot+Isolation+using+Optimistic+Concurrency+Control+for+multi-writers#RFC22:SnapshotIsolationusingOptimisticConcurrencyControlformultiwriters-OptimisticConcurrencyusingatomicrenameswithConflictResolution If we integrate with multi-writer, this can be done in a generic for any two operations (instead of adding very specific strategy for clustering.) Some changes may be needed in multi-writer implementation to enforce priority of operations (clustering is lower priority than ingestion). But this is better long term IMO. Happy to discuss more details/other alternatives. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
