nsivabalan commented on issue #17907: URL: https://github.com/apache/hudi/issues/17907#issuecomment-3955175906
Can we do the following: - While performing conflict resolution for clustering commit a. if a ingestion commit is detected in requested state, fail the clustering commit. b. if a ingestion commit is detected in inflight state, check for overlap with clustering commit. If overlap is found, abort the clustering commit. else proceed. the additional wait time may not buy us much and could only reduce the probability of clustering failing. I would recommend to leverage early conflict resolution to save compute costs on clustering. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
