nsivabalan commented on PR #20429: URL: https://github.com/apache/hudi/pull/20429#issuecomment-6450913281
Moving this back to draft. The run-long lock in `AsyncCompactService`/`AsyncClusteringService` (merged with #20415) makes `updateWriteClient` wait for a running compaction/clustering, and the caller is the writer's driver thread — HoodieStreamer on write-client re-creation, and the streaming sink inside `addBatch`. A compaction can run for an hour (e.g. waiting on cluster resources), and we cannot block ingestion that long. #20444 restores the non-blocking switch. This PR will be reworked on top of it: on a client swap the previous client is closed, the in-flight run is allowed to fail, and that failure is treated as retryable (the instant is rolled back and re-run with the new client); any other failure is reported to the writer with its cause. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
