PavithranRick opened a new pull request, #18073: URL: https://github.com/apache/hudi/pull/18073
### Describe the issue this Pull Request addresses rollback gets triggered (internally) when clean() is called. Rollbacks are failed due to marker file creation retries. https://github.com/apache/hudi/issues/18072 When using remote marker files, if a request times out but the marker is actually created successfully, the subsequent retry fails because the marker already exists. Retries in this scenario should succeed. ### Summary and Changelog This PR improves retry behavior for remote marker file creation by making marker operations idempotent. - Handles timeout scenarios where the marker was created but the client did not receive an acknowledgement. - Introduces a request ID (or similar tracking mechanism) to safely support idempotent retries without creating additional risks. | Scenario | Marker Exists? | Existing RequestId | Incoming RequestId | Idempotent Match? | Result | Explanation | |----------|---------------|-------------------|-------------------|-------------------|--------|-------------| | **New marker creation** | No | - | `abc123` | N/A | ✅ Success | Marker created and added to map with requestId | | **Idempotent retry (success case)** | Yes | `abc123` | `abc123` | ✅ Yes | ✅ Success | Same marker+requestId treated as success (safe retry) | | **Concurrent duplicate (different requestId)** | Yes | `abc123` | `xyz789` | ❌ No | ❌ Failure | Marker exists but requestId mismatch → not idempotent | | **Recovery scenario (null sentinel)** | Yes | `NULL_REQUEST_ID` (`""`) | `abc123` | ❌ No | ❌ Failure | Marker recovered from filesystem; no requestId stored during recovery | | **Null requestId from client (not possible)** | Yes | `abc123` | `null` | ❌ No | ❌ Failure | Missing requestId cannot match existing requestId | | **Both null requestId (not possible)** | Yes | `NULL_REQUEST_ID` (`""`) | `null` | ❌ No | ❌ Failure | Sentinel value doesn't match null requestId | **Key Behaviors:** 1. **Idempotency condition**: `(!NULL_REQUEST_ID.equals(existingRequestId)) && (requestId != null) && (requestId.equals(existingRequestId))` - All three conditions must be true for idempotent success 2. **NULL_REQUEST_ID sentinel**: Empty string (`""`) used when markers are recovered from filesystem after TLS crash (no request context available) 3. **Deduplication**: `getOrCreateMarkerCreationFuture()` uses `(markerName, requestId)` pair as dedup key to prevent concurrent duplicate processing 4. **Thread-safety**: All operations protected by `synchronized(inflightRequestMap)` and `synchronized(markerCreationProcessingLock)` ### Impact Reduces rollback failures (via cleaner) caused by non-idempotent marker retries and improves robustness of remote marker handling under transient network issues. ### Risk Level low — change is scoped to retry/idempotency behavior for remote marker requests. Verification includes ensuring retries succeed when markers already exist due to previous timed-out requests. ### Documentation Update none ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
