fhan688 commented on code in PR #19960:
URL: https://github.com/apache/hudi/pull/19960#discussion_r4056559690
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/common/AbstractStreamWriteFunction.java:
##########
@@ -301,7 +302,8 @@ public void handleOperatorEvent(OperatorEvent event) {
* @return The instant time
*/
protected String instantToWrite(boolean hasData) {
- return
Preconditions.checkNotNull(this.correspondent.requestInstantTime(this.checkpointId),
+ return Preconditions.checkNotNull(
Review Comment:
@danny0405 Thanks！ I’ve updated the implementation following your suggestion:
- Moved the bounded retry/polling logic to the writer side.
- Kept the coordinator path minimal: it returns READY when the
checkpoint-to-instant mapping exists; otherwise, it submits at most one
asynchronous creation task and returns PENDING.
- Removed the epoch fencing, InstantOp, and FAILED response state.
- Instant creation failures now use the coordinator’s existing
asynchronous failJob path.
- Recovery clears the submission marker so creation can be retried, with
buffer checks preventing duplicate creation.
We still install the event buffer before returning READY, which prevents
early write events from being dropped.
I also added coverage for blocked creation, concurrent writers,
deduplication, and resubmission after recovery. The related Flink 1.20 test
suite passes with 113 tests.
Could you please take another look and confirm whether this now matches
the direction you planned?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
