danny0405 opened a new pull request, #20677:
URL: https://github.com/apache/hudi/pull/20677

   ### Describe the issue this Pull Request addresses
   
   Issue: #20406; parent #20401. A Spark streaming write can commit its data 
and then fail during post-commit work such as metastore synchronization. The 
sink's retry loop currently retries the whole writer, potentially committing 
the same batch again and duplicating insert rows. #20495 covers failures 
outside this retry loop; this PR covers the inner window.
   
   ### Summary and Changelog
   
   - Reload durable batch metadata after a failed writer attempt, including 
when the first batch initialized the table before throwing.
   - Refuse to retry a committed batch. Fail the affected query with the 
original cause so post-commit failures remain visible; restart with the same 
checkpoint deduplicates the committed batch.
   - Stop on an unreadable commit outcome rather than retrying an uncertain 
write. Preserve normal pre-commit retries and the explicit ignore-failed-batch 
opt-out.
   - Add a real failing metadata-sync tool regression for new/existing COW and 
MOR targets. Assert one sync/write attempt, exact rows, no Spark checkpoint 
acknowledgment on failure, and no new Hudi commit after restart.
   - Document repair and restart behavior, including that checkpoint replay 
does not rerun a failed metastore sync. No external code was copied.
   
   ### Impact
   
   Changes streaming-sink retry handling only after a failed write attempt. No 
new config, public API or storage format. Successful writes incur no additional 
timeline lookup. Operators still receive the post-commit failure and must 
repair its external effects before restarting. Fan-in rejection and the 
async-service client-swap work in #20445 remain separate.
   
   ### Risk Level
   
   medium. Failure handling controls commit deduplication. JDK 17 / Spark 4.1 / 
Scala 2.13 compilation, style and license checks passed. All 13 selected 
regression cases passed: 10 in TestStreamingSinkRecovery (including four new 
post-commit cases) and three existing TestStructuredStreaming cases for retry 
exhaustion/query isolation and ignore-failed-batch behavior. git diff --check 
passed. Cleaning/archival-specific injection and an unreadable-timeline fault 
are not separately tested; the real metadata-sync failure establishes the 
post-commit boundary.
   
   ### Documentation Update
   
   Added hudi-integ-test/spark-pipelines/SINK_RECOVERY.md with the failure 
contract, manual repair of post-commit operations and same-checkpoint restart 
instructions. Website publication of this operational note remains follow-up 
before release; this PR does not claim it is published.
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to