SEZ9 commented on issue #13321:
URL: https://github.com/apache/seatunnel/issues/13321#issuecomment-6995551950

   Thanks for the report and for the method-level reproduction. Together they 
show that `getLatestCheckpointStateByType` runs `deserializeCheckpoint` on 
every stored candidate, and that a truncated newer candidate makes it throw a 
`RuntimeException` instead of selecting the healthy older one.
   
   A PR is welcome. The approach already proposed in this thread sounds good: 
handle deserialization failures per candidate, try older candidates, keep 
storage access failures visible, and fail only when no healthy candidate 
remains.
   
   A few requests for the PR:
   - Log a WARN for each skipped candidate with the job ID, pipeline ID and 
checkpoint ID, and state explicitly that the job is restoring from an older 
checkpoint, since more data may be reprocessed.
   - Include the two tests mentioned above: a corrupted checkpoint with a valid 
older one, and the case where no valid candidate remains.
   - As the reproduction notes, it is an in-process fault injection and does 
not establish the safety or semantics of a fallback policy. Please describe the 
intended fallback behavior in the PR so we can discuss it in review.
   
   <!-- streview-comment:2450 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to