SEZ9 commented on PR #12678:
URL: https://github.com/apache/seatunnel/pull/12678#issuecomment-6937367030

   @kalayciburak thanks for the direct answers on both points.
   
   **Delete failure during retention pruning**
   
   - The description saying a reported delete failure doesn't fail the 
checkpoint, plus 
`CheckpointCoordinatorTest#testRetentionDeleteFailureRetriesOnNextCompletion` 
with mocked checked and unchecked delete failures, covers the coordinator-side 
requeue and retry.
   - As you note, `notifyCompleted` is stubbed there and 
`CheckpointStorageRetentionPruneTest` only covers successful pruning, so no 
test exercises a real storage failure end to end yet. Please push the 
storage-backed failure/retry test you mentioned. It should show that the 
checkpoint still completes and that the id whose delete failed is removed on a 
later completion rather than left behind.
   - One part of the original point is still unanswered: pruning is exercised 
on every completion instead of once per `max-retained`. Is that still the case 
at `71ac656652f0`? If so, please confirm that an id whose delete keeps failing 
stays queued for retry and is never dropped.
   
   **Bound after a restart**
   
   - Thanks for confirming the deque still starts empty after a coordinator 
restart, so the bound only holds for a fresh job.
   - I can't find an earlier agreement in this thread to keep that out of 
scope, so I can't confirm it as an existing plan. Could you link the comment 
you mean?
   - I'm fine with deferring it to the follow-up you linked, on two conditions:
     - The documented limitation states the concrete effect: after a pipeline 
restart, storage can hold up to 2x `max-retained`, and the pre-restart files 
are never pruned.
     - The follow-up explicitly covers seeding `completedCheckpointIds` on 
restore.
   - Please also point me to where the limitation is documented, since the text 
isn't quoted in the thread.
   
   **Verification**
   
   Your local results at `71ac656652f0` (18 and 1 passing) are noted, but as 
you say they aren't an exact-head Build. Once the storage-backed test is on the 
branch, please link a Build run for that head.
   
   <!-- streview-comment:2401 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to