viirya commented on code in PR #828:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/828#discussion_r4021047657
##########
spark-operator/src/main/java/org/apache/spark/k8s/operator/reconciler/reconcilesteps/AppInitStep.java:
##########
@@ -67,6 +67,10 @@ public ReconcileProgress reconcile(
return proceed();
}
SparkApplication app = context.getResource();
+ if (app.getSpec().isSuspend()) {
Review Comment:
This handles a previous attempt's pod once the informer has observed its
deletion, but the stale pre-deletion snapshot can still pass this filter.
The remaining sequence is:
1. Consecutive attempts reuse the driver pod name.
2. Cleanup successfully deletes the old pod and persists
`ScheduledToRestart`.
3. With `suspend: true` and zero or elapsed backoff, the next reconcile
still sees the old informer snapshot: the expected name and a null
`deletionTimestamp`.
4. This helper returns the old pod, so `AppInitStep` bypasses the hold.
5. Initialization queries the API server, finds that the pod is gone, and
creates a new driver despite suspend.
I agree that this is informer lag, but the missing-driver grace period does
not make this path safe. That flow defers a decision and eventually performs
live verification; this path immediately authorizes resource creation. The
grace-period observer also does not run for `ScheduledToRestart`.
Could we perform a live lookup by the desired pod name before allowing the
suspend bypass, require the pod to exist and not be terminating, and requeue if
verification fails?
The regression test should keep the **pre-deletion** pod snapshot in the
informer while the API server reports the pod absent, then verify that a
suspended application creates no driver. The current test sets
`deletionTimestamp` on the cached object, so it covers the already-updated
cache rather than this remaining window.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
