dongjoon-hyun commented on PR #828: URL: https://github.com/apache/spark-kubernetes-operator/pull/828#issuecomment-5689696036
Thank you, @viirya. You are right that the cached `deletionTimestamp` only covers a cache that has already observed the delete, and that this path authorizes creation immediately rather than deferring. Addressed in 18636ac. **Live verification before the bypass**: `SparkAppContext.getCurrentAttemptDriverPod()` still uses the informer cache to find a candidate (driver labels, the desired driver pod name, no `deletionTimestamp`), but it now verifies that candidate against the API server with a `GET` by the desired pod name and returns it only if the pod exists there and is not terminating. A stale pre-deletion snapshot therefore no longer bypasses the hold: the live lookup finds nothing, `AppInitStep` keeps the hold and requeues at the regular interval. A `KubernetesClientException` during the lookup is logged and treated as absent, so an API error keeps the hold for that reconciliation instead of authorizing creation. When the cache has no candidate, neither the driver spec is built nor the API is called, so a held application still costs nothing per reconcile. **Regression coverage**: - `AppInitStepTest.staleInformerSnapshotDoesNotBypassSuspend` runs the step against a real `SparkAppContext` whose informer stream returns the pre-deletion snapshot (same name, no `deletionTimestamp`) while the mock API server has no such pod, with `suspend: true` and an elapsed backoff in `ScheduledToRestart`. It asserts that no driver pod is created, no status is persisted and the app stays in `ScheduledToRestart`. - `SparkAppContextTest` adds `stalePreDeletionSnapshotIsNotCurrentAttemptDriver`, `podTerminatingOnApiServerIsNotCurrentAttemptDriver` and `apiErrorDuringVerificationIsTreatedAsAbsent`, and checks that no API call is made when the cache has no matching candidate. The javadoc and the PR description describe the verification step. The Java 21 CI failure on the previous run was `SparkOperatorConfigMapReconcilerTest.initializationError` failing to download the kubeapitest binary, unrelated to this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
