[
https://issues.apache.org/jira/browse/SPARK-58725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-58725:
-----------------------------------
Labels: pull-request-available (was: )
> spark-submit --kill reports nothing and exits 0 when the driver pod does not
> exist
> ----------------------------------------------------------------------------------
>
> Key: SPARK-58725
> URL: https://issues.apache.org/jira/browse/SPARK-58725
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 5.0.0
> Reporter: Yang Jie
> Priority: Major
> Labels: pull-request-available
>
> `KillApplication.executeOnPod` null-checks the request handle returned by
> `getPod` instead of the pod it resolves to:
> {code}
> val podToDelete = getPod(namespace, pName)
> if (Option(podToDelete).isDefined) {
> ... .delete()
> } else {
> printMessage("Application not found.")
> }
> {code}
> `getPod` returns `client.pods.inNamespace(ns).withName(name)`, a fabric8
> `PodResource` request handle. `withName` never returns null: it either throws
> `IllegalArgumentException` for a null or empty name, or constructs a new
> handle. So `Option(podToDelete).isDefined` is a constant `true`, and the
> `printMessage("Application not found.")` branch is unreachable.
> `PodResource.delete()` on a name the API server does not have issues a real
> DELETE, catches the resulting 404 inside `BaseOperation.deleteAll()`, and
> returns an empty list. The caller discards that list.
> Net effect: `spark-submit --kill <namespace>:<wrong-pod-name> --master
> k8s://...` prints only the "Submitting a request to kill submission ..."
> banner, prints no "Application not found.", and exits 0, which is
> byte-for-byte what a successful kill prints. `--status` on the same
> submission ID correctly reports "Application not found.", because
> `ListStatus.executeOnPod` resolves the handle with `.get()` before the null
> check. The two subcommands in the same file disagree about the same
> nonexistent pod.
> The check has been unreachable since the feature was added in SPARK-24248
> (2019); fabric8 4.x had the same never-null `withName` and also swallowed the
> 404, returning `Boolean.FALSE` rather than an empty list.
> Scope: the silence is specific to the API server answering 404. A 403 or 5xx
> propagates a `KubernetesClientException` and a nonzero exit, so this is a
> wrong-name or wrong-namespace defect rather than "kill never reports
> failure". Note that standalone mode does report a nonexistent driver ("Driver
> ... has already finished or does not exist"), so reporting is the convention
> this path is missing.
> Fix: resolve the handle with `get()` before the null check, mirroring
> `ListStatus.executeOnPod`, and reuse the same handle for the delete so only
> one extra GET is added.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
