FrankChen021 commented on code in PR #20738:
URL: https://github.com/apache/druid/pull/20738#discussion_r4388807733
##########
extensions-core/kubernetes-overlord-extensions/src/main/java/org/apache/druid/k8s/overlord/KubernetesTaskRunner.java:
##########
@@ -270,10 +271,18 @@ protected TaskStatus doTask(Task task, boolean run)
}
peonLifecycle = workItem.getPeonLifeycle();
+ isShutdown = workItem.isShutdown();
}
TaskStatus taskStatus;
- if (run) {
+ if (isShutdown) {
+ // Shut down while waiting for a free thread.
+ if (!run) {
+ // A restored job already has a pod, so persist its logs and close
the log watch opened by the shutdown.
+ peonLifecycle.saveLogs();
+ }
+ taskStatus = TaskStatus.failure(task.getId(), "Task was shut down
before it started");
Review Comment:
[P2] Complete the startup future for canceled restores
**Finding:** When a restored job is queued in joinAsync and shut down before
its worker starts, this branch returns a failure without calling
KubernetesPeonLifecycle.join(), which normally completes
taskStartedSuccessfullyFuture, or otherwise completing that future.
KubernetesTaskRunner.start() coalesces those futures and waits up to
taskJoinTimeout (one minute by default), so killing a queued restored job can
keep startup in the timeout path and report the job as uninitialized.
**Suggestion:** Complete the lifecycle startup future with a failure result
for the shutdown-before-join path, using a small lifecycle helper if needed,
before returning the failure status.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
