FrankChen021 commented on code in PR #20740:
URL: https://github.com/apache/druid/pull/20740#discussion_r4388804430
##########
indexing-service/src/main/java/org/apache/druid/indexing/overlord/hrtr/HttpRemoteTaskRunner.java:
##########
@@ -1255,6 +1256,14 @@ public void shutdown(String taskId, String reason)
if (workerHolderRunningTask == null) {
log.info("Can't shutdown! No worker running task[%s]", taskId);
}
+ } else if (taskRunnerWorkItem.getState() ==
HttpRemoteTaskRunnerWorkItem.State.PENDING) {
+ // Removing it here stops the pending task executor from picking it
up before it is completed below.
+ pendingTaskIds.remove(taskId);
+ pendingWorkItem = taskRunnerWorkItem;
+ } else if (taskRunnerWorkItem.getState() ==
HttpRemoteTaskRunnerWorkItem.State.PENDING_WORKER_ASSIGN) {
+ // The thread assigning this task owns the work item. Once the
worker runs the task, or the item goes back
+ // to PENDING, a later shutdown request handles it.
+ pendingTaskIds.remove(taskId);
Review Comment:
[P2] Preserve cancellation when the assignment fails
**Finding:** When shutdown() removes a task ID while the work item is
PENDING_WORKER_ASSIGN, the assigning thread can subsequently see
runTaskOnWorker() return false and revert the item to PENDING. Because this
branch already removed the ID and does not record cancellation or complete the
result, no pending executor will revisit the item; the killed task remains
reported as pending and its future and cleanup are delayed until a later
shutdown or storage sync.
**Suggestion:** Propagate an explicit shutdown marker to the assigning
thread so a failed assignment completes the work item without re-adding it, and
add coverage for the disabled or removed-worker race.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
