Jackie-Jiang opened a new pull request, #19265:
URL: https://github.com/apache/pinot/pull/19265
## Summary
`waitForMinionTaskCompletion` waited only for the task-level state to reach
`COMPLETED`, then immediately asserted that every subtask state is `COMPLETED`.
The two reads come from different Helix znodes (the task state from the
WorkflowContext, the subtask map from the JobContext) that are not updated
atomically, so the assertion can read a stale subtask state right after the
task turns `COMPLETED` and fail spuriously:
```
Not all subtasks are completed for task
Task_SegmentPurgeTask_..._1786749842426 :
{..._0=COMPLETED, ..._1=RUNNING} expected [true] but found [false]
```
Since Helix only marks a job `COMPLETED` after every partition reaches a
terminal state, a `RUNNING` (or `null`, not yet started — `getSubtaskStates`
can genuinely return null values) subtask under a `COMPLETED` task is a stale
read, not a still-running subtask.
The wait condition now also requires a non-empty subtask map with every
subtask in a terminal state, converting the stale read into a slightly longer
wait. The final assertion (all subtasks `COMPLETED`) is unchanged, so a job
that completes with genuinely failed subtasks (failure-threshold semantics —
the case the assertion exists to catch) still fails immediately with the
informative state map instead of burning the timeout.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
