luoyuan3471 opened a new pull request, #8905: URL: https://github.com/apache/hadoop/pull/8905
## Problem - `CapacitySchedulerPreemptionUtils.getPartitionByNodeId()` calls `getPartition()` without checking whether `getSchedulerNode()` returns null. - `updateNodeResource()` temporarily removes a scheduler node from the node tracker before adding it back. Concurrent preemption candidate selection can query the node in this interval and throw `NullPointerException`, aborting the current preemption check. ## Solution - Look up the scheduler node once and return null when it is unavailable. - Return false from `tryPreemptContainerAndDeductResToObtain()` before changing resource demand, the preemption budget, or candidate maps when the node cannot be resolved. Preserve normal default-partition and labeled-partition selection. - Add tests for removed nodes, the concurrent node-resource-update window, subsequent valid candidates, and existing nodes in both partition types. Leave event dispatch and reservation handling unchanged. ## Validation - Local environment: macOS, JDK 17.0.8, Maven Wrapper 3.9.15; no native profile. - On unpatched trunk `e66f40daf3cc4ebbb5164cec1ce54b71eddec8a4`, `TestCapacitySchedulerPreemptionUtils` runs 5 tests: 2 pass and 3 raise `NullPointerException` at `getPartitionByNodeId()`. - `testSkipContainerDuringNodeResourceUpdate` deterministically reproduces the error using two threads and the real `updateNodeResource()` and node tracker. A spied scheduler node is used only to pause the update between removal and re-registration; this is a unit-level concurrency reproduction, not a full RM integration test. - With the fix, `TestCapacitySchedulerPreemptionUtils`: 5 tests passed. - `TestProportionalCapacityPreemptionPolicy`: 33 tests passed. - `TestProportionalCapacityPreemptionPolicyForNodePartitions`: 11 tests passed. - `TestProportionalCapacityPreemptionPolicyForReservedContainers`: 7 tests passed. - `TestProportionalCapacityPreemptionPolicyInterQueueWithDRF`: 4 tests passed. - Total with the fix: 60 tests, 0 failures, 0 errors, 0 skipped. - `git diff --check` passed. ## Related Jira [YARN-12054](https://issues.apache.org/jira/browse/YARN-12054) Contains content generated by Codex. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
