joseluisll opened a new pull request, #8815:
URL: https://github.com/apache/hadoop/pull/8815
### Description of PR
`CapacityOverTimePolicy#validate` rejects reservations that are well within
the user's average and instantaneous limits. There are two causes:
1. **Integral overflow when an allocation is open at the end of the
window.** The user's consumption is read with `getRangeOverlapping(checkStart,
checkEnd)`. This keeps the start of an allocation that is still in progress at
`checkEnd`, but not its end. The merged view went up to `Long.MAX_VALUE`, so
that allocation stayed open until `Long.MAX_VALUE` and the integral computation
overflowed. This happens when the user has an allocation in progress at the end
of the checked window (reservation end + validity window), or a periodic
allocation that crosses the period boundary.
Fix: end the merge at `checkEnd + validWindow` instead of
`Long.MAX_VALUE`. Only the integral over `[checkStart, checkEnd]` is checked,
and it depends only on points up to the time it is evaluated at. So the check
is the same as before in that range, without the overflow. Ending the merge at
`checkEnd` would not work: it adds a point inside the checked range, where the
approximated integral overestimates the window and could still reject
reservations within the quota. A code comment explains this.
2. **Old version counted twice on update.** `InMemoryPlan#updateReservation`
validates the new version while the old one is still in the plan, so the user's
consumption already includes it. `validate` then merged the old version in
again with `RLEOperator.add` instead of removing it, and updates well within
the average limit were rejected. Fix: use `RLEOperator.subtract`. This matches
the existing comment above that code and what `availableResources()` already
does.
### How was this patch tested?
Three new tests in `TestCapacityOverTimePolicy`, each of which fails without
the fix:
- `testUserAllocationInProgressAtEndOfCheckedWindow`: a reservation within
the quota is accepted while the user has an allocation in progress past the
checked window.
- `testPeriodicAllocationCrossingPeriodBoundary`: same, with a periodic
allocation that crosses the period boundary.
- `testUpdateReservationWithinQuota`: an update within the quota is
accepted, and an update over the limit is still rejected with
`PlanningQuotaException`.
The existing `TestCapacityOverTimePolicy` and reservation tests pass.
### For code changes:
- [x] Does the title of this PR start with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: Have the integration tests been executed and the
endpoint declared according to the connector-specific documentation? *(N/A)*
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under ASF 2.0? *(N/A, no new
dependencies)*
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files? *(N/A)*
### AI Tooling
- [x] The PR includes the phrase "Contains content generated by Claude Code"
- [x] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
Contains content generated by Claude Code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
