viirya commented on PR #6813: URL: https://github.com/apache/datafusion-comet/pull/6813#issuecomment-6101277938
Rebased onto #6310 and addressed the review: - `MissingTaskEntryRetry` now lives under `org.apache.comet`, which fixes the `check-jar-contents` failure. It takes an optional task id, so `CometTaskMemoryManager` keeps #6310's exact match. - `CometTaskMemoryManagerSuite` keeps one race test plus the refusal and rethrow tests. The waiting-request harness now lives in `TaskMemoryTestUtils`, shared with the shuffle suite. - `memory_management.md` has a single paragraph for both call sites. The stale #6304 sentence and "Only the shuffle allocator's callers are guarded" are gone. One behavior change was added, deferred from the #6310 review: the retry no longer runs once the task has completed or been killed. A test reproduces the issue. Releasing the task's last bytes after the task ended made the retried acquire register the task again and wait in Spark while holding the `TaskMemoryManager` monitor. It is in a separate commit, and the title and description now cover it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
