VuMartin opened a new pull request, #7122: URL: https://github.com/apache/texera/pull/7122
### What changes were proposed in this PR? This PR fixes a region termination hang caused by unresponsive `EndWorker` RPC calls. Previously, `Future.collect(endWorkerRequests)` could wait forever if a worker never responded. This prevented the termination retry mechanism from running because the failure was never propagated, causing the workflow to silently stall. This change adds timeout handling around the termination futures so stuck `EndWorker` and `gracefulStop` operations fail and trigger the existing retry budget. Cleanup of actor references and control channels is delayed until graceful termination succeeds, preventing retries from using stale worker state. **Before:** gracefulStop timeout → silent hang forever. <img width="1327" height="619" alt="Screenshot 2026-07-29 at 1 26 05 AM" src="https://github.com/user-attachments/assets/e5ca9a61-057a-4533-a79f-94ce83fedd2f"; /> **After:** gracefulStop timeout → retries 150 times → error. <img width="1338" height="359" alt="Screenshot 2026-07-29 at 11 21 16 PM" src="https://github.com/user-attachments/assets/1d299430-fdd9-42bd-8268-3248f7fcca25"; /> ### Any related issues, documentation, discussions? Fixes #6920 ### How was this PR tested? Manual: Simulated slow worker shutdown (> 5s delay in postStop). Confirmed retry loop executes all 150 attempts and error fires with stuck workers listed. No silent hang. Ran the Amber test suite locally. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: ChatGPT (5.5-mini) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
