Ted-Jiang opened a new issue #1336: URL: https://github.com/apache/arrow-datafusion/issues/1336
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** In current code, each poll work request will scan all waiting task in the same name space.https://github.com/apache/arrow-datafusion/blob/d7ae8c2631b5ea86ac2328530f0a4745daaf0bda/ballista/rust/scheduler/src/state/mod.rs#L283 `` plan1 -> shuffle wirte -> plan2 `` Seems p1 and p2 run on the same executor may avoid sending intermediate results. **Describe the solution you'd like** 1. executor request job with jobId 2. scheduler send same job task back **Describe alternatives you've considered** -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
