marin-ma opened a new pull request, #12918:
URL: https://github.com/apache/gluten/pull/12918
Currently the async shuffle reader threads fetch and deserialize shuffle
data in the order the readers are created in Spark. In a Spark stage this
creation order is usually different from the order the Velox pipeline actually
reads them: Velox runs the build side of each hash join before the probe side,
so build inputs are read first and the main probe input last.
This PR computes the read order from the plan and passes it to the native
reader as the prefetch priority.
In theory matching the two orders should help, but no significant perf
improvement on TPC-DS q95 so far.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
