Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19788 I think we can do this better, to make it a purely server-side optimization. The shuffle protocol can already fetch multiple blocks in one request, i.e. the `OpenBlocks` request. The `OneForOneBlockFetcher`(client side) sends `OpenBlocks` request to `NettyBlockRpcServer`(serve side), and use a callback to process the fetched data. The server side can detect if the requested blocks are continuous shuffle blocks, and then optimize it into one IO request. Then this is totally compatible for the shuffle service.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org