Github user cloud-fan commented on the issue:

    https://github.com/apache/spark/pull/19788
  
    I think we can do this better, to make it a purely server-side 
optimization. The shuffle protocol can already fetch multiple blocks in one 
request, i.e. the `OpenBlocks` request. 
    
    The `OneForOneBlockFetcher`(client side) sends `OpenBlocks` request to 
`NettyBlockRpcServer`(serve side), and use a callback to process the fetched 
data. The server side can detect if the requested blocks are continuous shuffle 
blocks, and then optimize it into one IO request.
    
    Then this is totally compatible for the shuffle service.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to