Re: Q. How to interrupt ShuffleClient and avoid revive requests due to HARD_SPLIT

2023-07-31 Thread Keyong Zhou
To be more detailed, ShuffleClient.pushData is a synchronous API, data will be sent to wire before return, but callbacks are asynchronously handled in netty's thread pool. If the callback says that the push data should be retried (for example, HARD_SPLIT, or push failed), the callback thread first

Re: Q. How to interrupt ShuffleClient and avoid revive requests due to HARD_SPLIT

2023-07-31 Thread Keyong Zhou
Hi Sungwoo, Thanks for your letter and apologize for the late reply :P For your questions: 1. No we can't, the thread pool is stopped by calling ShuffleClient.shutdown(). In addition, The thread pool is shared for all shuffle ids, if you shut it down, the ShuffleClient will not work properly for