advancedxy commented on issue #339: URL: https://github.com/apache/incubator-uniffle/issues/339#issuecomment-1328014871
> > the client could send the same data to different shuffle server concurrently? > > It has done this. you can see > > https://github.com/apache/incubator-uniffle/blob/57a834b083d9dae291091e4f2359d060ba30abe0/client/src/main/java/org/apache/uniffle/client/impl/ShuffleWriteClientImpl.java#L240-L258 Thanks for pointing out that. I did a quick look at the shuffle write client code, it seems that the `sendShuffleData` process is blocked by the first round to the primary shuffle servers. I think rather to reduce the `rss.client.send.check.timeout.ms` settings(-> which causes task retry), how about refactor `sendShuffleDataAsync` to return early and the send round of `sendShuffleDataAsync` could start asap. The purpose of sending data concurrently is that once a request finished, the whole sending process would finish, which avoids busy/flaky servers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
