jerqi commented on issue #89: URL: https://github.com/apache/incubator-uniffle/issues/89#issuecomment-1198914310
> @smallzhongfeng The workload of Shuffle Server depends on a lot of things, eg, Memory, Disk IO, NetworkIO, etc. To simplify the assignment strategy, memory is chosen as the most important metric, because any problem in shuffle server will cause much memory usage. For your case, if there has problem in Disk IO, data won't be flushed as expected, and more and more data will be stored in memory. Uniffle is kind of producer & consumer model, and memory is the cache, I think we can check the workload according to memory usage and do the assignment. @colinmjj One question? If server A, server B have equal memory, but they have different quantity disks, should we allocate them the same shuffle partitions? The server which has fewer disks will be more slower. Although we don't allocate extra shuffle partitions to it, when it have processed these partitions, the coordinator will allocate excessive partitions to it, it will be slower again. I think it's meaningful to consider disk performance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
