jerqi commented on issue #372: URL: https://github.com/apache/incubator-uniffle/issues/372#issuecomment-1331705508
> Feel free to discuss more. Now we also suffer from the problem of single partition data too large and need this to make job fast fail to improve stability. For huge shuffle, we really recommend to use MEMORY_LOCALFILE_HDFS. You can use a large value `rss.server.flush.cold.storage.threshold.size` to reduce the data size of shuffle which were written to HDFS. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
