xumanbu commented on issue #695: URL: https://github.com/apache/incubator-uniffle/issues/695#issuecomment-1494511043
> > We could use more MEMORY_LOCALFILE_HDFS, HDFS have more capacity to process the problems. > > Yes, I think so. But users like VIP shop hope this optimization could be applied in local disk. Could you help ping? Let's discuss more. use hdfs can solve capacity issues, but hdfs write rate is slower than ssd, the memory will be full quickly in large data written. in my mind if we use hdfs storage,uniffle client should have backpressure to limit client write rate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
