leixm commented on issue #134: URL: https://github.com/apache/incubator-uniffle/issues/134#issuecomment-1227470284
Currently we can support 2^20 tasks, which is not a small number. If spark.rss.writer.buffer.size is set to the default value of 3m, then the data written by a taskAttempt does not exceed 3m*2^19=1536G It can be guaranteed, how to measure whether AtomicInteger can be reduced? How much can be reduced? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
