zuston commented on issue #398: URL: https://github.com/apache/incubator-uniffle/issues/398#issuecomment-1346130844
If the single buffer flush mechanism is enabled, it will be applied all partition data. But when the memory is free and the concurrent apps number is small, there is unnecessary to flush data to localfile or disk. In my view, it will make the local disk free if the single buffer flush size > cold storage threshold size. And it will make memory free if the single buffer flush size < cold storage threshold size. Overall, this mechanism will not make full use of memory and then leads to performance regression. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
