saurabhd336 commented on PR #3644: URL: https://github.com/apache/celeborn/pull/3644#issuecomment-4249903351
@RexXiong I agree https://github.com/apache/celeborn/pull/3653 helps with the disk limit breach problems. And yes there are overheads associated with this accurate disk usage accounting (CAS contentions across eventLoop / flush / sorter threads). For that reason I've put all of this behind a feature flag, disabled by default. We plan to enable this only for some very disk usage sensitive cluster (tier1 clusters) But happy to take feedback from others if they see value in having this accurate accounting. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
