This is an automated email from the ASF dual-hosted git repository.
ethanfeng pushed a commit to branch branch-0.5
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/branch-0.5 by this push:
new ac82c6157 [CELEBORN-1587] Change to debug logging on client side for
SortBasedPusher trigger push
ac82c6157 is described below
commit ac82c61571a4aede1949b6b35371e3ab2293a092
Author: Sanskar Modi <[email protected]>
AuthorDate: Fri Sep 6 15:08:14 2024 +0800
[CELEBORN-1587] Change to debug logging on client side for SortBasedPusher
trigger push
### What changes were proposed in this pull request?
Changed a noisy log in SortBasedPusher to debug.
### Why are the changes needed?
Current SortBasedPusher logging is too noisy. It keeps on printing below
log many times.
```
SortBasedPusher: Memory used 72.0 MiB exceeds threshold 64.0 MiB, need to
trigger push. currentPage size: 64.0 MiB
```
### Does this PR introduce _any_ user-facing change?
NA
### How was this patch tested?
NA
Closes #2728 from s0nskar/debug_log_sort_pusher.
Authored-by: Sanskar Modi <[email protected]>
Signed-off-by: mingji <[email protected]>
(cherry picked from commit 9ec345bba70c83295d790a90d56d7b5e0eda46af)
Signed-off-by: mingji <[email protected]>
---
.../main/java/org/apache/spark/shuffle/celeborn/SortBasedPusher.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/client-spark/common/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedPusher.java
b/client-spark/common/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedPusher.java
index 8cd2a4874..dd272d6c2 100644
---
a/client-spark/common/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedPusher.java
+++
b/client-spark/common/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedPusher.java
@@ -292,7 +292,7 @@ public class SortBasedPusher extends MemoryConsumer {
long threshold = pushSortMemoryThreshold;
if (getUsed() > threshold
&& pageCursor + required > currentPage.getBaseOffset() +
currentPage.size()) {
- logger.info(
+ logger.debug(
"Memory used {} exceeds threshold {}, need to trigger push.
currentPage size: {}",
Utils.bytesToString(getUsed()),
Utils.bytesToString(pushSortMemoryThreshold),
