[incubator-uniffle] branch master updated: [ISSUE-173][FOLLOWUP] The size of single buffer flush should reach rss.server.flush.cold.storage.threshold.size (#178)

[roryqi]

This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-uniffle.git


The following commit(s) were added to refs/heads/master by this push:
     new ab6be6ed [ISSUE-173][FOLLOWUP] The size of single buffer flush should 
reach rss.server.flush.cold.storage.threshold.size  (#178)
ab6be6ed is described below

commit ab6be6ed4ab0d8bfe4bbcf26c6597ded28a0c1aa
Author: Xianming Lei <31424839+le...@users.noreply.github.com>
AuthorDate: Mon Aug 22 22:19:29 2022 +0800

    [ISSUE-173][FOLLOWUP] The size of single buffer flush should reach 
rss.server.flush.cold.storage.threshold.size  (#178)
    
    ###What changes were proposed in this pull request?
    follow  #173, when single buffer flush is triggered, the buffer size should 
reach `rss.server.flush.cold.storage.threshold.size`,  we keep the same logic 
as MultiStorageManager#selectStorageManager
    
    ###Why are the changes needed?
    we keep the same logic as MultiStorageManager#selectStorageManager
    
    ###Does this PR introduce any user-facing change?
    No
    
    ###How was this patch tested?
    Already added
    
    Co-authored-by: leixianming <leixianm...@didiglobal.com>
---
 .../java/org/apache/uniffle/server/buffer/ShuffleBufferManager.java   | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git 
a/server/src/main/java/org/apache/uniffle/server/buffer/ShuffleBufferManager.java
 
b/server/src/main/java/org/apache/uniffle/server/buffer/ShuffleBufferManager.java
index f934f0f6..660cd5c5 100644
--- 
a/server/src/main/java/org/apache/uniffle/server/buffer/ShuffleBufferManager.java
+++ 
b/server/src/main/java/org/apache/uniffle/server/buffer/ShuffleBufferManager.java
@@ -165,7 +165,9 @@ public class ShuffleBufferManager {
 
   void flushSingleBufferIfNecessary(ShuffleBuffer buffer, String appId,
       int shuffleId, int startPartition, int endPartition) {
-    if (this.bufferFlushEnabled && buffer.getSize() >= 
this.bufferFlushThreshold) {
+    // When we use multistorage and trigger single buffer flush, the buffer 
size should be bigger
+    // than rss.server.flush.cold.storage.threshold.size, otherwise cold 
storage will be useless.
+    if (this.bufferFlushEnabled && buffer.getSize() > 
this.bufferFlushThreshold) {
       flushBuffer(buffer, appId, shuffleId, startPartition, endPartition);
     }
   }