(doris) branch master updated: [improve](routine load) ensure load balance after scaling up BE nodes (#44693)

[dataroaring]

This is an automated email from the ASF dual-hosted git repository.

dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
     new fed46d2d846 [improve](routine load) ensure load balance after scaling 
up BE nodes (#44693)
fed46d2d846 is described below

commit fed46d2d8467148c87e88ca8bec69280125dd8e3
Author: hui lai <lai...@selectdb.com>
AuthorDate: Fri Nov 29 20:43:43 2024 +0800

    [improve](routine load) ensure load balance after scaling up BE nodes 
(#44693)
    
    We scaling up from three BE nodes to five BE nodes, but from the
    monitoring perspective, only the previous three nodes have written
    traffic.
    
    
![image](https://github.com/user-attachments/assets/947fe8a9-745d-4c37-93d4-15b0e27f12fc)
    
    This pr aims to ensure load balance after scaling up BE nodes.
---
 .../main/java/org/apache/doris/load/routineload/RoutineLoadManager.java  | 1 +
 1 file changed, 1 insertion(+)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java
 
b/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java
index 8b0ebab3fbe..97f6aba8c58 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java
@@ -487,6 +487,7 @@ public class RoutineLoadManager implements Writable {
         // check if be has idle slot
         readLock();
         try {
+            updateBeIdToMaxConcurrentTasks();
             Map<Long, Integer> beIdToConcurrentTasks = 
getBeCurrentTasksNumMap();
             int previousBeIdleTaskNum = 0;
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org