github-actions[bot] commented on code in PR #66083:
URL: https://github.com/apache/doris/pull/66083#discussion_r3655038403
##########
fe/fe-core/src/main/java/org/apache/doris/planner/OlapTableSink.java:
##########
@@ -1169,17 +1169,21 @@ public List<TOlapTableLocationParam>
createDummyLocation(OlapTable table) throws
final long fakeTabletId = 0;
SystemInfoService clusterInfo = Env.getCurrentSystemInfo();
- List<Long> aliveBe = clusterInfo.getAllBackendIds(true);
- if (aliveBe.isEmpty()) {
+ List<Long> availableBeIds =
clusterInfo.getBackendsByCurrentCluster().values().stream()
+ .filter(Backend::isLoadAvailable)
+ .filter(backend -> !backend.isDecommissioned() &&
!backend.isDecommissioning())
+ .map(Backend::getId)
+ .collect(Collectors.toList());
+ if (availableBeIds.isEmpty()) {
throw new UserException(InternalErrorCode.REPLICA_FEW_ERR, "no
available BE in cluster");
}
for (int i = 0; i < table.getIndexNumber(); i++) {
// only one fake tablet here
- Long[] nodes = aliveBe.toArray(new Long[0]);
+ Long[] nodes = availableBeIds.toArray(new Long[0]);
Random random = new SecureRandom();
int nodeIndex = random.nextInt(nodes.length);
if (singleReplicaLoad) {
- List<Long> slaveBe = aliveBe;
+ List<Long> slaveBe = availableBeIds;
Review Comment:
[P1] Keep backend candidates intact across index iterations
`slaveBe` aliases `availableBeIds`, so `remove(nodeIndex)` permanently
shrinks the source used by the next loop iteration. With single-replica load
enabled, an empty automatic-partition table with two indexes and one eligible
BE in the current compute group handles index 0, then builds an empty `nodes`
array and throws from `nextInt(0)` for index 1. The current-group filter makes
this reachable even when other alive BEs exist outside the group. Keep the
removal local to each index and add a multi-index/single-replica test:
```suggestion
List<Long> slaveBe = new ArrayList<>(availableBeIds);
```
##########
fe/fe-core/src/main/java/org/apache/doris/planner/OlapTableSink.java:
##########
@@ -1169,17 +1169,21 @@ public List<TOlapTableLocationParam>
createDummyLocation(OlapTable table) throws
final long fakeTabletId = 0;
SystemInfoService clusterInfo = Env.getCurrentSystemInfo();
- List<Long> aliveBe = clusterInfo.getAllBackendIds(true);
- if (aliveBe.isEmpty()) {
+ List<Long> availableBeIds =
clusterInfo.getBackendsByCurrentCluster().values().stream()
Review Comment:
[P1] Avoid auto-start waits while the table read lock is held
`getBackendsByCurrentCluster()` is not just a snapshot lookup in cloud mode:
it calls `waitForAutoStart()`, which can issue a meta-service RPC and then
retry for roughly five minutes. Both `StreamLoadHandler.generatePlan` and
`RoutineLoadJob.plan` hold the destination table read lock across this sink
initialization. If the group transitions out of `NORMAL`, planning can
therefore retain the metadata lock throughout the external wait and block
schema changes or other table writers; the old `getAllBackendIds(true)` call
could not do that. Resolve/wake the compute group before taking the table lock,
or keep sink initialization on a nonblocking already-resolved backend snapshot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
