caishunfeng commented on code in PR #12217:
URL: https://github.com/apache/dolphinscheduler/pull/12217#discussion_r984153632
##########
dolphinscheduler-worker/src/test/java/org/apache/dolphinscheduler/server/worker/registry/WorkerRegistryClientTest.java:
##########
@@ -78,28 +79,21 @@ public class WorkerRegistryClientTest {
// workerGroups = Sets.newHashSet(DEFAULT_WORKER_GROUP,
TEST_WORKER_GROUP);
}
- @Before
+ @BeforeEach
public void before() {
Review Comment:
remove if no need.
##########
dolphinscheduler-worker/src/test/java/org/apache/dolphinscheduler/server/worker/registry/WorkerRegistryClientTest.java:
##########
@@ -78,28 +79,21 @@ public class WorkerRegistryClientTest {
// workerGroups = Sets.newHashSet(DEFAULT_WORKER_GROUP,
TEST_WORKER_GROUP);
}
- @Before
+ @BeforeEach
public void before() {
-
given(workerConfig.getGroups()).willReturn(Sets.newHashSet("127.0.0.1"));
- //
given(heartBeatExecutor.getWorkerGroups()).willReturn(Sets.newHashSet("127.0.0.1"));
- // scheduleAtFixedRate
- given(heartBeatExecutor.scheduleAtFixedRate(Mockito.any(),
Mockito.anyLong(), Mockito.anyLong(),
- Mockito.any(TimeUnit.class))).willReturn(null);
-
}
@Test
public void testStart() {
- workerRegistryClient.initWorkRegistry();
-
- given(workerManagerThread.getThreadPoolQueueSize()).willReturn(1);
-
- given(registryClient.checkNodeExists(Mockito.anyString(),
Mockito.any(NodeType.class))).willReturn(true);
+
given(workerConfig.getWorkerAddress()).willReturn(NetUtils.getAddr(1234));
given(workerConfig.getHeartbeatInterval()).willReturn(Duration.ofSeconds(1));
+ given(registryClient.checkNodeExists(Mockito.anyString(),
Mockito.any(NodeType.class))).willReturn(true);
+ workerRegistryClient.initWorkRegistry();
workerRegistryClient.start();
+ Assertions.assertTrue(true);
Review Comment:
What is the meaning of this assert?
##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/registry/ServerNodeManager.java:
##########
@@ -327,6 +268,50 @@ private void updateMasterNodes() {
}
+ private void updateWorkerNodes() {
+ workerGroupWriteLock.lock();
+ try {
+ Map<String, String> workerNodeMaps =
registryClient.getServerMaps(NodeType.WORKER);
+ for (Map.Entry<String, String> entry : workerNodeMaps.entrySet()) {
+ workerNodeInfo.put(entry.getKey(),
JSONUtils.parseObject(entry.getValue(), WorkerHeartBeat.class));
+ }
+ } finally {
+ workerGroupWriteLock.unlock();
+ }
+ }
+
+ private void updateWorkerGroupMappings() {
+ List<WorkerGroup> workerGroups =
workerGroupMapper.queryAllWorkerGroup();
+ Map<String, Set<String>> tmpWorkerGroupMappings = new HashMap<>();
+ try {
+ workerNodeInfoReadLock.lock();
+ for (WorkerGroup workerGroup : workerGroups) {
+ String workerGroupName = workerGroup.getName();
+ String[] workerAddresses =
workerGroup.getAddrList().split(Constants.COMMA);
+ if (ArrayUtils.isEmpty(workerAddresses)) {
+ continue;
+ }
+ Set<String> activeWorkerNodes = Arrays.stream(workerAddresses)
+
.filter(workerNodeInfo::containsKey).collect(Collectors.toSet());
+ tmpWorkerGroupMappings.put(workerGroupName, activeWorkerNodes);
+ }
+ if
(!tmpWorkerGroupMappings.containsKey(Constants.DEFAULT_WORKER_GROUP)) {
+ tmpWorkerGroupMappings.put(Constants.DEFAULT_WORKER_GROUP,
workerNodeInfo.keySet());
+ }
Review Comment:
If we remove the worker group config, it seems the default worker group
should always be added.
```suggestion
tmpWorkerGroupMappings.put(Constants.DEFAULT_WORKER_GROUP,
workerNodeInfo.keySet());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
