jackjlli commented on issue #10607:
URL: https://github.com/apache/pinot/issues/10607#issuecomment-1509404993
Discussed with @priyen offline. We modified the existing unit test in order
to print out the output of the instance partition mapping, and we don't see the
scenarios where only partial of the pool are assigned. We suspected that some
configs might have been messed up during that time. Based on the output of the
edited unit test, we don't see the current algorithm behaves abnormally.
The below is the edited unit test:
```
@Test
public void testPoolBased() {
// 10 instances in 2 pools, each with 5 instances. Pool 0: 0-4, Pool 1:
5-9
int numInstances = 10;
int numPools = 2;
int numInstancesPerPool = numInstances / numPools;
int numReplicaGroups = 2;
List<InstanceConfig> instanceConfigs = new ArrayList<>(numInstances);
for (int i = 0; i < numInstances; i++) {
InstanceConfig instanceConfig = new
InstanceConfig(SERVER_INSTANCE_ID_PREFIX + i);
instanceConfig.addTag(OFFLINE_TAG);
int pool = i / numInstancesPerPool;
instanceConfig.getRecord()
.setMapField(InstanceUtils.POOL_KEY,
Collections.singletonMap(OFFLINE_TAG, Integer.toString(pool)));
instanceConfigs.add(instanceConfig);
}
InstanceReplicaGroupPartitionConfig replicaPartitionConfig = new
InstanceReplicaGroupPartitionConfig(true, 10, numReplicaGroups, 5, 24, 5, true);
InstanceTagPoolConfig tagPoolConfig = new
InstanceTagPoolConfig(OFFLINE_TAG, true, numPools, null);
TableConfig tableConfig = new
TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME)
.setInstanceAssignmentConfigMap(Collections.singletonMap(InstancePartitionsType.OFFLINE.toString(),
new InstanceAssignmentConfig(tagPoolConfig, null,
replicaPartitionConfig))).build();
tableConfig.setInstanceAssignmentConfigMap(Collections.singletonMap(InstancePartitionsType.OFFLINE.toString(),
new InstanceAssignmentConfig(tagPoolConfig, null,
replicaPartitionConfig)));
InstanceAssignmentDriver driver = new
InstanceAssignmentDriver(tableConfig);
InstancePartitions existingInstancePartitions = null;
InstancePartitions instancePartitions =
driver.assignInstances(InstancePartitionsType.OFFLINE,
instanceConfigs, existingInstancePartitions);
assertEquals(instancePartitions.getNumReplicaGroups(), numReplicaGroups);
assertEquals(instancePartitions.getNumPartitions(), 24);
System.out.println(instancePartitions);
// Now 30 instances in 2 pools, each with 15 instances. Pool 0: 10-19,
Pool 1: 20-29
numInstances = 30;
int moreNumInstancesPerPool = 10;
for (int i = 10; i < numInstances; i++) {
InstanceConfig instanceConfig = new
InstanceConfig(SERVER_INSTANCE_ID_PREFIX + i);
instanceConfig.addTag(OFFLINE_TAG);
int pool = (i - 10) / moreNumInstancesPerPool;
instanceConfig.getRecord()
.setMapField(InstanceUtils.POOL_KEY,
Collections.singletonMap(OFFLINE_TAG, Integer.toString(pool)));
instanceConfigs.add(instanceConfig);
}
replicaPartitionConfig = new InstanceReplicaGroupPartitionConfig(true,
30, numReplicaGroups, 15, 24, 5, true);
tableConfig.setInstanceAssignmentConfigMap(Collections.singletonMap(InstancePartitionsType.OFFLINE.toString(),
new InstanceAssignmentConfig(tagPoolConfig, null,
replicaPartitionConfig)));
driver = new InstanceAssignmentDriver(tableConfig);
instancePartitions =
driver.assignInstances(InstancePartitionsType.OFFLINE,
instanceConfigs, existingInstancePartitions);
assertEquals(instancePartitions.getNumReplicaGroups(), numReplicaGroups);
assertEquals(instancePartitions.getNumPartitions(), 24);
System.out.println(instancePartitions);
}
```
Since there is no issue found out from the tests, I don't think it's worthy
of keeping this issue open. I'll close this issue for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
