akpatnam25 commented on code in PR #3347:
URL: https://github.com/apache/celeborn/pull/3347#discussion_r2206527284
##########
master/src/main/java/org/apache/celeborn/service/deploy/master/SlotsAllocator.java:
##########
@@ -341,22 +481,53 @@ static List<WorkerInfo>
generateRackAwareWorkers(List<WorkerInfo> workers) {
return Collections.unmodifiableList(result);
}
+ /**
+ * Assigns slots in a roundrobin fashion given lists of primary and replica
worker candidates and
+ * other restrictions.
+ *
+ * @param slots the slots that have been assigned for each partitionId
+ * @param partitionIds the partitionIds that require slot selection still
+ * @param primaryWorkers list of worker candidates that can be used for
primary workers.
+ * @param replicaWorkers list of worker candidates that can be used for
replica workers.
+ * @param slotsRestrictions restrictions for each available slot based on
worker characteristics
+ * @param shouldReplicate if replication is enabled within the cluster
+ * @param shouldRackAware if rack-aware replication is enabled within the
cluster.
+ * @param availableStorageTypes available storage types coming from the
offer slots request.
+ * @param replicaSameAsPrimary if the worker candidates list for primaries
and replicas is the
+ * same. This is to prevent index mismatch while assigning slots across
both lists.
+ * @return the partitionIds that were not able to be assigned slots in this
iteration with the
+ * current primary and replica worker candidates and slot restrictions..
+ */
private static List<Integer> roundRobin(
Map<WorkerInfo, Tuple2<List<PartitionLocation>,
List<PartitionLocation>>> slots,
List<Integer> partitionIds,
- List<WorkerInfo> workers,
+ List<WorkerInfo> primaryWorkers,
+ List<WorkerInfo> replicaWorkers,
Map<WorkerInfo, List<UsableDiskInfo>> slotsRestrictions,
boolean shouldReplicate,
boolean shouldRackAware,
- int availableStorageTypes) {
+ int availableStorageTypes,
+ boolean replicaSameAsPrimary) {
Review Comment:
sounds good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
