Jackie-Jiang commented on code in PR #12079:
URL: https://github.com/apache/pinot/pull/12079#discussion_r1419411451
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/routing/WorkerManager.java:
##########
@@ -208,10 +346,31 @@ private RoutingTable getRoutingTable(String tableName,
TableType tableType, long
CalciteSqlCompiler.compileToBrokerRequest("SELECT * FROM " +
tableNameWithType), requestId);
}
+ // --------------------------------------------------------------------------
+ // Partitioned leaf stage assignment
+ // --------------------------------------------------------------------------
private void assignWorkersToPartitionedLeafFragment(DispatchablePlanMetadata
metadata,
- DispatchablePlanContext context, String partitionKey, int numPartitions,
int partitionParallelism) {
+ DispatchablePlanContext context, String partitionKey, Map<String,
String> tableOptions) {
+ // when partition key exist, we assign workers for leaf-stage in
partitioned fashion.
+
+ String numPartitionsStr =
tableOptions.get(PinotHintOptions.TableHintOptions.PARTITION_SIZE);
+ Preconditions.checkState(numPartitionsStr != null, "'%s' must be provided
for partition key: %s",
+ PinotHintOptions.TableHintOptions.PARTITION_SIZE, partitionKey);
+ int numPartitions = Integer.parseInt(numPartitionsStr);
+ Preconditions.checkState(numPartitions > 0, "'%s' must be positive, got:
%s",
+ PinotHintOptions.TableHintOptions.PARTITION_SIZE, numPartitions);
+
+ String partitionFunction =
tableOptions.getOrDefault(PinotHintOptions.TableHintOptions.PARTITION_FUNCTION,
Review Comment:
Understood. Currently we assume all table options using the same partition
function. Putting `Murmur` as the default might be safer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
