Jackie-Jiang opened a new pull request, #10886: URL: https://github.com/apache/pinot/pull/10886
When partitioning is enabled, and 2 tables have segments colocated for each partition, we may process each partition in parallel. This PR leverages the partition information gathered in `SegmentPartitionMetadataManager`, and when `is_colocated_by_join_keys` is provided in the query hint, query planner will pick a server for each partition that serves all the segments for the partition. Then it will distribute the work for each partition into multiple workers to achieve the parallelism. To enable `SegmentPartitionMetadataManager`, the following config needs to be set in the pinot broker: `pinot.broker.enable.partition.metadata.manager = true`. Tested in `ColocatedJoinEngineQuickStart`. Will add more tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
