gortiz commented on code in PR #15135:
URL: https://github.com/apache/pinot/pull/15135#discussion_r1973339777
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/DispatchablePlanContext.java:
##########
@@ -130,14 +134,26 @@ public List<DispatchablePlanFragment>
constructDispatchablePlanFragmentList(Plan
dispatchablePlanFragment.setTimeBoundaryInfo(dispatchablePlanMetadata.getTimeBoundaryInfo());
}
}
- return Arrays.asList(dispatchablePlanFragmentArray);
+ return dispatchablePlanFragmentMap;
}
- private void createDispatchablePlanFragmentList(DispatchablePlanFragment[]
dispatchablePlanFragmentArray,
- PlanFragment planFragmentRoot) {
- dispatchablePlanFragmentArray[planFragmentRoot.getFragmentId()] = new
DispatchablePlanFragment(planFragmentRoot);
- for (PlanFragment childPlanFragment : planFragmentRoot.getChildren()) {
- createDispatchablePlanFragmentList(dispatchablePlanFragmentArray,
childPlanFragment);
+ private Map<Integer, DispatchablePlanFragment>
createDispatchablePlanFragmentMap(PlanFragment planFragmentRoot) {
+ HashMap<Integer, DispatchablePlanFragment> result =
+ Maps.newHashMapWithExpectedSize(_dispatchablePlanMetadataMap.size());
+ Queue<PlanFragment> pendingPlanFragmentIds = new ArrayDeque<>();
+ pendingPlanFragmentIds.add(planFragmentRoot);
+ while (!pendingPlanFragmentIds.isEmpty()) {
+ PlanFragment planFragment = pendingPlanFragmentIds.poll();
+ int planFragmentId = planFragment.getFragmentId();
+
+ if (result.containsKey(planFragmentId)) {
+ LOGGER.info("plan fragment {} found twice", planFragmentId);
+ continue;
+ }
Review Comment:
Yes, it can happen and in fact it does each time a spool is used. Here we
iterate from the root stage and visit the children recursively. Given spools
have been already applied, some stages can be visited more than once. By doing
this we reduce the work that needs to be done.
I'm going to change the log to debug and add a javadoc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
