Hello Kurt Deschler, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20475 to look at the new patch set (#2). Change subject: IMPALA-12444: Fix minimum parallelism bug in scan fragment ...................................................................... IMPALA-12444: Fix minimum parallelism bug in scan fragment Scan fragment did not follow PROCESSING_COST_MIN_THREADS set by user even if total scan ranges allow to do so. This patch fix the issue by exposing ScanNode.maxScannerThreads_ to PlanFragment.adjustToMaxParallelism(). By using ScanNode.maxScannerThreads_ as an upper bound, ScanNode does not need to artificially lower ProcessingCost if maxScannerThreads_ is lower than minimum parallelism dictated by the original ProcessingCost. Thus, the synthetic ProcessingCost logic in ScanNode class is revised to only apply if input cardinality is unknown (-1). This patch also does the following adjustments: - Remove some dead codes in Frontend.java and PlanFragment.java. - Add sanity check such that PROCESSING_COST_MIN_THREADS <= MAX_FRAGMENT_INSTANCES_PER_NODE. - Tidy up test_query_cpu_count_divisor_default to reduce number of SET query. Testing: - Update test_query_cpu_count_divisor_default to ensure that PROCESSING_COST_MIN_THREADS is respected by scan fragment and error is returned if PROCESSING_COST_MIN_THREADS is greater than MAX_FRAGMENT_INSTANCES_PER_NODE. - Pass test_executor_groups.py. Change-Id: I69e5a80146d4ac41de5ef406fc2bdceffe3ec394 --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/planner/CostingSegment.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/main/java/org/apache/impala/planner/ScanNode.java M fe/src/main/java/org/apache/impala/service/Frontend.java M tests/custom_cluster/test_executor_groups.py 6 files changed, 61 insertions(+), 85 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/20475/2 -- To view, visit http://gerrit.cloudera.org:8080/20475 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I69e5a80146d4ac41de5ef406fc2bdceffe3ec394 Gerrit-Change-Number: 20475 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>