Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19033 )
Change subject: IMPALA-11604 Planner changes for CPU usage ...................................................................... Patch Set 42: (7 comments) http://gerrit.cloudera.org:8080/#/c/19033/41/common/thrift/Frontend.thrift File common/thrift/Frontend.thrift: http://gerrit.cloudera.org:8080/#/c/19033/41/common/thrift/Frontend.thrift@769 PS41, Line 769: // The optional max_mem_limit to determine which executor group set to run for a query. : // The max_mem_limit value is set to the max_query_mem_limit attribute of the group set : // with name prefix 'exec_group_name_prefix' from the pool service. For each query, : // the frontend computes the per host estimated-memory after a compilation with a : // number of executor nodes from this group set and compares it with this variable. : 4: optional i64 max_mem_limit : : // The optional num_cores_per_executor is used to determine which > Restore the comment. Done http://gerrit.cloudera.org:8080/#/c/19033/41/common/thrift/Query.thrift File common/thrift/Query.thrift: http://gerrit.cloudera.org:8080/#/c/19033/41/common/thrift/Query.thrift@912 PS41, Line 912: > This need comment. Done http://gerrit.cloudera.org:8080/#/c/19033/41/common/thrift/Query.thrift@914 PS41, Line 914: > Consider dropping this and put it to LOG instead. Done http://gerrit.cloudera.org:8080/#/c/19033/41/fe/src/main/java/org/apache/impala/planner/PlanFragment.java File fe/src/main/java/org/apache/impala/planner/PlanFragment.java: http://gerrit.cloudera.org:8080/#/c/19033/41/fe/src/main/java/org/apache/impala/planner/PlanFragment.java@150 PS41, Line 150: adjustedInstanceCount_ > This can be set multiple time during effective parallelism computation. Added isFixedParallelism_. http://gerrit.cloudera.org:8080/#/c/19033/41/fe/src/main/java/org/apache/impala/planner/PlanFragment.java@953 PS41, Line 953: "ProcessingCost Fragment " + getId() + " has not been computed!"); : int nodeStepCount = getNumInstances() % getNumNodes() == 0 ? getNumNodes() : 1; : boolean canTryLower = adjus > I believe effective parallelism adjustment and processing cost finding can Done http://gerrit.cloudera.org:8080/#/c/19033/41/fe/src/main/java/org/apache/impala/planner/Planner.java File fe/src/main/java/org/apache/impala/planner/Planner.java: http://gerrit.cloudera.org:8080/#/c/19033/41/fe/src/main/java/org/apache/impala/planner/Planner.java@317 PS41, Line 317: : hasHeader = true; : } > This might still be useful to show in EXTENDED explain level. Done http://gerrit.cloudera.org:8080/#/c/19033/41/fe/src/main/java/org/apache/impala/planner/Planner.java@320 PS41, Line 320: arn if the planner is running in DEBUG mode. : if (request.query_ctx.client_request.query_options.planner_testcase_mode) { : str.append("WARNING > move to LOG instead. Done -- To view, visit http://gerrit.cloudera.org:8080/19033 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If32dc770dfffcdd0be2b5555a789a7720952c68a Gerrit-Change-Number: 19033 Gerrit-PatchSet: 42 Gerrit-Owner: Qifan Chen <qfc...@hotmail.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Qifan Chen <qfc...@hotmail.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com> Gerrit-Comment-Date: Sat, 04 Feb 2023 03:43:38 +0000 Gerrit-HasComments: Yes
