Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/20104 )
Change subject: IMPALA-11842: Improve memory estimation for Aggregate ...................................................................... Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/20104/1/common/thrift/Query.thrift File common/thrift/Query.thrift: http://gerrit.cloudera.org:8080/#/c/20104/1/common/thrift/Query.thrift@118 PS1, Line 118: const i64 MIN_HASH_TBL_MEM = 10485760 // 10MB > Setting a min size may negatively affect L1/L2 caching. This is just a constant move from AggregationNode.java to Query.thrift. Value remains unchanged. It is mainly used in this formula at AggregationNode.java if (aggInfo.getGroupingExprs().isEmpty()) { perInstanceMemEstimate = MIN_PLAIN_AGG_MEM; } else { perInstanceMemEstimate = Math.max(perInstanceDataBytes, QueryConstants.MIN_HASH_TBL_MEM); } Thrift compiler does not accept asterisk sign. Added comment instead. http://gerrit.cloudera.org:8080/#/c/20104/1/common/thrift/Query.thrift@645 PS1, Line 645: 159: optional i64 large_agg_bytes_threshold = 536870912 // 512MB > Change to 512 * 1024 *1024 for readability Thrift compiler does not accept asterisk sign. Added comment instead. http://gerrit.cloudera.org:8080/#/c/20104/1/fe/src/main/java/org/apache/impala/planner/PlanFragment.java File fe/src/main/java/org/apache/impala/planner/PlanFragment.java: http://gerrit.cloudera.org:8080/#/c/20104/1/fe/src/main/java/org/apache/impala/planner/PlanFragment.java@567 PS1, Line 567: maxNdv = Math.max(maxNdv, numDistinct); > Please add a correlation factor like in IMPALA-12006 so that the correlatio Added corrFactor in AggregationNode.java -- To view, visit http://gerrit.cloudera.org:8080/20104 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4b4b2e519ee89f0a13fdb62d0471ee4047f6421 Gerrit-Change-Number: 20104 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Abhishek Rawat <ara...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Comment-Date: Wed, 21 Jun 2023 22:39:00 +0000 Gerrit-HasComments: Yes
