Abhishek Rawat has posted comments on this change. ( http://gerrit.cloudera.org:8080/21762 )
Change subject: IMPALA-13333: Limit memory estimation if PlanNode can spill ...................................................................... Patch Set 37: (2 comments) http://gerrit.cloudera.org:8080/#/c/21762/37/fe/src/main/java/org/apache/impala/planner/ResourceProfileBuilder.java File fe/src/main/java/org/apache/impala/planner/ResourceProfileBuilder.java: http://gerrit.cloudera.org:8080/#/c/21762/37/fe/src/main/java/org/apache/impala/planner/ResourceProfileBuilder.java@103 PS37, Line 103: memEstimateBytes_ = Math.min(memEstimateBytes_, scaleBasedMemEstimate); For the case where `minMemReservationBytes_ > maxMemEstimateBytesAfterScaling_`, the query will anyways be rejected at admission. Should we skip updating the `memEstimateBytes_` in that case, I guess it's kind of immaterial but it might be confusing if we log mem estimate same as min reservation in query profile. http://gerrit.cloudera.org:8080/#/c/21762/37/fe/src/main/java/org/apache/impala/service/Frontend.java File fe/src/main/java/org/apache/impala/service/Frontend.java: http://gerrit.cloudera.org:8080/#/c/21762/37/fe/src/main/java/org/apache/impala/service/Frontend.java@2136 PS37, Line 2136: new_entry.setMax_mem_limit( Ideally, we want a logic similar to Analyzer.getMaxMemLimitPerHost(false) to set the max mem limit for executors. I see that we can't call it here because we call setPoolMemLimit() much later. -- To view, visit http://gerrit.cloudera.org:8080/21762 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I290c4e889d4ab9e921e356f0f55a9c8b11d0854e Gerrit-Change-Number: 21762 Gerrit-PatchSet: 37 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]> Gerrit-Comment-Date: Tue, 11 Mar 2025 23:17:07 +0000 Gerrit-HasComments: Yes
