Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24883 )
Change subject: IMPALA-13052: Estimate reservoir sample memory in aggregations ...................................................................... Patch Set 1: (2 comments) the current jenkins run is likely to fail, the fix is to rebase to get https://gerrit.cloudera.org/#/c/24898/ http://gerrit.cloudera.org:8080/#/c/24883/1/fe/src/main/java/org/apache/impala/planner/AggregationNode.java File fe/src/main/java/org/apache/impala/planner/AggregationNode.java: http://gerrit.cloudera.org:8080/#/c/24883/1/fe/src/main/java/org/apache/impala/planner/AggregationNode.java@86 PS1, Line 86: The state and the array are allocated by FunctionContext::Allocate() : // outside the reservation, so running short of this memory doesn't make the : // aggregation spill (IMPALA-3304). Would it be hard to do it within reservation? Currently these aggregates can cause trouble, for example by using excessive memory during preaggregation. Reserving the worst case memory based on RESERVOIR_MAX_CAPACITY would allow limiting pre-agg and properly spill in the final aggregation. It may make some queries slower, but avoid OOM errors. http://gerrit.cloudera.org:8080/#/c/24883/1/fe/src/main/java/org/apache/impala/planner/AggregationNode.java@1190 PS1, Line 1190: getFreePoolBytes this is not aggregation specific, some more central class would be better place for the function IMO -- To view, visit http://gerrit.cloudera.org:8080/24883 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id6eec3daf7fea9b027ff0be9c95c6ee5a3cf323d Gerrit-Change-Number: 24883 Gerrit-PatchSet: 1 Gerrit-Owner: Aleksandr Efimov <[email protected]> Gerrit-Reviewer: Aleksandr Efimov <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Comment-Date: Sun, 20 Sep 2026 19:34:37 +0000 Gerrit-HasComments: Yes
