Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11692 )
Change subject: IMPALA-7351: Add estimates to Exchange node ...................................................................... Patch Set 3: (3 comments) http://gerrit.cloudera.org:8080/#/c/11692/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11692/3//COMMIT_MSG@10 PS3, Line 10: method in the in-line comments. Do we know what the likely upper bound is for the memory estimates on larger clusters and data sets? It seems like the worst case is probably a merging exchange on a large cluster right, since you have a large number of queues? My understanding is that in the other cases the queue size provides a reasonable cap because there is only one queue. http://gerrit.cloudera.org:8080/#/c/11692/3/fe/src/main/java/org/apache/impala/planner/ExchangeNode.java File fe/src/main/java/org/apache/impala/planner/ExchangeNode.java: http://gerrit.cloudera.org:8080/#/c/11692/3/fe/src/main/java/org/apache/impala/planner/ExchangeNode.java@77 PS3, Line 77: if (!streamSink.getOutputPartition().isPartitioned() && fragment_.isPartitioned()) { nit: can just return this expression http://gerrit.cloudera.org:8080/#/c/11692/3/fe/src/main/java/org/apache/impala/planner/ExchangeNode.java@241 PS3, Line 241: long deferredBatchQueueSize = avgRowBatchByteSize * numSenders; nit: unnecessary intermediate variable -- To view, visit http://gerrit.cloudera.org:8080/11692 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Gerrit-Change-Number: 11692 Gerrit-PatchSet: 3 Gerrit-Owner: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Vuk Ercegovac <vercego...@cloudera.com> Gerrit-Comment-Date: Wed, 24 Oct 2018 18:15:50 +0000 Gerrit-HasComments: Yes
