Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16242 )
Change subject: IMPALA-9979: part 2: partitioned top-n ...................................................................... Patch Set 12: (2 comments) http://gerrit.cloudera.org:8080/#/c/16242/12//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16242/12//COMMIT_MSG@68 PS12, Line 68: * We evict all in memory partitions when under memory pressure - I am thinking now about trying to do this as part of the initial change. I think we want to evict at least a fixed percentage of rows each time we hit memory pressure and also factor in whether the heaps were effective at filtering. If we can keep some of the heaps in memory it'd reduce the sort data size. I think we should also be adaptive and fall back to a sort if there is no filtering happening after the heaps fill up a certain number of times. http://gerrit.cloudera.org:8080/#/c/16242/12//COMMIT_MSG@70 PS12, Line 70: * The top-n node rebuilds an in-memory heap per partition I'd also like to fix this because building the heap does require a bunch of extra comparisons and work. -- To view, visit http://gerrit.cloudera.org:8080/16242 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic638af9495981d889a4cb7455a71e8be0eb1a8e5 Gerrit-Change-Number: 16242 Gerrit-PatchSet: 12 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: David Rorke <dro...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Shant Hovsepian <sh...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Comment-Date: Fri, 31 Jul 2020 21:44:47 +0000 Gerrit-HasComments: Yes
