Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20612 )
Change subject: IMPALA-3825: Delegate runtime filter aggregation to some executors ...................................................................... Patch Set 16: Code-Review+1 (4 comments) http://gerrit.cloudera.org:8080/#/c/20612/10/be/src/runtime/runtime-filter-bank.cc File be/src/runtime/runtime-filter-bank.cc: http://gerrit.cloudera.org:8080/#/c/20612/10/be/src/runtime/runtime-filter-bank.cc@139 PS10, Line 139: esc.filter_id); > I think what you mean is, it is ok to allocate later as long as the whole t Yes, this is the way I meant it, thanks http://gerrit.cloudera.org:8080/#/c/20612/16/be/src/runtime/runtime-filter-bank.cc File be/src/runtime/runtime-filter-bank.cc: http://gerrit.cloudera.org:8080/#/c/20612/16/be/src/runtime/runtime-filter-bank.cc@241 PS16, Line 241: LOG(ERROR) << "Cannot allocate scratch bloom filter for pending_remote_filter " : << "of filter " << params.filter_id(); : bloom_filter = BloomFilter::ALWAYS_TRUE_FILTER; Is my understanding right that we should fail the query in this case as this means a program error? If failing the query from here is tricky, a DCHECK could be hit in DEBUG while RELEASE mode could rely on setting to ALWAYS_TRUE_FILTER. My concern with the graceful handling is that if there is an error like underestimating memory it may remain undiscovered in tests. http://gerrit.cloudera.org:8080/#/c/20612/16/be/src/runtime/runtime-filter-bank.cc@291 PS16, Line 291: turn nit: turned http://gerrit.cloudera.org:8080/#/c/20612/16/be/src/runtime/runtime-filter-bank.cc@293 PS16, Line 293: produced_filter.pending_remotes = 0; Should we also set pending_producers to 0? It would be make sense for produced_filter.IsComplete() to return true below. Or it can be a problem if we won't wait for local filters? Also, can you add a VLOG(2) line here to be able to trace this scenario? -- To view, visit http://gerrit.cloudera.org:8080/20612 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I11d38ed0f223d6e5b32a19ebe725af7738ee4ab0 Gerrit-Change-Number: 20612 Gerrit-PatchSet: 16 Gerrit-Owner: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Abhishek Rawat <ara...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Comment-Date: Mon, 18 Dec 2023 17:36:49 +0000 Gerrit-HasComments: Yes
