Kurt Deschler has posted comments on this change. ( http://gerrit.cloudera.org:8080/19807 )
Change subject: IMPALA-12091: Control scan parallelism by its processing cost ...................................................................... Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/19807/3/be/src/exec/scan-node.cc File be/src/exec/scan-node.cc: http://gerrit.cloudera.org:8080/#/c/19807/3/be/src/exec/scan-node.cc@298 PS3, Line 298: max_row_batches = max(2, max_row_batches / parent->plan_node().effective_dop()); > The max_row_batch value that has FLAGS_max_queued_row_batches_per_scanner_t I think the scanner threads should be computed last, based on the planned parallelism. Even better if they can be determined in the planner and factored in during EG assignment. http://gerrit.cloudera.org:8080/#/c/19807/3/be/src/service/query-options.h File be/src/service/query-options.h: http://gerrit.cloudera.org:8080/#/c/19807/3/be/src/service/query-options.h@293 PS3, Line 293: QUERY_OPT_FN(processing_cost_max_threads, PROCESSING_COST_MAX_THREADS, \ > This is intended to match PROCESSING_COST_MIN_THREADS that already exist, a We should find a name that implies per-fragment per-instance. This is a limit and may not be tied to processing cost alone so would rather not mention that term. -- To view, visit http://gerrit.cloudera.org:8080/19807 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If948e45455275d9a61a6cd5d6a30a8b98a7c729a Gerrit-Change-Number: 19807 Gerrit-PatchSet: 3 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Comment-Date: Thu, 04 May 2023 15:39:10 +0000 Gerrit-HasComments: Yes
