Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20273 )
Change subject: IMPALA-12312: Using correct executor group set info for planning ...................................................................... Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/20273/6/be/src/scheduling/cluster-membership-mgr.cc File be/src/scheduling/cluster-membership-mgr.cc: http://gerrit.cloudera.org:8080/#/c/20273/6/be/src/scheduling/cluster-membership-mgr.cc@671 PS6, Line 671: // Add a default exec group set if no expected group sets were specified. : exec_group_sets.emplace_back(); : exec_group_sets.back().__set_expected_num_executors(FLAGS_num_expected_executors); > Agree with Riza. We only add 'default' EG if expected EGs are not set. With and without this change, if some backends/executors in a cluster are configured with '--executor_groups=default' or configured without setting the 'executor_groups' flag(an empty string in this flag means 'default'), the coordinator can find the 'default' group in 'all_groups'. Actually, all executors and executor groups in the cluster can be found in the cluster membership snapshot. Impala now(without this change), only sends the 'default' EG to the frontend as long as a 'default' group can be found in the cluster membership snapshot, no matter whether the flag 'expected_exec_group_sets' is set or not. IIUC, 'cluster in multiple executor group set mode' means multiple EG sets are configured in the startup flag 'expected_exec_group_sets'. Impala should assume that the cluster only has one 'default' EG if the flag 'expected_exec_group_sets' is not set. Is this right? If so, seems we need a check on whether EGs in 'expected_exec_group_sets' and EGs in the cluster membership snapshot are consistent and default groups should not exist with other non-default groups. If not, frontend will see different cluster members with backends, and can't make good query plans. -- To view, visit http://gerrit.cloudera.org:8080/20273 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia13fb40558441d4dcc0b3e7910d3746bb61e6b80 Gerrit-Change-Number: 20273 Gerrit-PatchSet: 7 Gerrit-Owner: Yifan Zhang <chinazhangyi...@163.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com> Gerrit-Reviewer: Yifan Zhang <chinazhangyi...@163.com> Gerrit-Comment-Date: Tue, 26 Sep 2023 08:56:19 +0000 Gerrit-HasComments: Yes
