Abhishek Rawat created IMPALA-13163: ---------------------------------------
Summary: Use proper defaults when executor membership snapshot is empty Key: IMPALA-13163 URL: https://issues.apache.org/jira/browse/IMPALA-13163 Project: IMPALA Issue Type: Bug Components: Backend Reporter: Abhishek Rawat During initialization, the executor group membership snapshot may not arrive (i.e empty executor group membership) at coordinator before queries get submitted, and in that case coordinator uses some default resources (executor group size, mem and cpu) for planning the query https://github.com/apache/impala/blob/branch-4.4.0/fe/src/main/java/org/apache/impala/service/Frontend.java#L2053-L2059 {code:java} I0610 15:05:27.176682 338152 Frontend.java:2035] 4a4b33e82891e4df:b235a7a900000000] The original executor group sets from executor membership snapshot: [] I0610 15:05:27.176748 338152 Frontend.java:2056] 4a4b33e82891e4df:b235a7a900000000] A total of 1 executor group sets to be considered for auto-scaling: [TExecutorGroupSet(curr_num_executors:1, expected_num_executors:20, exec_group_name_prefix:default-pool, max_mem_limit:9223372036854775807, num_cores_per_executor:2147483647)] {code} Using these defaults could result in query getting planned using unrealistic resources and may become unschedulable and will cause delays for other queries behind it in the admission queue. The current defaults are not ideal and in most cases these resources and cluster size could be provided as flagfile options during startup, since the cluster size and resources is generally known at startup. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org