[ https://issues.apache.org/jira/browse/IMPALA-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Quanlong Huang reassigned IMPALA-10620: --------------------------------------- Assignee: Bikramjeet Vig Haven't seen this in later runs. Maybe it's flaky. [~bikramjeet.vig], assigning this to you first. Feel free to reassign it. Thanks! > TestExecutorGroups.test_executor_group_num_queries_executing_metric fails in > exhaustive release build > ----------------------------------------------------------------------------------------------------- > > Key: IMPALA-10620 > URL: https://issues.apache.org/jira/browse/IMPALA-10620 > Project: IMPALA > Issue Type: Bug > Components: Backend > Reporter: Quanlong Huang > Assignee: Bikramjeet Vig > Priority: Critical > > custom_cluster.test_executor_groups.TestExecutorGroups.test_executor_group_num_queries_executing_metric > fails in an exhaustive release build. > *Stacktrace* > {code:java} > custom_cluster/test_executor_groups.py:436: in > test_executor_group_num_queries_executing_metric > assert self._get_num_queries_executing_for_exec_group(group_names[0]) == > 1, \ > E AssertionError: The lingering query should be accounted for when the > group comes back up. > E assert 0 == 1 > E + where 0 = <bound method > TestExecutorGroups._get_num_queries_executing_for_exec_group of > <test_executor_groups.TestExecutorGroups object at 0x7f870cecba50>>('group1') > E + where <bound method > TestExecutorGroups._get_num_queries_executing_for_exec_group of > <test_executor_groups.TestExecutorGroups object at 0x7f870cecba50>> = > <test_executor_groups.TestExecutorGroups object at > 0x7f870cecba50>._get_num_queries_executing_for_exec_group{code} > *Standard Error* > {code} > -- 2021-03-29 03:45:18,032 INFO MainThread: Starting cluster with > command: > /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/bin/start-impala-cluster.py > '--state_store_args=--statestore_update_frequency_ms=50 > --statestore_priority_update_frequency_ms=50 > --statestore_heartbeat_frequency_ms=50' --cluster_size=1 --num_coordinators=1 > --log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests > --log_level=1 --use_exclusive_coordinators > --impalad_args=--enable_orc_scanner=true '--impalad_args= > -executor_groups=coordinator ' --impalad_args=--default_query_options= > 03:45:18 MainThread: Starting impala cluster without executors > 03:45:18 MainThread: Found 0 impalad/0 statestored/0 catalogd process(es) > 03:45:18 MainThread: Starting State Store logging to > /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/statestored.INFO > 03:45:18 MainThread: Starting Catalog Service logging to > /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/catalogd.INFO > 03:45:18 MainThread: Starting Impala Daemon logging to > /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad.INFO > 03:45:21 MainThread: Found 1 impalad/1 statestored/1 catalogd process(es) > 03:45:21 MainThread: Found 1 impalad/1 statestored/1 catalogd process(es) > 03:45:21 MainThread: Getting num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > 03:45:21 MainThread: Waiting for num_known_live_backends=1. Current value: 0 > 03:45:22 MainThread: Found 1 impalad/1 statestored/1 catalogd process(es) > 03:45:22 MainThread: Getting num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > 03:45:22 MainThread: Waiting for num_known_live_backends=1. Current value: 0 > 03:45:23 MainThread: Found 1 impalad/1 statestored/1 catalogd process(es) > 03:45:23 MainThread: Getting num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > 03:45:23 MainThread: num_known_live_backends has reached value: 1 > 03:45:24 MainThread: Impala Cluster Running with 1 nodes (1 coordinators, 0 > executors). > -- 2021-03-29 03:45:25,419 DEBUG MainThread: Found 1 impalad/1 > statestored/1 catalogd process(es) > -- 2021-03-29 03:45:25,419 INFO MainThread: Getting metric: > statestore.live-backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25010 > -- 2021-03-29 03:45:25,422 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:25,425 INFO MainThread: Metric > 'statestore.live-backends' has reached desired value: 2 > -- 2021-03-29 03:45:25,425 DEBUG MainThread: Getting > num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > -- 2021-03-29 03:45:25,428 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:25,430 INFO MainThread: num_known_live_backends has > reached value: 1 > SET > client_identifier=custom_cluster/test_executor_groups.py::TestExecutorGroups::()::test_executor_group_num_queries_executing_metric; > -- connecting to: localhost:21000 > -- connecting to localhost:21050 with impyla > -- 2021-03-29 03:45:25,640 INFO MainThread: Closing active operation > -- connecting to localhost:28000 with impyla > -- 2021-03-29 03:45:25,769 INFO MainThread: Closing active operation > -- 2021-03-29 03:45:25,778 INFO MainThread: Adding 1 executors to group > default-pool-group1 with minimum size 1 > -- 2021-03-29 03:45:25,778 INFO MainThread: Starting cluster with > command: > /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/bin/start-impala-cluster.py > '--state_store_args=--statestore_update_frequency_ms=50 > --statestore_priority_update_frequency_ms=50 > --statestore_heartbeat_frequency_ms=50' --cluster_size=1 --num_coordinators=0 > --log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests > --log_level=1 --add_executors --impalad_args=--enable_orc_scanner=true > --impalad_args=-admission_control_slots=1 > --impalad_args=-executor_groups=default-pool-group1:1 > --impalad_args=--default_query_options= > 03:45:26 MainThread: Found 1 impalad/1 statestored/1 catalogd process(es) > 03:45:26 MainThread: Starting Impala Daemon logging to > /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO > 03:45:29 MainThread: Found 2 impalad/1 statestored/1 catalogd process(es) > 03:45:29 MainThread: Found 2 impalad/1 statestored/1 catalogd process(es) > 03:45:29 MainThread: Getting num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > 03:45:29 MainThread: num_known_live_backends has reached value: 2 > 03:45:29 MainThread: Found 2 impalad/1 statestored/1 catalogd process(es) > 03:45:29 MainThread: Getting num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25001 > 03:45:29 MainThread: num_known_live_backends has reached value: 2 > 03:45:29 MainThread: Impala Cluster Running with 1 nodes (0 coordinators, 1 > executors). > -- 2021-03-29 03:45:30,939 DEBUG MainThread: Found 2 impalad/1 > statestored/1 catalogd process(es) > -- 2021-03-29 03:45:30,939 INFO MainThread: Getting metric: > statestore.live-backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25010 > -- 2021-03-29 03:45:30,943 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:30,945 INFO MainThread: Metric > 'statestore.live-backends' has reached desired value: 3 > -- 2021-03-29 03:45:30,946 DEBUG MainThread: Getting > num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > -- 2021-03-29 03:45:30,948 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:30,950 INFO MainThread: num_known_live_backends has > reached value: 2 > -- 2021-03-29 03:45:30,950 DEBUG MainThread: Getting > num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25001 > -- 2021-03-29 03:45:30,953 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:30,955 INFO MainThread: num_known_live_backends has > reached value: 2 > -- 2021-03-29 03:45:30,955 INFO MainThread: Adding 2 executors to group > default-pool-group2 with minimum size 2 > -- 2021-03-29 03:45:30,956 INFO MainThread: Starting cluster with > command: > /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/bin/start-impala-cluster.py > '--state_store_args=--statestore_update_frequency_ms=50 > --statestore_priority_update_frequency_ms=50 > --statestore_heartbeat_frequency_ms=50' --cluster_size=2 --num_coordinators=0 > --log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests > --log_level=1 --add_executors --impalad_args=--enable_orc_scanner=true > --impalad_args=-admission_control_slots=1 > --impalad_args=-executor_groups=default-pool-group2:2 > --impalad_args=--default_query_options= > 03:45:31 MainThread: Found 2 impalad/1 statestored/1 catalogd process(es) > 03:45:31 MainThread: Starting Impala Daemon logging to > /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO > 03:45:31 MainThread: Starting Impala Daemon logging to > /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad_node3.INFO > 03:45:34 MainThread: Found 4 impalad/1 statestored/1 catalogd process(es) > 03:45:34 MainThread: Found 4 impalad/1 statestored/1 catalogd process(es) > 03:45:34 MainThread: Getting num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > 03:45:34 MainThread: num_known_live_backends has reached value: 4 > 03:45:35 MainThread: Found 4 impalad/1 statestored/1 catalogd process(es) > 03:45:35 MainThread: Getting num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25001 > 03:45:35 MainThread: num_known_live_backends has reached value: 4 > 03:45:35 MainThread: Found 4 impalad/1 statestored/1 catalogd process(es) > 03:45:35 MainThread: Getting num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25002 > 03:45:35 MainThread: num_known_live_backends has reached value: 4 > 03:45:35 MainThread: Found 4 impalad/1 statestored/1 catalogd process(es) > 03:45:35 MainThread: Getting num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25003 > 03:45:35 MainThread: num_known_live_backends has reached value: 4 > 03:45:35 MainThread: Impala Cluster Running with 2 nodes (0 coordinators, 2 > executors). > -- 2021-03-29 03:45:36,249 DEBUG MainThread: Found 4 impalad/1 > statestored/1 catalogd process(es) > -- 2021-03-29 03:45:36,249 INFO MainThread: Getting metric: > statestore.live-backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25010 > -- 2021-03-29 03:45:36,253 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:36,256 INFO MainThread: Metric > 'statestore.live-backends' has reached desired value: 5 > -- 2021-03-29 03:45:36,256 DEBUG MainThread: Getting > num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > -- 2021-03-29 03:45:36,260 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:36,262 INFO MainThread: num_known_live_backends has > reached value: 4 > -- 2021-03-29 03:45:36,262 DEBUG MainThread: Getting > num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25001 > -- 2021-03-29 03:45:36,266 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:36,268 INFO MainThread: num_known_live_backends has > reached value: 4 > -- 2021-03-29 03:45:36,268 DEBUG MainThread: Getting > num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25002 > -- 2021-03-29 03:45:36,271 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:36,273 INFO MainThread: num_known_live_backends has > reached value: 4 > -- 2021-03-29 03:45:36,273 DEBUG MainThread: Getting > num_known_live_backends from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25003 > -- 2021-03-29 03:45:36,277 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:36,279 INFO MainThread: num_known_live_backends has > reached value: 4 > -- 2021-03-29 03:45:36,279 INFO MainThread: Getting metric: > cluster-membership.executor-groups.total-healthy from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > -- 2021-03-29 03:45:36,282 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:36,286 INFO MainThread: Metric > 'cluster-membership.executor-groups.total-healthy' has reached desired value: > 2 > -- 2021-03-29 03:45:36,289 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:36,295 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- executing async: localhost:21000 > select * from functional_parquet.alltypestiny where month < 3 > and id + random() < sleep(500);; > -- 2021-03-29 03:45:37,146 INFO MainThread: Started query > cb44d9e10462384b:6dc51c0e00000000 > -- waiting for completion of the admission control processing of the query: > <tests.common.impala_connection.OperationHandle object at 0x7f870ce96410> > -- 2021-03-29 03:45:37,201 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:37,207 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- executing async: localhost:21000 > select * from functional_parquet.alltypestiny where month < 3 > and id + random() < sleep(500);; > -- 2021-03-29 03:45:37,225 INFO MainThread: Started query > 034cf0551469f82f:2b27a3d500000000 > -- waiting for completion of the admission control processing of the query: > <tests.common.impala_connection.OperationHandle object at 0x7f870cb4fcd0> > -- 2021-03-29 03:45:37,279 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:37,285 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- closing query for operation handle: > <tests.common.impala_connection.OperationHandle object at 0x7f870ce96410> > -- closing query for operation handle: > <tests.common.impala_connection.OperationHandle object at 0x7f870cb4fcd0> > -- 2021-03-29 03:45:37,295 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:37,301 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:38,323 INFO MainThread: Killing <ImpaladProcess PID: > 14833 > (/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/be/build/latest/service/impalad > -is_coordinator=false -disconnected_session_timeout 21600 > -kudu_client_rpc_timeout_ms 0 -kudu_master_hosts localhost > -mem_limit=12884901888 -logbufsecs=5 -v=1 -max_log_files=0 > -log_filename=impalad_node3 > -log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests > -beeswax_port=21003 -hs2_port=21053 -hs2_http_port=28003 -krpc_port=27003 > -state_store_subscriber_port=23003 -webserver_port=25003 > --enable_orc_scanner=true -admission_control_slots=1 > -executor_groups=default-pool-group2:2 --default_query_options=)> with signal > 9 > -- 2021-03-29 03:45:38,348 INFO MainThread: Getting metric: > cluster-membership.executor-groups.total-healthy from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > -- 2021-03-29 03:45:38,352 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:38,355 INFO MainThread: Waiting for metric value > 'cluster-membership.executor-groups.total-healthy'=1. Current value: 2 > -- 2021-03-29 03:45:38,356 INFO MainThread: Sleeping 1s before next retry. > -- 2021-03-29 03:45:39,357 INFO MainThread: Getting metric: > cluster-membership.executor-groups.total-healthy from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > -- 2021-03-29 03:45:39,360 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:39,363 INFO MainThread: Metric > 'cluster-membership.executor-groups.total-healthy' has reached desired value: > 1 > -- 2021-03-29 03:45:39,367 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:39,372 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:39,379 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:40,386 INFO MainThread: Killing <ImpaladProcess PID: > 14830 > (/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/be/build/latest/service/impalad > -is_coordinator=false -disconnected_session_timeout 21600 > -kudu_client_rpc_timeout_ms 0 -kudu_master_hosts localhost > -mem_limit=12884901888 -logbufsecs=5 -v=1 -max_log_files=0 > -log_filename=impalad_node2 > -log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests > -beeswax_port=21002 -hs2_port=21052 -hs2_http_port=28002 -krpc_port=27002 > -state_store_subscriber_port=23002 -webserver_port=25002 > --enable_orc_scanner=true -admission_control_slots=1 > -executor_groups=default-pool-group2:2 --default_query_options=)> with signal > 9 > -- 2021-03-29 03:45:40,411 INFO MainThread: Getting metric: > cluster-membership.executor-groups.total from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > -- 2021-03-29 03:45:40,414 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:40,417 INFO MainThread: Waiting for metric value > 'cluster-membership.executor-groups.total'=1. Current value: 2 > -- 2021-03-29 03:45:40,417 INFO MainThread: Sleeping 1s before next retry. > -- 2021-03-29 03:45:41,418 INFO MainThread: Getting metric: > cluster-membership.executor-groups.total from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > -- 2021-03-29 03:45:41,421 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:41,424 INFO MainThread: Metric > 'cluster-membership.executor-groups.total' has reached desired value: 1 > -- 2021-03-29 03:45:41,427 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:41,434 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > SET > client_identifier=custom_cluster/test_executor_groups.py::TestExecutorGroups::()::test_executor_group_num_queries_executing_metric; > SET debug_action=QUERY_CANCELLATION_THREAD:SLEEP@10000; > -- executing async: localhost:21000 > select * from functional_parquet.alltypestiny where month < 3 > and id + random() < sleep(500);; > -- 2021-03-29 03:45:41,448 INFO MainThread: Started query > 314f065fbf045208:62949c6e00000000 > -- waiting for completion of the admission control processing of the query: > <tests.common.impala_connection.OperationHandle object at 0x7f870cb4f510> > -- 2021-03-29 03:45:41,503 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:42,501 INFO MainThread: Killing <ImpaladProcess PID: > 14376 > (/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/be/build/latest/service/impalad > -is_coordinator=false -disconnected_session_timeout 21600 > -kudu_client_rpc_timeout_ms 0 -kudu_master_hosts localhost > -mem_limit=12884901888 -logbufsecs=5 -v=1 -max_log_files=0 > -log_filename=impalad_node1 > -log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests > -beeswax_port=21001 -hs2_port=21051 -hs2_http_port=28001 -krpc_port=27001 > -state_store_subscriber_port=23001 -webserver_port=25001 > --enable_orc_scanner=true -admission_control_slots=1 > -executor_groups=default-pool-group1:1 --default_query_options=)> with signal > 9 > -- 2021-03-29 03:45:42,523 INFO MainThread: Getting metric: > cluster-membership.executor-groups.total from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > -- 2021-03-29 03:45:42,526 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:42,530 INFO MainThread: Waiting for metric value > 'cluster-membership.executor-groups.total'=0. Current value: 1 > -- 2021-03-29 03:45:42,530 INFO MainThread: Sleeping 1s before next retry. > -- 2021-03-29 03:45:43,531 INFO MainThread: Getting metric: > cluster-membership.executor-groups.total from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > -- 2021-03-29 03:45:43,533 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:43,537 INFO MainThread: Metric > 'cluster-membership.executor-groups.total' has reached desired value: 0 > -- 2021-03-29 03:45:43,539 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:43,543 INFO MainThread: Starting Impalad process with > args: ['-is_coordinator=false', '-disconnected_session_timeout', '21600', > '-kudu_client_rpc_timeout_ms', '0', '-kudu_master_hosts', 'localhost', > '-mem_limit=12884901888', '-logbufsecs=5', '-v=1', '-max_log_files=0', > '-log_filename=impalad_node1', > '-log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests', > '-beeswax_port=21001', '-hs2_port=21051', '-hs2_http_port=28001', > '-krpc_port=27001', '-state_store_subscriber_port=23001', > '-webserver_port=25001', '--enable_orc_scanner=true', > '-admission_control_slots=1', '-executor_groups=default-pool-group1:1', > '--default_query_options='] > -- 2021-03-29 03:45:43,561 INFO MainThread: Getting metric: > impala-server.ready from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25001 > -- 2021-03-29 03:45:43,564 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:43,565 INFO MainThread: Debug webpage not yet > available: ('Connection aborted.', error(111, 'Connection refused')) > Turning perftools heap leak checking off > Redirecting stderr to > /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad_node1.ERROR > -- 2021-03-29 03:45:44,569 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:44,569 INFO MainThread: Debug webpage not yet > available: ('Connection aborted.', error(111, 'Connection refused')) > -- 2021-03-29 03:45:45,573 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:45,576 INFO MainThread: Metric 'impala-server.ready' > has reached desired value: True > -- 2021-03-29 03:45:45,576 INFO MainThread: Getting metric: > cluster-membership.executor-groups.total-healthy from > impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000 > -- 2021-03-29 03:45:45,579 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > -- 2021-03-29 03:45:45,582 INFO MainThread: Metric > 'cluster-membership.executor-groups.total-healthy' has reached desired value: > 1 > -- 2021-03-29 03:45:45,585 INFO MainThread: Starting new HTTP connection > (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org