[
https://issues.apache.org/jira/browse/IMPALA-9262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar resolved IMPALA-9262.
----------------------------------
Fix Version/s: Impala 3.4.0
Resolution: Fixed
> Blacklist test test_kill_impalad_with_running_queries fails in exhaustive mode
> ------------------------------------------------------------------------------
>
> Key: IMPALA-9262
> URL: https://issues.apache.org/jira/browse/IMPALA-9262
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
> Affects Versions: Impala 3.4.0
> Reporter: Laszlo Gaal
> Assignee: Sahil Takiar
> Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.4.0
>
>
> Test proc test_kill_impalad_with_running_queries in test_blacklist.py failed
> in multiple builds in executive mode with identical symptoms:
> Backtrace is
> {code}
> custom_cluster/test_blacklist.py:145: in
> test_kill_impalad_with_running_queries
> assert "TransmitData() to " in str(e) or "EndDataStream() to " in str(e)
> E assert ('TransmitData() to ' in 'Query was expected to fail\nassert
> False' or 'EndDataStream() to ' in 'Query was expected to fail\nassert False')
> E + where 'Query was expected to fail\nassert False' =
> str(AssertionError(u'Query was expected to fail\nassert False',))
> E + and 'Query was expected to fail\nassert False' =
> str(AssertionError(u'Query was expected to fail\nassert False',))
> {code}
> Standard error is:
> {code}
> 02:19:19 MainThread: Found 0 impalad/0 statestored/0 catalogd process(es)
> 02:19:19 MainThread: Starting State Store logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/custom_cluster_tests/statestored.INFO
> 02:19:19 MainThread: Starting Catalog Service logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
> 02:19:20 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad.INFO
> 02:19:20 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
> 02:19:20 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
> 02:19:23 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 02:19:23 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 02:19:23 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-1066.vpc.cloudera.com:25000
> 02:19:23 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 02:19:24 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 02:19:24 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-1066.vpc.cloudera.com:25000
> 02:19:24 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 02:19:25 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 02:19:25 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-1066.vpc.cloudera.com:25000
> 02:19:25 MainThread: num_known_live_backends has reached value: 3
> 02:19:25 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 02:19:25 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-1066.vpc.cloudera.com:25001
> 02:19:25 MainThread: num_known_live_backends has reached value: 3
> 02:19:25 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 02:19:25 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-1066.vpc.cloudera.com:25002
> 02:19:25 MainThread: num_known_live_backends has reached value: 3
> 02:19:25 MainThread: Impala Cluster Running with 3 nodes (1 coordinators, 2
> executors).
> DEBUG:impala_cluster:Found 3 impalad/1 statestored/1 catalogd process(es)
> INFO:impala_service:Getting metric: statestore.live-backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-1066.vpc.cloudera.com:25010
> INFO:impala_service:Metric 'statestore.live-backends' has reached desired
> value: 4
> DEBUG:impala_service:Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-1066.vpc.cloudera.com:25000
> INFO:impala_service:num_known_live_backends has reached value: 3
> DEBUG:impala_service:Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-1066.vpc.cloudera.com:25001
> INFO:impala_service:num_known_live_backends has reached value: 3
> DEBUG:impala_service:Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-1066.vpc.cloudera.com:25002
> INFO:impala_service:num_known_live_backends has reached value: 3
> ERROR:impala.hiveserver2:Failed to open transport (tries_left=3)
> Traceback (most recent call last):
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/hiveserver2.py",
> line 1009, in _execute
> return func(request)
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py",
> line 176, in OpenSession
> return self.recv_OpenSession()
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py",
> line 188, in recv_OpenSession
> (fname, mtype, rseqid) = iprot.readMessageBegin()
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
> line 126, in readMessageBegin
> sz = self.readI32()
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
> line 206, in readI32
> buff = self.trans.readAll(4)
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TTransport.py",
> line 58, in readAll
> chunk = self.read(sz - have)
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TTransport.py",
> line 159, in read
> self.__rbuf = StringIO(self.__trans.read(max(sz, self.__rbuf_size)))
> File
> "/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/thrift-0.9.3-p7/python/lib64/python2.7/site-packages/thrift/transport/TSocket.py",
> line 120, in read
> message='TSocket read 0 bytes')
> TTransportException: TSocket read 0 bytes
> INFO:impala_cluster:Killing <ImpaladProcess PID: 23167
> (/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/build/latest/service/impalad
> -is_coordinator=false -disconnected_session_timeout 3600
> -kudu_client_rpc_timeout_ms 0 -kudu_master_hosts localhost
> -mem_limit=12884901888 -logbufsecs=5 -v=1 -max_log_files=0
> -log_filename=impalad_node2
> -log_dir=/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/custom_cluster_tests
> -beeswax_port=21002 -hs2_port=21052 -hs2_http_port=28002 -be_port=22002
> -krpc_port=27002 -state_store_subscriber_port=23002 -webserver_port=25002
> --default_query_options=)> with signal 9
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
