[
https://issues.apache.org/jira/browse/IMPALA-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502919#comment-17502919
]
ASF subversion and git services commented on IMPALA-10999:
----------------------------------------------------------
Commit 2d6300ac59918d6e8a45a5d4669e252f177d6fcc in impala's branch
refs/heads/master from Qifan Chen
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=2d6300a ]
IMPALA-10999 Flakiness in TestAsyncLoadData.test_async_load
This patch addresses the flakness by allowing RUNNING state to be a
legit exec state returned from execute_query_async_using_client() in
python. This call submits the load query to Impala backend.
The corresponding Impala backend code for the beewax protocol is
ImpalaServer::query() which utilizes a wait thread executing
ClientRequestState::Wait() to set the exec state from RUNNING to
FINISH. Sometime, when this wait thread does not run fast to do so,
the returned state from the main thread is RUNNING.
The fix is purely a modification to the test itself.
Testing:
1. ran core test successfully
Change-Id: Ic2ac954b0494b7413ce0ec405718fcc354dba9e0
Reviewed-on: http://gerrit.cloudera.org:8080/18268
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Flakiness in TestAsyncLoadData.test_async_load
> ----------------------------------------------
>
> Key: IMPALA-10999
> URL: https://issues.apache.org/jira/browse/IMPALA-10999
> Project: IMPALA
> Issue Type: Bug
> Reporter: Bikramjeet Vig
> Assignee: Qifan Chen
> Priority: Major
> Labels: broken-build, flaky-test
>
> This test failed in one of the GVO's recently.
> [Link|https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/15097/testReport/junit/metadata.test_load/TestAsyncLoadData/test_async_load_enable_async_load_data_execution__False___protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/]
>
> {noformat}
> Error Message
> metadata/test_load.py:197: in test_async_load assert(exec_end_state ==
> finished_state) E assert 3 == 4
> Stacktrace
> metadata/test_load.py:197: in test_async_load
> assert(exec_end_state == finished_state)
> E assert 3 == 4
> Standard Error
> SET
> client_identifier=metadata/test_load.py::TestAsyncLoadData::()::test_async_load[enable_async_load_data_execution:False|protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node;
> -- connecting to: localhost:21000
> -- connecting to localhost:21050 with impyla
> -- 2021-10-30 01:38:55,203 INFO MainThread: Closing active operation
> -- connecting to localhost:28000 with impyla
> -- 2021-10-30 01:38:55,237 INFO MainThread: Closing active operation
> SET
> client_identifier=metadata/test_load.py::TestAsyncLoadData::()::test_async_load[enable_async_load_data_execution:False|protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node;
> SET sync_ddl=False;
> -- executing against localhost:21000
> DROP DATABASE IF EXISTS `test_async_load_ff1c20a7` CASCADE;
> -- 2021-10-30 01:38:55,281 INFO MainThread: Started query
> df43a0ff6165a9eb:33b0d69f00000000
> SET
> client_identifier=metadata/test_load.py::TestAsyncLoadData::()::test_async_load[enable_async_load_data_execution:False|protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node;
> SET sync_ddl=False;
> -- executing against localhost:21000
> CREATE DATABASE `test_async_load_ff1c20a7`;
> -- 2021-10-30 01:39:01,148 INFO MainThread: Started query
> e64bd28a97339b44:e76523a800000000
> -- 2021-10-30 01:39:01,253 INFO MainThread: Created database
> "test_async_load_ff1c20a7" for test ID
> "metadata/test_load.py::TestAsyncLoadData::()::test_async_load[enable_async_load_data_execution:
> False | protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]"
> SET
> client_identifier=metadata/test_load.py::TestAsyncLoadData::()::test_async_load[enable_async_load_data_execution:False|protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node;
> -- connecting to: localhost:21000
> -- executing against localhost:21000
> create table test_async_load_ff1c20a7.test_load_nopart_beeswax_False like
> functional.alltypesnopart location
> '/test-warehouse/test_load_staging_beeswax_False';
> -- 2021-10-30 01:39:09,435 INFO MainThread: Started query
> e543635533874c9e:fe238ca900000000
> -- executing against localhost:21000
> select count(*) from test_async_load_ff1c20a7.test_load_nopart_beeswax_False;
> -- 2021-10-30 01:39:13,178 INFO MainThread: Started query
> 5c4969e81b1b614b:26754a2200000000
> SET
> client_identifier=metadata/test_load.py::TestAsyncLoadData::()::test_async_load[enable_async_load_data_execution:False|protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node;
> -- executing against localhost:21000
> use functional;
> -- 2021-10-30 01:39:13,413 INFO MainThread: Started query
> d340e3650cba2d6f:a35a14bb00000000
> SET
> client_identifier=metadata/test_load.py::TestAsyncLoadData::()::test_async_load[enable_async_load_data_execution:False|protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET enable_async_load_data_execution=False;
> SET debug_action=CRS_DELAY_BEFORE_LOAD_DATA:SLEEP@3000;
> SET exec_single_node_rows_threshold=0;
> -- executing async: localhost:21000
> load data inpath '/test-warehouse/test_load_staging_beeswax_False'
> into table test_async_load_ff1c20a7.test_load_nopart_beeswax_False;
> -- 2021-10-30 01:39:16,472 INFO MainThread: Started query
> ac404aa17515170d:cbd45efd00000000
> -- getting state for operation:
> <tests.common.impala_connection.OperationHandle object at 0x7f9426833990>
> -- getting state for operation:
> <tests.common.impala_connection.OperationHandle object at 0x7f9426833990>
> -- getting state for operation:
> <tests.common.impala_connection.OperationHandle object at 0x7f9426833990>
> -- getting state for operation:
> <tests.common.impala_connection.OperationHandle object at 0x7f9426833990>
> -- closing query for operation handle:
> <tests.common.impala_connection.OperationHandle object at 0x7f9426833990>
> -- closing connection to: localhost:21000
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
