[
https://issues.apache.org/jira/browse/IMPALA-8957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937180#comment-16937180
]
ASF subversion and git services commented on IMPALA-8957:
---------------------------------------------------------
Commit 750f65912046d39a82adbc4907ea8bc673c18964 in impala's branch
refs/heads/master from Sahil Takiar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=750f659 ]
IMPALA-8926, IMPALA-8957: Fix result spooling flaky tests
TestResultSpooling::_test_full_queue was flaky because there was a race
condition in the test where the result spooling queue would not fill up
quickly enough. The original way around this was to sleep for a fixed
amount of time in hope that the queue would fill up by the time the
thread woke up. The new approach periodically searches the runtime
profile for specific patterns that indicate the queue is full.
TestFetchAndSpooling.test_rows_sent_counters was flaky because the
RowsSentRate can be 0 if the results are spooled fast enough (because
the time spent spooling results is 0). The fix is to use the DEBUG_ACTION
BPRS_BEFORE_ADD_BATCH to introduce a delay when spooling results, so that
the RowsSentRate is guaranteed to be non-zero.
TestFetch.test_rows_sent_counters was flaky because ClientFetchWaitTimer
can be 0 if the Coordinator does not end up waiting any time for results
to be fetched. The fix is to wait until the query has 'FINISHED'
(results are available to fetch) and then sleep so that the
ClientFetchWaitTimer is a non-zero value.
Cleaned up a few other tests as well.
Testing:
* Looped both tests for a few hours without failure
Change-Id: I3042f592bc79785e43ebc7b09ac1270eae8ed66f
Reviewed-on: http://gerrit.cloudera.org:8080/14275
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> TestFetchAndSpooling.test_rows_sent_counters is flaky
> -----------------------------------------------------
>
> Key: IMPALA-8957
> URL: https://issues.apache.org/jira/browse/IMPALA-8957
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
>
> Error Details
> {noformat}
> query_test/test_fetch.py:77: in test_rows_sent_counters assert
> re.search("RowsSentRate: [1-9]", result.runtime_profile) E assert None E +
> where None = <function search at 0xb14488>('RowsSentRate: [1-9]', 'Query
> (id=3946b19649af9ce3:7f38be6700000000):\n DEBUG MODE WARNING: Query profile
> created while running a DEBUG buil... - OptimizationTime: 59.000ms\n -
> PeakMemoryUsage: 213.50 KB (218624)\n - PrepareTime: 26.000ms\n') E + where
> <function search at 0xb14488> = re.search E + and 'Query
> (id=3946b19649af9ce3:7f38be6700000000):\n DEBUG MODE WARNING: Query profile
> created while running a DEBUG buil... - OptimizationTime: 59.000ms\n -
> PeakMemoryUsage: 213.50 KB (218624)\n - PrepareTime: 26.000ms\n' =
> <tests.beeswax.impala_beeswax.ImpalaBeeswaxResult object at
> 0xbbfa5d0>.runtime_profile{noformat}
> Stack Trace
> {noformat}
> query_test/test_fetch.py:77: in test_rows_sent_counters
> assert re.search("RowsSentRate: [1-9]", result.runtime_profile)
> E assert None
> E + where None = <function search at 0xb14488>('RowsSentRate: [1-9]',
> 'Query (id=3946b19649af9ce3:7f38be6700000000):\n DEBUG MODE WARNING: Query
> profile created while running a DEBUG buil... - OptimizationTime: 59.000ms\n
> - PeakMemoryUsage: 213.50 KB (218624)\n - PrepareTime:
> 26.000ms\n')
> E + where <function search at 0xb14488> = re.search
> E + and 'Query (id=3946b19649af9ce3:7f38be6700000000):\n DEBUG MODE
> WARNING: Query profile created while running a DEBUG buil... -
> OptimizationTime: 59.000ms\n - PeakMemoryUsage: 213.50 KB
> (218624)\n - PrepareTime: 26.000ms\n' =
> <tests.beeswax.impala_beeswax.ImpalaBeeswaxResult object at
> 0xbbfa5d0>.runtime_profile{noformat}
> Standard Error
> {noformat}
> SET
> client_identifier=query_test/test_fetch.py::TestFetchAndSpooling::()::test_rows_sent_counters[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|table;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select id from functional.alltypes limit 10;
> -- 2019-09-18 18:51:20,759 INFO MainThread: Started query
> 3946b19649af9ce3:7f38be6700000000{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
