[
https://issues.apache.org/jira/browse/IMPALA-9889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17144245#comment-17144245
]
Wenzhe Zhou edited comment on IMPALA-9889 at 6/24/20, 10:19 PM:
----------------------------------------------------------------
This is similar to IMPALA-8064: (test_min_max_filters is flaky) due to runtime
filters not arriving in time.
There is only one query in min_max_filters.test with expected probe rows as
619. That is following query:
select STRAIGHT_JOIN count (* ) from decimal_rtf_tbl a join [BROADCAST]
decimal_rtf_tiny_tbl b
where a.d28_28 = b.d28_28 and b.d28_28 != 0;
All_runtime_filters.test also has this query with expected probe rows as 37.
When applied min_max filter on target, the aggregated probe rows equals 619.
When applied bloom filter or both bloom and min-max filters on target, the
aggregated probe rows equals 37. When don't apply runtime filter on target
(for example, set runtime_filter_mode as off), the aggregated probe rows equals
718.
Since the actual probe rows equals 718 in both cases, it seems that the runtime
filters were not applied on the target during the test when the issues
happened. Also the issue happened randomly, it maybe caused by some timing
issue. Most likely the Kudu scanner on the executer did not receive the runtime
filters when executing the query, hence filter were not apply to the Kudu scan.
Maybe we should increase RUNTIME_FILTER_WAIT_TIME_MS value for the query in the
test.
was (Author: wzhou):
It's similar as IMPALA-8064: test_min_max_filters is flaky.
There is only one query in min_max_filters.test with expected probe rows as
619. That is following query:
select STRAIGHT_JOIN count (* ) from decimal_rtf_tbl a join [BROADCAST]
decimal_rtf_tiny_tbl b
where a.d28_28 = b.d28_28 and b.d28_28 != 0;
All_runtime_filters.test also has this query with expected probe rows as 37.
When applied min_max filter on target, the aggregated probe rows equals 619.
When applied bloom filter or both bloom and min-max filters on target, the
aggregated probe rows equals 37. When don't apply runtime filter on target
(for example, set runtime_filter_mode as off), the aggregated probe rows equals
718.
Since the actual probe rows equals 718 in both cases, it seems that the runtime
filters were not applied on the target during the test when the issues
happened. Also the issue happened randomly, it maybe caused by some timing
issue. Most likely the Kudu scanner on the executer did not receive the runtime
filters when executing the query, hence filter were not apply to the Kudu scan.
Maybe we should increase RUNTIME_FILTER_WAIT_TIME_MS value for the query in the
test.
> test_runtime_filters flaky on Kudu table format
> -----------------------------------------------
>
> Key: IMPALA-9889
> URL: https://issues.apache.org/jira/browse/IMPALA-9889
> Project: IMPALA
> Issue Type: Bug
> Affects Versions: Impala 4.0
> Reporter: Vihang Karajgaonkar
> Assignee: Wenzhe Zhou
> Priority: Blocker
> Labels: broken-build, flaky-test
>
> Couple of tests in test_runtime_filters test fail on Kudu table formats
> randomly with the stack trace below:
> {noformat}
> query_test/test_runtime_filters.py:208: in test_min_max_filters
> test_file_vars={'$RUNTIME_FILTER_WAIT_TIME_MS': str(WAIT_TIME_MS)})
> common/impala_test_suite.py:718: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:627: in verify_runtime_profile
> % (function, field, expected_value, actual_value, actual))
> E AssertionError: Aggregation of SUM over ProbeRows did not match expected
> results.
> E EXPECTED VALUE:
> E 619
> E
> E ACTUAL VALUE:
> E 718
> {noformat}
> {noformat}
> query_test/test_runtime_filters.py:277: in test_all_runtime_filters
> test_file_vars={'$RUNTIME_FILTER_WAIT_TIME_MS': str(WAIT_TIME_MS)})
> common/impala_test_suite.py:718: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:627: in verify_runtime_profile
> % (function, field, expected_value, actual_value, actual))
> E AssertionError: Aggregation of SUM over ProbeRows did not match expected
> results.
> E EXPECTED VALUE:
> E 37
> E
> E ACTUAL VALUE:
> E 718
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
