[
https://issues.apache.org/jira/browse/IMPALA-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752395#comment-16752395
]
ASF subversion and git services commented on IMPALA-8064:
---------------------------------------------------------
Commit e5af7f92926f94fd4c91a8c30089de142ea5f3ac in impala's branch
refs/heads/master from Janaki Lahorani
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e5af7f9 ]
IMPALA-8064: Fix intermittent test failures from test_min_max_filters
test_min_max_filters and test_decimal_min_max_filters records the aggregated
probe rows to
check whether min-max filter was exercised. In the case of ASAN builds, the
probe side
started processing before the filters reached the probe side, because ASAN
builds are a
little slower. The resolution was to increase RUNTIME_FILTER_WAIT_TIME_MS to
accommodate ASAN.
After this patch was merged, https://gerrit.cloudera.org/#/c/12224/ the issue
is still seen.
After further analysis, it was found from the profile that though
RUNTIME_FILTER_WAIT_TIME_MS
was set to 100000, the probe side waited for much less than 100000.
-- Begin Profile Snippet where probe side waited for 36768 ms instead of 100000
Instance 2b4a14d9933eb108:7565b86500000005
(host=downstream-jenkins-host:22002):(Total: 1m42s, non-child: 0.000ns, %
non-child: 0.00%)
HASH_JOIN_NODE (id=2):(Total: 39s251ms, non-child: 6.999ms, %
non-child: 0.01%)
Runtime filters: 1 of 1 Runtime Filter Published
KUDU_SCAN_NODE (id=0):(Total: 36s812ms, non-child: 36s812ms, %
non-child: 100.00%)
Runtime filters: Not all filters arrived (arrived: [], missing [1]),
waited for 36s768ms
-- End Profile Snippet
After analyzing the code we found that in ScanNode::WaitForRuntimeFilters,
int32 was used for time.
But it should be using int64. This is something that needs to be fixed. There
is some probability
that using 32 bit caused an overflow resulting in the actual wait time shown in
profile to be less
than 100000.
This patch fixes the time variables in ScanNode::WaitForRuntimeFilters to int64.
Change-Id: I7d560adfb098be455a5a89c6de76ac1f17d1d3e3
Reviewed-on: http://gerrit.cloudera.org:8080/12269
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> test_min_max_filters is flaky
> ------------------------------
>
> Key: IMPALA-8064
> URL: https://issues.apache.org/jira/browse/IMPALA-8064
> Project: IMPALA
> Issue Type: Bug
> Reporter: Pooja Nilangekar
> Assignee: Pooja Nilangekar
> Priority: Blocker
> Labels: broken-build, flaky-test
> Attachments: profile.txt
>
>
> The following configuration of the test_min_max_filters:
> {code:java}
> query_test.test_runtime_filters.TestMinMaxFilters.test_min_max_filters[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
> 0} | table_format: kudu/none]{code}
> It produces a higher aggregation of sum over the proberows than expected:
> {code:java}
> query_test/test_runtime_filters.py:113: in test_min_max_filters
> self.run_test_case('QueryTest/min_max_filters', vector)
> common/impala_test_suite.py:518: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:612: in verify_runtime_profile %
> (function, field, expected_value, actual_value, actual))
> E AssertionError: Aggregation of SUM over ProbeRows did not match expected
> results.
> E EXPECTED VALUE: E 619
> E ACTUAL VALUE: E 652
> {code}
> This test was introduced in the patch for IMPALA-6533. The failure occurred
> during an ASAN build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
