[
https://issues.apache.org/jira/browse/IMPALA-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616719#comment-17616719
]
ASF subversion and git services commented on IMPALA-11160:
----------------------------------------------------------
Commit b92a8ad745e2499103a55aeb5e58aa090aee05e6 in impala's branch
refs/heads/branch-4.1.1 from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b92a8ad74 ]
IMPALA-11160: Ignore stale ALTER_PARTITION events on transactional tables
When applying ALTER_PARTITION events on transactional tables, we refresh
the partition using the metadata in events if
hms_event_incremental_refresh_transactional_table is enabled (which is
the default). This could be wrong if the ALTER_PARTITION event is stale.
The partition metadata will be rolled back to a stale state.
This patch compares the eventId with the createEventId of the table and
ignores those ALTER_PARTITION events that have older (smaller) event
ids. Note that we already do this for many other event types,
ALTER_PARTITION is somehow missing the checks.
Eventually we should depend on the lastSyncedEventId and replace
createEventId with it. The self-event detection can also be replaced
since self-events are also stale events. These will be addressed in
IMPALA-10976.
Tests
- Verified locally with local-catalog mode and event-processor enabled
and iterated test_acid_compute_stats for 1400 times. Without the fix,
the test would fail in tens of runs.
Change-Id: I5bb8cfc213093f3bbd0359c7084b277a3bd5264a
Reviewed-on: http://gerrit.cloudera.org:8080/19020
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Reviewed-on: http://gerrit.cloudera.org:8080/19126
Reviewed-by: Csaba Ringhofer <csringho...@cloudera.com>
Tested-by: Quanlong Huang <huangquanl...@gmail.com>
> TestAcid.test_acid_compute_stats failed in ubuntu-16.04-dockerised-tests
> ------------------------------------------------------------------------
>
> Key: IMPALA-11160
> URL: https://issues.apache.org/jira/browse/IMPALA-11160
> Project: IMPALA
> Issue Type: Bug
> Reporter: Zoltán Borók-Nagy
> Assignee: Quanlong Huang
> Priority: Critical
> Labels: broken-build
> Fix For: Impala 4.2.0
>
>
> h3. Error Message
> {noformat}
> query_test/test_acid.py:220: in test_acid_compute_stats
> self.run_test_case('QueryTest/acid-compute-stats', vector,
> use_db=unique_database) common/impala_test_suite.py:718: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:554: in __verify_results_and_errors
> replace_filenames_with_placeholder) common/test_result_verifier.py:469: in
> verify_raw_results VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results E assert Comparing
> QueryTestResults (expected vs actual): E '1',1,1,'2B','NOT CACHED','NOT
> CACHED',regex:.*,'true',regex:.* != '1',1,1,'2B','NOT CACHED','NOT
> CACHED','TEXT','false','hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compute_stats_69ccf940.db/pt/p=1'
> E 'Total',1,1,'2B','0B','','','','' == 'Total',1,1,'2B','0B','','','',''
> {noformat}
> h3. Stacktrace
> {noformat}
> query_test/test_acid.py:220: in test_acid_compute_stats
> self.run_test_case('QueryTest/acid-compute-stats', vector,
> use_db=unique_database)
> common/impala_test_suite.py:718: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:554: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:469: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E '1',1,1,'2B','NOT CACHED','NOT CACHED',regex:.*,'true',regex:.* !=
> '1',1,1,'2B','NOT CACHED','NOT
> CACHED','TEXT','false','hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compute_stats_69ccf940.db/pt/p=1'
> E 'Total',1,1,'2B','0B','','','','' == 'Total',1,1,'2B','0B','','','',''
> {noformat}
> It happend in
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5359/
> *Please click on "Don't keep this build forever" once this issue is resolved*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
