[
https://issues.apache.org/jira/browse/IMPALA-9057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121417#comment-17121417
]
Vihang Karajgaonkar commented on IMPALA-9057:
---------------------------------------------
I looked into this and I think I found the problem. In case of transactional
tables, the events are generated in a transactional context. So for example, in
case of an insert into a transactional table, we see a sequence of OPEN_TXN,
ALLOCATE_WRITE_ID, ALTER_PARTITION, COMMIT_TXN event on that table. Currently
EventsProcessor refreshes the table when we see ALTER_PARTITION however, it is
possible that the transaction has not been committed by that time. Hence we
will skip the new files which were added by the insert after the reload. The
solution is to wait until we see a COMMIT_TXN event before refreshing the
tables. I will send out a fix for this soon.
> TestEventProcessing.test_insert_events_transactional is flaky
> -------------------------------------------------------------
>
> Key: IMPALA-9057
> URL: https://issues.apache.org/jira/browse/IMPALA-9057
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Alice Fan
> Assignee: Vihang Karajgaonkar
> Priority: Blocker
> Labels: build-failure, flaky
>
> Assertion failure for
> custom_cluster.test_event_processing.TestEventProcessing.test_insert_events_transactional
>
> {code:java}
> Error Message
> assert ['101', 'x', ..., '3', '2019'] == ['101', 'z', '28', '3', '2019'] At
> index 1 diff: 'x' != 'z' Full diff: - ['101', 'x', '28', '3', '2019'] ?
> ^ + ['101', 'z', '28', '3', '2019'] ? ^
> Stacktrace
> custom_cluster/test_event_processing.py:49: in
> test_insert_events_transactional
> self.run_test_insert_events(is_transactional=True)
> custom_cluster/test_event_processing.py:131: in run_test_insert_events
> assert data.split('\t') == ['101', 'z', '28', '3', '2019']
> E assert ['101', 'x', ..., '3', '2019'] == ['101', 'z', '28', '3', '2019']
> E At index 1 diff: 'x' != 'z'
> E Full diff:
> E - ['101', 'x', '28', '3', '2019']
> E ? ^
> E + ['101', 'z', '28', '3', '2019']
> E ? ^
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
