Hello Riza Suminto, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21437
to look at the new patch set (#8).
Change subject: IMPALA-12277: Fix NullPointerException for partitioned inserts
when EP is turned off
......................................................................
IMPALA-12277: Fix NullPointerException for partitioned inserts when
EP is turned off
When event processor is turned off, inserting values into partitioned
table can lead to NullPointerException if the partition is deleted
outside impala (eg: HMS). Since event processor is turned off, impala
is unaware of the metadata changes to the table.
Currently in impala, we always reuse the metadata when reloading a
table. This can lead to data inconsistency issue especially in the case
of event processor being turned off. This patch address this issue by
reusing metadata only when event processor state is active. If it is
not, we should always fetch the latest metadata from HMS.
The issue can be seen with the following steps:
- Turn off the event processor
- create a partitioned table and add a partition from impala
- drop the same partition from hive
- from impala, insert values into the partition (expectation is that
if the partition didn't exist, it will create a new one).
Testing:
- Verified manually that NullPointerException is avoided with this patch
- Added end-to-end tests to verify the above scenario for external and
manged tables.
Change-Id: Ide8f1f6bf017e9a040b53bb5d5291ff2ea3e0d18
---
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/util/AcidUtils.java
M tests/custom_cluster/test_events_custom_configs.py
3 files changed, 32 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/21437/8
--
To view, visit http://gerrit.cloudera.org:8080/21437
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ide8f1f6bf017e9a040b53bb5d5291ff2ea3e0d18
Gerrit-Change-Number: 21437
Gerrit-PatchSet: 8
Gerrit-Owner: Sai Hemanth Gantasala <saihema...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <saihema...@cloudera.com>
