-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72756/
-----------------------------------------------------------
(Updated Aug. 11, 2020, 3:16 p.m.)
Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon
Rodrigues, and Sarath Subramanian.
Bugs: ATLAS-3918
https://issues.apache.org/jira/browse/ATLAS-3918
Repository: atlas
Description (updated)
-------
Issue : Messages are being sent for DDL operations one from HMS and
HiveServer2(hive_on_tez) which causes duplicate messages hence it throws
exception.
message
{"version":{"version":"1.0.0","versionParts":[1]},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1,"msgCreationTime":1594802485264,"message":{"type":"ENTITY_PARTIAL_UPDATE_V2","user":"hrt_qa","entityId":{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"default.t1@cm"}},"entity":{"entity":{"typeName":"hive_table","attributes":{"owner":"hrt_qa","tableType":"MANAGED_TABLE","temporary":false,"lastAccessTime":1594802469000,"aliases":["t1"],"createTime":1594802469000,"qualifiedName":"default.t2@cm","name":"t2","comment":null,"parameters":{"last_modified_time":"1594802481","rawDataSize":"0","numRows":"0","transient_lastDdlTime":"1594802481","bucketing_version":"2","last_modified_by":"hrt_qa","numFilesErasureCoded":"0","totalSize":"0","transactional_properties":"default","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"id\":\"true\"}}","numFiles":"0","transactional":"true"},"retention":0},"guid":"-21782793831367970","isIncomplete":false
,"provenanceType":0,"version":0,"proxy":false}}}}
This patch fix it by providing a property in configuration when enabled the
HiveServer2 will send ddl messages and if disabled then it will only send non
ddl messages.
Diffs
-----
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java
651323490
Diff: https://reviews.apache.org/r/72756/diff/3/
Testing (updated)
-------
Tested by creating and renaming the hive_table and it doesn't throw above error
message when the property is set to false :
> create table t1(id int);
> alter table t1 rename to t2;
Thanks,
chaitali
