----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70605/#review215117 -----------------------------------------------------------
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java Lines 154 (patched) <https://reviews.apache.org/r/70605/#comment301583> Since 'renamedTableEntity' was included in the previous notification (line #148), it is not necessary to include the same entity as a referredEntity. Please see more comments in getHiveDDLEntity(). Consider the following optimization: AtlasEntity ddlEntity = getHiveDDLEntity(renamedTableEntity.getEntity(), true); if (ddlEntity != null) { ret.add(new EntityCreateRequestV2(getUserName(), new AtlasEntitiesWithExtInfo(ddlEntity); } Please review other such occurrence of entity-update. addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java Lines 675 (patched) <https://reviews.apache.org/r/70605/#comment301581> By adding unique-attributes to object-id created here, we can avoid sending 'dbOrTable' as referred entity for 'alter' statements. Consider the following: protected AtlasEntity getHiveDDLEntity(AtlasEntity dbOrTable) { return getHiveDDLEntity(dbOrTable, false); } protected AtlasEntity getHiveDDLEntity(AtlasEntity dbOrTable, boolean excludeEntityGuid) { AtlasObjectId objId = BaseHiveEvent.getObjectId(dbOrTable); if (excludeEntityGuid) { objId.setGuid(null); } final AtlasEntity hiveDDL; if (StringUtils.equals(objId.getTypeName(), HIVE_TYPE_DB)) { hiveDDL = new AtlasEntity(HIVE_DB_DDL, ATTRIBUTE_DB, objId); } else if (StringUtils.equals(objId.getTypeName(), HIVE_TYPE_TABLE)) hiveDDL = new AtlasEntity(HIVE_TB_DDL, ATTRIBUTE_TABLE, objId); } else { hiveDDL = null; } if (hiveDDL != null) { ... } With this change, the notification doesn't need to include 'dbOrTable' entity as a referred entity. addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java Lines 681 (patched) <https://reviews.apache.org/r/70605/#comment301582> handle 'hiveDDL == null' here. addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java Lines 684 (patched) <https://reviews.apache.org/r/70605/#comment301584> Consider setting ATTRIBUTE_QUALIFIED_NAME to same value as ATTRIBUTE_NAME below: hiveDDL.setAttribute(ATTRIBUTE_NAME, getQueryString() + QNAME_SEP_PROCESS + getQueryStartTime().toString()); hiveDDL.setAttribute(ATTRIBUTE_QUALIFIED_NAME, hiveDDL.getAttribute(ATTRIBUTE_NAME); addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java Lines 778 (patched) <https://reviews.apache.org/r/70605/#comment301585> getAttributeQualifiedName() method is not needed, per suggested changes in #685. - Madhan Neethiraj On May 7, 2019, 6:35 p.m., Le Ma wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70605/ > ----------------------------------------------------------- > > (Updated May 7, 2019, 6:35 p.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Madhan > Neethiraj, and Sarath Subramanian. > > > Bugs: ATLAS-3197 > https://issues.apache.org/jira/browse/ATLAS-3197 > > > Repository: atlas > > > Description > ------- > > This patch introduced a model called Hive_table_ddl and hive_db_ddl which are > used to track commands that are executed either on a table entity or bd > entity. DDL entities will be added to a table/db entity through > relationshipAttribute call HIVE_DB_TO_DDL and hive_tb_to_ddl. Every time when > a new command is executed, a new ddl entity will be created, it will append > to the existing DDLQueries field if a table/db has already been created. Once > a table/db is deleted, all these relationship attribute will be deleted as > well. > > > Diffs > ----- > > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java > d3d83490f > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java > de8de63b6 > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateDatabase.java > b01f61f76 > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java > b5ce09332 > addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java > 24de30f5a > addons/models/1000-Hadoop/1030-hive_model.json ee6945a77 > > > Diff: https://reviews.apache.org/r/70605/diff/3/ > > > Testing > ------- > > 1. table : > > CREATE TABLE IF NOT EXISTS testDDL (d int, n string); > > > ALTER TABLE testDDL RENAME TO renameTestDDL; > > ALTER TABLE renameTestDDL CHANGE n changedCol string; > > ALTER TABLE renameTestDDL ADD COLUMNS(addedCol int); > > DROP TABLE renameTestDDL; > > > 2. db > > create database testdb; > > ALTER DATABASE database SET OWNER USER changedOwner; > > DROP DATABSE testdb; > > > File Attachments > ---------------- > > CREATE TABLE IF NOT EXISTS testDDL.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/5047d461-f288-4776-a843-556e45217fef__CREATE_TABLE_IF_NOT_EXISTS_testDDL.png > ALTER TABLE testDDL RENAME TO renameTestDDL; .png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/373a656d-8ab3-4710-b9a1-3eac20adc947__ALTER_TABLE_testDDL_RENAME_TO_renameTestDDL_.png > ALTER TABLE renameTestDDL CHANGE n changedCol string; - ddl.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/c9f57d67-c16b-4142-a6b4-9d372e7fd765__ALTER_TABLE_renameTestDDL_CHANGE_n_changedCol_string_-_ddl.png > ALTER TABLE renameTestDDL CHANGE n changedCol string;.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/42612eb6-83c2-4825-8601-5a1629c9de6f__ALTER_TABLE_renameTestDDL_CHANGE_n_changedCol_string.png > ALTER TABLE renameTestDDL ADD COLUMNS(addedCol int);.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/f5d3522d-17b6-4227-8072-6fc07d88d515__ALTER_TABLE_renameTestDDL_ADD_COLUMNSaddedCol_int.png > ALTER TABLE renameTestDDL ADD COLUMNS.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/73b9046a-2667-4049-9e11-c5c2b375e9ca__ALTER_TABLE_renameTestDDL_ADD_COLUMNS.png > DROP TABLE renameTestDDL;.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/ebfa01f2-e886-4f94-aa75-ff45bb519ca0__DROP_TABLE_renameTestDDL.png > ddl - relationship.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/7b30282b-6419-4791-b107-f938918e1cfd__ddl_-_relationship.png > ALTER DATABASE testdb SET OWNER USER changedOwner;.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/388e7ffa-6691-4c2b-bcd9-d8755dc13e3e__ALTER_DATABASE_testdb_SET_OWNER_USER_changedOwner.png > ALTER DATABASE testdb SET OWNER USER changedOwner; - db entity.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/fd22b6a5-6cb9-40eb-aa10-5389f6f74792__ALTER_DATABASE_testdb_SET_OWNER_USER_changedOwner_-_db_entity.png > drop database testdb;.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/7c889d62-66c8-4c26-bf94-257bd01cdb70__drop_database_testdb.png > drop database testdb;.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/4163043b-c2f1-42ee-9e5c-cfffbc589fb3__drop_database_testdb.png > > > Thanks, > > Le Ma > >
