----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45784/#review127310 -----------------------------------------------------------
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java (line 472) <https://reviews.apache.org/r/45784/#comment190571> We need to fix the clusterName mess later - can't pickup hdfs clustername from hive conf addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java (line 220) <https://reviews.apache.org/r/45784/#comment190572> Earlier one was more readable. You can use set methods instead of this long constructor? addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java (line 454) <https://reviews.apache.org/r/45784/#comment190574> Aren't there cases where input/output is local fs, for example load from local path? addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java (line 493) <https://reviews.apache.org/r/45784/#comment190573> This should be part of HiveMetaStoreBridge and should be used in import-hive as well? Because this lineage will be created in import-hive, process name should be just tablename for create table so that its created just once. repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java (line 165) <https://reviews.apache.org/r/45784/#comment190575> Use these in Process type definition. Actually, these should be in AtlasClient? - Shwetha GS On April 5, 2016, 11:58 p.m., Suma Shivaprasad wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45784/ > ----------------------------------------------------------- > > (Updated April 5, 2016, 11:58 p.m.) > > > Review request for atlas. > > > Bugs: ATLAS-527 > https://issues.apache.org/jira/browse/ATLAS-527 > > > Repository: atlas > > > Description > ------- > > Added support to track lineage between HDFS Paths and hive tables in > > a. LOAD( at table, partition level) - input is a HDFS path and output is > table( even though we dont create partition entities, we are still tracking > the lineage at table level for partitions. This could be an issue if there > are large number of partition queries which is not being addressed in this > jira - https://issues.apache.org/jira/browse/ATLAS-619) . refer > https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML > b. IMPORT, EXPORT to and from hdfs paths - Refer > https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML > c. CREATE EXTERNAL TABLE - input is hdfs path and o/p is table > d. ALTER TABLE LOCATION for an external table - input is the new hdfs path > and o/p is the table. > > Also changed the ordering of model registration by sorting them by > modifiedTime to ensure they are registered in correct order > > > Diffs > ----- > > > addons/hdfs-model/src/main/java/org/apache/atlas/fs/model/FSDataModelGenerator.java > 555d565 > > addons/hdfs-model/src/main/scala/org/apache/atlas/fs/model/FSDataModel.scala > c964f73 > addons/hive-bridge/pom.xml e125f18 > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java > 3a802d7 > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java > 68e32ff > addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java > e17afb8 > > addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java > 5665856 > > repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java > 0a04c5f > > repository/src/main/java/org/apache/atlas/services/ReservedTypesRegistrar.java > 430bb6b > > Diff: https://reviews.apache.org/r/45784/diff/ > > > Testing > ------- > > Added tests in HiveHookIT > > > Thanks, > > Suma Shivaprasad > >
