----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70619/ -----------------------------------------------------------
(Updated May 22, 2019, 5:36 p.m.) Review request for atlas, Fredy Wijaya, Madhan Neethiraj, and Sarath Subramanian. Repository: atlas Description ------- Integrate with Impala hook using Impala private jar with post exec hook changes: 1) handle changed lineage record format change The lineage record format has changed from having seperate table vertex { "id":4, "vertexType":"TABLE", "vertexId":"db_1.table_1", "createTime":1554750000 } To embed the table create time inside column vertex { "id": 3, "vertexType": "COLUMN", "vertexId": "db_1.table_1.tinyint_col", "metadata": { "createTime": 1557860978, "tableName": "db_1.table_1" } } 2) add Impala hook API code to allow Atlas integration code to compile without Impala updates Diffs (updated) ----- addons/impala-bridge-shim/pom.xml PRE-CREATION addons/impala-bridge-shim/src/main/java/org/apache/atlas/impala/hook/ImpalaLineageHook.java PRE-CREATION addons/impala-bridge/pom.xml 8e7e952 addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/ImpalaHook.java PRE-CREATION addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/ImpalaLineageHook.java fdb6748 addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/BaseImpalaEvent.java afe296c addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java e4a38a3 addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/IImpalaLineageHook.java 7c1103a addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/LineageVertex.java 82672c9 addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/LineageVertexMetadata.java PRE-CREATION addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageITBase.java cc62955 addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java 6e4d321 addons/impala-bridge/src/test/resources/impala1.json 8f747f6 addons/impala-bridge/src/test/resources/impala2.json 239797b addons/impala-bridge/src/test/resources/impala3.json 6a7d171 addons/impala-bridge/src/test/resources/impala5.json 854969b addons/impala-bridge/src/test/resources/impala6.json f136180 addons/impala-bridge/src/test/resources/impala7.json f9ee670 addons/impala-bridge/src/test/resources/impalaAlterViewAsSelect.json PRE-CREATION addons/impala-bridge/src/test/resources/impalaCreateTableAsSelect.json PRE-CREATION addons/impala-bridge/src/test/resources/impalaCreateView.json PRE-CREATION addons/impala-bridge/src/test/resources/impalaCreateViewNoCreateTime.json PRE-CREATION addons/impala-bridge/src/test/resources/impalaInsertIntoAsSelect.json PRE-CREATION addons/impala-hook-api/pom.xml PRE-CREATION addons/impala-hook-api/src/main/java/org/apache/impala/hooks/PostQueryHookContext.java PRE-CREATION addons/impala-hook-api/src/main/java/org/apache/impala/hooks/QueryExecHook.java PRE-CREATION pom.xml 0a8f9b7 Diff: https://reviews.apache.org/r/70619/diff/11/ Changes: https://reviews.apache.org/r/70619/diff/10-11/ Testing ------- add integration tests ImpalaLineageToolIT.testCreateViewFromFileWithoutCreateTime to handle the situation that there is no createTime in lineage record change the file content for other integration tests in ImpalaLineageToolIT to handle new lineage record format Thanks, Na Li