----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70619/#review215475 -----------------------------------------------------------
Ship it! Ship It! - Sridhar K On May 22, 2019, 5:36 p.m., Na Li wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70619/ > ----------------------------------------------------------- > > (Updated May 22, 2019, 5:36 p.m.) > > > Review request for atlas, Fredy Wijaya, Madhan Neethiraj, and Sarath > Subramanian. > > > Repository: atlas > > > Description > ------- > > Integrate with Impala hook using Impala private jar with post exec hook > > changes: > > 1) handle changed lineage record format change > The lineage record format has changed from having seperate table vertex > > { > "id":4, > "vertexType":"TABLE", > "vertexId":"db_1.table_1", > "createTime":1554750000 > } > > To embed the table create time inside column vertex > > { > "id": 3, > "vertexType": "COLUMN", > "vertexId": "db_1.table_1.tinyint_col", > "metadata": { > "createTime": 1557860978, > "tableName": "db_1.table_1" > } > } > > 2) add Impala hook API code to allow Atlas integration code to compile > without Impala updates > > > Diffs > ----- > > addons/impala-bridge-shim/pom.xml PRE-CREATION > > addons/impala-bridge-shim/src/main/java/org/apache/atlas/impala/hook/ImpalaLineageHook.java > PRE-CREATION > addons/impala-bridge/pom.xml 8e7e952 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/ImpalaHook.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/ImpalaLineageHook.java > fdb6748 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/BaseImpalaEvent.java > afe296c > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java > e4a38a3 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/IImpalaLineageHook.java > 7c1103a > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/LineageVertex.java > 82672c9 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/LineageVertexMetadata.java > PRE-CREATION > > addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageITBase.java > cc62955 > > addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java > 6e4d321 > addons/impala-bridge/src/test/resources/impala1.json 8f747f6 > addons/impala-bridge/src/test/resources/impala2.json 239797b > addons/impala-bridge/src/test/resources/impala3.json 6a7d171 > addons/impala-bridge/src/test/resources/impala5.json 854969b > addons/impala-bridge/src/test/resources/impala6.json f136180 > addons/impala-bridge/src/test/resources/impala7.json f9ee670 > addons/impala-bridge/src/test/resources/impalaAlterViewAsSelect.json > PRE-CREATION > addons/impala-bridge/src/test/resources/impalaCreateTableAsSelect.json > PRE-CREATION > addons/impala-bridge/src/test/resources/impalaCreateView.json PRE-CREATION > addons/impala-bridge/src/test/resources/impalaCreateViewNoCreateTime.json > PRE-CREATION > addons/impala-bridge/src/test/resources/impalaInsertIntoAsSelect.json > PRE-CREATION > addons/impala-hook-api/pom.xml PRE-CREATION > > addons/impala-hook-api/src/main/java/org/apache/impala/hooks/PostQueryHookContext.java > PRE-CREATION > > addons/impala-hook-api/src/main/java/org/apache/impala/hooks/QueryExecHook.java > PRE-CREATION > pom.xml 0a8f9b7 > > > Diff: https://reviews.apache.org/r/70619/diff/11/ > > > Testing > ------- > > add integration tests > ImpalaLineageToolIT.testCreateViewFromFileWithoutCreateTime to handle the > situation that there is no createTime in lineage record > change the file content for other integration tests in ImpalaLineageToolIT to > handle new lineage record format > > > Thanks, > > Na Li > >
