[
https://issues.apache.org/jira/browse/ATLAS-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16632816#comment-16632816
]
aflyary commented on ATLAS-2893:
--------------------------------
Under the Atlas hive-bridge module, i modified the CreateHiveProcess
Class, and added the method to get the column lineage from the hive
lineageState of SessionState, it could work with the pre-released hive version
before hive-1.2.1.
I test the code with the hive version 1.2.0 and 0.12.0, it could get the
hive column lineage correctly.
The hive sql which i test are below:
create table table1(id int, name string, age int, address string) ROW
FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
create table table2 as select id,name,age,address from table1;
> Atlas Column Lineage of Hive Hook to support the hive old version (before
> hive version 1.2.1 after the patch for HIVE-13112)
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: ATLAS-2893
> URL: https://issues.apache.org/jira/browse/ATLAS-2893
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-intg
> Affects Versions: 1.0.0
> Reporter: aflyary
> Priority: Major
> Fix For: 1.1.0
>
> Attachments: ATLAS-2893.001.patch
>
>
> Right now atlas hook for apache hive of column lineage is limited,
> only "Column level lineage works with Hive version 1.2.1 after the patch for
> HIVE-13112 is applied to Hive source" could support the hive column lineage.
> In some product env, hive version still very old, but we should also
> support the hive column lineage.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
