[jira] [Commented] (ATLAS-247) hive_column leve lineage
[ https://issues.apache.org/jira/browse/ATLAS-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15506095#comment-15506095 ] Vimal Sharma commented on ATLAS-247: Deleted the older patch as it didn't have the newly added file. > hive_column leve lineage > > > Key: ATLAS-247 > URL: https://issues.apache.org/jira/browse/ATLAS-247 > Project: Atlas > Issue Type: New Feature >Affects Versions: 0.5-incubating >Reporter: Herman Yu >Assignee: Harish Butani > Fix For: 0.8-incubating > > Attachments: ATLAS-247-v4.patch, ATLAS-247.2.patch, ATLAS-247.patch > > > hive_column is not inherited from DataSet, thus can't be using hive_process > to track column level lineages > Is there specific reason that hive_column is not inheriting from Data Set? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-247) hive_column leve lineage
[ https://issues.apache.org/jira/browse/ATLAS-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15179486#comment-15179486 ] Shwetha G S commented on ATLAS-247: --- Since the hive patch will be in a future hive release and can't expect all to use the new hive version, we shouldn't fail the hook with old hive releases. We should add catch throwable around adding column level lineage and log and ignore. We should document this as well > hive_column leve lineage > > > Key: ATLAS-247 > URL: https://issues.apache.org/jira/browse/ATLAS-247 > Project: Atlas > Issue Type: New Feature >Affects Versions: 0.5-incubating >Reporter: Herman Yu >Assignee: Harish Butani > Attachments: ATLAS-247.2.patch, ATLAS-247.patch > > > hive_column is not inherited from DataSet, thus can't be using hive_process > to track column level lineages > Is there specific reason that hive_column is not inheriting from Data Set? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-247) hive_column leve lineage
[ https://issues.apache.org/jira/browse/ATLAS-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15179418#comment-15179418 ] Shwetha G S commented on ATLAS-247: --- Can you add asserts on the column level lineage in HiveHookIT? Can you add the documentation in https://github.com/hbutani/incubator-atlas/wiki/Column-Level-Lineage to Bridge-Hive.twiki? Looks like hive publishes snapshot jars. So, we can change hive version to 2.1.0-SNAPSHOT and commit this after hive patch is committed, right? > hive_column leve lineage > > > Key: ATLAS-247 > URL: https://issues.apache.org/jira/browse/ATLAS-247 > Project: Atlas > Issue Type: New Feature >Affects Versions: 0.5-incubating >Reporter: Herman Yu >Assignee: Harish Butani > Attachments: ATLAS-247.2.patch, ATLAS-247.patch > > > hive_column is not inherited from DataSet, thus can't be using hive_process > to track column level lineages > Is there specific reason that hive_column is not inheriting from Data Set? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-247) hive_column leve lineage
[ https://issues.apache.org/jira/browse/ATLAS-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178296#comment-15178296 ] Harish Butani commented on ATLAS-247: - updated patch with fix for case when a column links to input table(s). > hive_column leve lineage > > > Key: ATLAS-247 > URL: https://issues.apache.org/jira/browse/ATLAS-247 > Project: Atlas > Issue Type: New Feature >Affects Versions: 0.5-incubating >Reporter: Herman Yu >Assignee: Harish Butani > Attachments: ATLAS-247.2.patch, ATLAS-247.patch > > > hive_column is not inherited from DataSet, thus can't be using hive_process > to track column level lineages > Is there specific reason that hive_column is not inheriting from Data Set? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-247) hive_column leve lineage
[ https://issues.apache.org/jira/browse/ATLAS-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166929#comment-15166929 ] Shwetha G S commented on ATLAS-247: --- Details: https://github.com/hbutani/incubator-atlas/wiki/Column-Level-Lineage > hive_column leve lineage > > > Key: ATLAS-247 > URL: https://issues.apache.org/jira/browse/ATLAS-247 > Project: Atlas > Issue Type: New Feature >Affects Versions: 0.5-incubating >Reporter: Herman Yu >Assignee: Harish Butani > Attachments: ATLAS-247.patch > > > hive_column is not inherited from DataSet, thus can't be using hive_process > to track column level lineages > Is there specific reason that hive_column is not inheriting from Data Set? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-247) hive_column leve lineage
[ https://issues.apache.org/jira/browse/ATLAS-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14980014#comment-14980014 ] Shwetha G S commented on ATLAS-247: --- [~teeupdata], implementing column level lineage requires hive to provide this lineage information. Once the lineage information is available from hive, we need to re-model such that atlas can provide lineage for both column and table in a generic way. Right now, we are waiting for lineage info from hive > hive_column leve lineage > > > Key: ATLAS-247 > URL: https://issues.apache.org/jira/browse/ATLAS-247 > Project: Atlas > Issue Type: New Feature >Affects Versions: 0.5-incubating >Reporter: Herman Yu > > hive_column is not inherited from DataSet, thus can't be using hive_process > to track column level lineages > Is there specific reason that hive_column is not inheriting from Data Set? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-247) hive_column leve lineage
[ https://issues.apache.org/jira/browse/ATLAS-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971512#comment-14971512 ] Venkatesh Seetharam commented on ATLAS-247: --- [~teeupdata], questions should be discussed on the dev mailing list and not jira's please. Column is not data, a Table represents data but column is part of the schema for the data. There is enough information captured today but not exposed. We will in the future. > hive_column leve lineage > > > Key: ATLAS-247 > URL: https://issues.apache.org/jira/browse/ATLAS-247 > Project: Atlas > Issue Type: New Feature >Affects Versions: 0.5-incubating >Reporter: Herman Yu > > hive_column is not inherited from DataSet, thus can't be using hive_process > to track column level lineages > Is there specific reason that hive_column is not inheriting from Data Set? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
