[
https://issues.apache.org/jira/browse/ATLAS-751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-751:
-----------------------------------
Description:
Persisting the qualified Name for an entity has multiple issues
1. In case of soft deletes,consider the following scenario
a. Table A -> insert overwrite -> Table B
If the table A and B are dropped and recreated again and the insert overwrite
query is rerun, then it should create another lineage process since the tables
are different here. Similarly for CTAS etc. However the same process will get
updated due to the current way of storing qualifiedName
2. Storing qualified names inherently leads to a lot of updates during
operations renames for eg: if a table is renames, all its
columns.qualifiedName, sd.qualifiedName etc get updated. If this is done with
partitions, then the updates will definitely take a lot of time.
was:
The qualified Name for an entity should be an expression to solve multiple
usecases
1. In case of soft deletes,consider the following scenario
a. Table A -> insert overwrite -> Table B
If the table A and B are dropped and recreated again and the insert overwrite
query is rerun, then it should create another lineage process since the tables
are different here. Similarly for CTAS etc. However the same process will get
updated due to the current way of storing qualifiedName
2. Storing qualified names inherently leads to a lot of updates during
operations renames for eg: if a table is renames, all its
columns.qualifiedName, sd.qualifiedName etc get updated. If this is done with
partitions, then the updates will definitely take a lot of time.
> Add support for primary key constraint on class types
> -----------------------------------------------------
>
> Key: ATLAS-751
> URL: https://issues.apache.org/jira/browse/ATLAS-751
> Project: Atlas
> Issue Type: Improvement
> Affects Versions: 0.7-incubating
> Reporter: Suma Shivaprasad
> Fix For: 0.7-incubating
>
>
> Persisting the qualified Name for an entity has multiple issues
> 1. In case of soft deletes,consider the following scenario
> a. Table A -> insert overwrite -> Table B
> If the table A and B are dropped and recreated again and the insert
> overwrite query is rerun, then it should create another lineage process since
> the tables are different here. Similarly for CTAS etc. However the same
> process will get updated due to the current way of storing qualifiedName
> 2. Storing qualified names inherently leads to a lot of updates during
> operations renames for eg: if a table is renames, all its
> columns.qualifiedName, sd.qualifiedName etc get updated. If this is done with
> partitions, then the updates will definitely take a lot of time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
