[jira] [Updated] (ATLAS-918) UI : Deleted entities show status as DELETED and Datasets should be differentiable in lineage
[ https://issues.apache.org/jira/browse/ATLAS-918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-918: --- Summary: UI : Deleted entities show status as DELETED and Datasets should be differentiable in lineage (was: UI : Deleted entities show status as DELETED and should be differentiable in lineage) > UI : Deleted entities show status as DELETED and Datasets should be > differentiable in lineage > - > > Key: ATLAS-918 > URL: https://issues.apache.org/jira/browse/ATLAS-918 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > > Keval, > This jira is to track showing deleted entities with a diff color in lineage. > Is there anything missing in the lineage API to implement this? We need to > take care of this since no none looks at lineage via a process and will be > mostly looking at lineage from a table > Also can we show the property (status)__status for deleted entities as > "DELETED" . We could display that in "properties" section. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-918) UI : Deleted entities show status as DELETED and Datasets should be differentiable in lineage
[ https://issues.apache.org/jira/browse/ATLAS-918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-918: --- Description: Keval, This jira is to track showing deleted entities with a diff color in lineage. Is there anything missing in the lineage API to implement this? We need to take care of this since no one looks at lineage via a process and will be mostly looking at lineage from a table or a dataset. Also a table can be dropped and recreated in hive, leading to linage showing as attached in screen shot Also can we show the property (status)__status for deleted entities as "DELETED" . We could display that in "properties" section. was: Keval, This jira is to track showing deleted entities with a diff color in lineage. Is there anything missing in the lineage API to implement this? We need to take care of this since no none looks at lineage via a process and will be mostly looking at lineage from a table Also can we show the property (status)__status for deleted entities as "DELETED" . We could display that in "properties" section. > UI : Deleted entities show status as DELETED and Datasets should be > differentiable in lineage > - > > Key: ATLAS-918 > URL: https://issues.apache.org/jira/browse/ATLAS-918 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > > Keval, > This jira is to track showing deleted entities with a diff color in lineage. > Is there anything missing in the lineage API to implement this? We need to > take care of this since no one looks at lineage via a process and will be > mostly looking at lineage from a table or a dataset. Also a table can be > dropped and recreated in hive, leading to linage showing as attached in > screen shot > Also can we show the property (status)__status for deleted entities as > "DELETED" . We could display that in "properties" section. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-918) UI : Deleted entities show status as DELETED and Datasets should be differentiable in lineage
[ https://issues.apache.org/jira/browse/ATLAS-918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340420#comment-15340420 ] Suma Shivaprasad commented on ATLAS-918: cc [~madhan.neethiraj] > UI : Deleted entities show status as DELETED and Datasets should be > differentiable in lineage > - > > Key: ATLAS-918 > URL: https://issues.apache.org/jira/browse/ATLAS-918 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: Screen Shot 2016-06-20 at 2.20.32 PM.png > > > Keval, > This jira is to track showing deleted entities with a diff color in lineage. > Is there anything missing in the lineage API to implement this? We need to > take care of this since no one looks at lineage via a process and will be > mostly looking at lineage from a table or a dataset. Also a table can be > dropped and recreated in hive, leading to linage showing as attached in > screen shot > Also can we show the property (status)__status for deleted entities as > "DELETED" . We could display that in "properties" section. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-918) UI : Deleted entities show status as DELETED and Datasets should be differentiable in lineage
[ https://issues.apache.org/jira/browse/ATLAS-918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-918: --- Attachment: Screen Shot 2016-06-20 at 2.20.32 PM.png > UI : Deleted entities show status as DELETED and Datasets should be > differentiable in lineage > - > > Key: ATLAS-918 > URL: https://issues.apache.org/jira/browse/ATLAS-918 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: Screen Shot 2016-06-20 at 2.20.32 PM.png > > > Keval, > This jira is to track showing deleted entities with a diff color in lineage. > Is there anything missing in the lineage API to implement this? We need to > take care of this since no one looks at lineage via a process and will be > mostly looking at lineage from a table or a dataset. Also a table can be > dropped and recreated in hive, leading to linage showing as attached in > screen shot > Also can we show the property (status)__status for deleted entities as > "DELETED" . We could display that in "properties" section. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-919) UI : Deleted references should be shown in red color or filtered out
Suma Shivaprasad created ATLAS-919: -- Summary: UI : Deleted references should be shown in red color or filtered out Key: ATLAS-919 URL: https://issues.apache.org/jira/browse/ATLAS-919 Project: Atlas Issue Type: Bug Affects Versions: 0.7-incubating Reporter: Suma Shivaprasad Assignee: Keval Bhatt Priority: Critical Fix For: 0.7-incubating Foe eg: In a hive_table one of the columns could be dropped in which case the columns array will have the deleted column as well. This could be shown in red color or completely filtered out in the UI. the way to identify this is to check if the entity status is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-919) UI : Deleted references should be shown in red color or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340596#comment-15340596 ] Suma Shivaprasad commented on ATLAS-919: Attaching sample json > UI : Deleted references should be shown in red color or filtered out > > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be shown in > red color or completely filtered out in the UI. the way to identify this is > to check if the entity status is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-919) UI : Deleted references should be shown in red color or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-919: --- Attachment: 91529261-d568-40ee-9b2e-b488cff61de1.json > UI : Deleted references should be shown in red color or filtered out > > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be shown in > red color or completely filtered out in the UI. the way to identify this is > to check if the entity status is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-919) UI : Deleted references should be shown in red color or filtered out
[
https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340596#comment-15340596
]
Suma Shivaprasad edited comment on ATLAS-919 at 6/20/16 10:52 PM:
--
Attaching sample json . Refer column abc in json which has state =DELETED
{
jsonClass: "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
id: {
jsonClass: "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
id: "371b07b0-1a49-4491-9af6-953b376699e8",
version: 0,
typeName: "hive_column",
state: "DELETED"
},
typeName: "hive_column",
values: {
name: "abc",
qualifiedName: "testdb1.test_table.abc@primary",
comment: null,
type: "string",
table: {
jsonClass: "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
id: "91529261-d568-40ee-9b2e-b488cff61de1",
version: 0,
typeName: "hive_table",
state: "ACTIVE"
}
},
traitNames: [ ],
traits: { }
}
],
was (Author: suma.shivaprasad):
Attaching sample json
> UI : Deleted references should be shown in red color or filtered out
>
>
> Key: ATLAS-919
> URL: https://issues.apache.org/jira/browse/ATLAS-919
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
>Assignee: Keval Bhatt
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json
>
>
> Foe eg: In a hive_table one of the columns could be dropped in which case the
> columns array will have the deleted column as well. This could be shown in
> red color or completely filtered out in the UI. the way to identify this is
> to check if the entity status is DELETED when going through the references.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-919) UI : Deleted references should be filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-919: --- Summary: UI : Deleted references should be filtered out (was: UI : Deleted references should be shown in red color or filtered out) > UI : Deleted references should be filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be shown in > red color or completely filtered out in the UI. the way to identify this is > to check if the entity status is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-919) UI : Deleted references should be filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340600#comment-15340600 ] Suma Shivaprasad commented on ATLAS-919: [~kevalbhatt18] In general, if the state=DELETED for any of the referenced entities , filter that out in UI as well. > UI : Deleted references should be filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be shown in > red color or completely filtered out in the UI. the way to identify this is > to check if the entity status is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-919) UI : Deleted references should be filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-919: --- Description: Foe eg: In a hive_table one of the columns could be dropped in which case the columns array will have the deleted column as well. This could be completely filtered out in the UI. the way to identify this is to check if the entity status is DELETED when going through the references. (was: Foe eg: In a hive_table one of the columns could be dropped in which case the columns array will have the deleted column as well. This could be shown in red color or completely filtered out in the UI. the way to identify this is to check if the entity status is DELETED when going through the references.) > UI : Deleted references should be filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. the way to identify this is to check if the entity > status is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-919) UI : Deleted references should be filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-919: --- Description: Foe eg: In a hive_table one of the columns could be dropped in which case the columns array will have the deleted column as well. This could be completely filtered out in the UI. The way to identify this is to check if the entity state is DELETED when going through the references. (was: Foe eg: In a hive_table one of the columns could be dropped in which case the columns array will have the deleted column as well. This could be completely filtered out in the UI. the way to identify this is to check if the entity status is DELETED when going through the references.) > UI : Deleted references should be filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-919) UI : Deleted references should be filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-919: --- Attachment: Screen Shot 2016-06-20 at 3.56.33 PM.png Refer screenshot which shows that the column is shown as a normal column instead of being filtered out > UI : Deleted references should be filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, Screen Shot > 2016-06-20 at 3.56.33 PM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-920) Lineage graph is broken during alter view
[ https://issues.apache.org/jira/browse/ATLAS-920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-920: --- Attachment: Screen Shot 2016-06-20 at 4.09.02 PM.png > Lineage graph is broken during alter view > - > > Key: ATLAS-920 > URL: https://issues.apache.org/jira/browse/ATLAS-920 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-20 at 4.09.02 PM.png > > > Steps to reproduce > 1. CREATE TABLE test_table(abc string) partitioned by (dt date) > 2. CREATE TABLE test_lineage as SELECT * from test_table > 3. CREATE VIEW test_view as SELECT * from test_table > 4. ALTER VIEW test_view as SELECT * from test_lineage > The lineage is broken as seen in screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-920) Lineage graph is broken during alter view
Suma Shivaprasad created ATLAS-920: -- Summary: Lineage graph is broken during alter view Key: ATLAS-920 URL: https://issues.apache.org/jira/browse/ATLAS-920 Project: Atlas Issue Type: Bug Affects Versions: 0.7-incubating Reporter: Suma Shivaprasad Assignee: Keval Bhatt Priority: Blocker Fix For: 0.7-incubating Attachments: Screen Shot 2016-06-20 at 4.09.02 PM.png Steps to reproduce 1. CREATE TABLE test_table(abc string) partitioned by (dt date) 2. CREATE TABLE test_lineage as SELECT * from test_table 3. CREATE VIEW test_view as SELECT * from test_table 4. ALTER VIEW test_view as SELECT * from test_lineage The lineage is broken as seen in screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-920) Lineage graph is broken when there are multiple paths from same source table
[ https://issues.apache.org/jira/browse/ATLAS-920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-920: --- Summary: Lineage graph is broken when there are multiple paths from same source table (was: Lineage graph is broken during alter view) > Lineage graph is broken when there are multiple paths from same source table > > > Key: ATLAS-920 > URL: https://issues.apache.org/jira/browse/ATLAS-920 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-20 at 4.09.02 PM.png > > > Steps to reproduce > 1. CREATE TABLE test_table(abc string) partitioned by (dt date) > 2. CREATE TABLE test_lineage as SELECT * from test_table > 3. CREATE VIEW test_view as SELECT * from test_table > 4. ALTER VIEW test_view as SELECT * from test_lineage > The lineage is broken as seen in screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-920) Lineage graph is broken when there are multiple paths from same source table
[ https://issues.apache.org/jira/browse/ATLAS-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340650#comment-15340650 ] Suma Shivaprasad commented on ATLAS-920: cc [~madhan.neethiraj] > Lineage graph is broken when there are multiple paths from same source table > > > Key: ATLAS-920 > URL: https://issues.apache.org/jira/browse/ATLAS-920 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-20 at 4.09.02 PM.png > > > Steps to reproduce > 1. CREATE TABLE test_table(abc string) partitioned by (dt date) > 2. CREATE TABLE test_lineage as SELECT * from test_table > 3. CREATE VIEW test_view as SELECT * from test_table > 4. ALTER VIEW test_view as SELECT * from test_lineage > The lineage is broken as seen in screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-920) Lineage graph is broken when there are multiple paths from same source table
[ https://issues.apache.org/jira/browse/ATLAS-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340662#comment-15340662 ] Suma Shivaprasad commented on ATLAS-920: It might be worth taking a look at ATLAS-405 which seems to address a similiar issue > Lineage graph is broken when there are multiple paths from same source table > > > Key: ATLAS-920 > URL: https://issues.apache.org/jira/browse/ATLAS-920 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-20 at 4.09.02 PM.png > > > Steps to reproduce > 1. CREATE TABLE test_table(abc string) partitioned by (dt date) > 2. CREATE TABLE test_lineage as SELECT * from test_table > 3. CREATE VIEW test_view as SELECT * from test_table > 4. ALTER VIEW test_view as SELECT * from test_lineage > The lineage is broken as seen in screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-920) Lineage graph is broken when there are multiple paths from same source table
[ https://issues.apache.org/jira/browse/ATLAS-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340662#comment-15340662 ] Suma Shivaprasad edited comment on ATLAS-920 at 6/20/16 11:23 PM: -- It might be worth taking a look at ATLAS-405 which seems to address a similiar issue but with old UI was (Author: suma.shivaprasad): It might be worth taking a look at ATLAS-405 which seems to address a similiar issue > Lineage graph is broken when there are multiple paths from same source table > > > Key: ATLAS-920 > URL: https://issues.apache.org/jira/browse/ATLAS-920 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-20 at 4.09.02 PM.png > > > Steps to reproduce > 1. CREATE TABLE test_table(abc string) partitioned by (dt date) > 2. CREATE TABLE test_lineage as SELECT * from test_table > 3. CREATE VIEW test_view as SELECT * from test_table > 4. ALTER VIEW test_view as SELECT * from test_lineage > The lineage is broken as seen in screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-920) Lineage graph is broken when there are multiple paths from same source table
[ https://issues.apache.org/jira/browse/ATLAS-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340676#comment-15340676 ] Suma Shivaprasad commented on ATLAS-920: Similiar issue with screenshot > Lineage graph is broken when there are multiple paths from same source table > > > Key: ATLAS-920 > URL: https://issues.apache.org/jira/browse/ATLAS-920 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-20 at 4.09.02 PM.png > > > Steps to reproduce > 1. CREATE TABLE test_table(abc string) partitioned by (dt date) > 2. CREATE TABLE test_lineage as SELECT * from test_table > 3. CREATE VIEW test_view as SELECT * from test_table > 4. ALTER VIEW test_view as SELECT * from test_lineage > The lineage is broken as seen in screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-920) Lineage graph is broken when there are multiple paths from same source table
[ https://issues.apache.org/jira/browse/ATLAS-920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-920: --- Attachment: Screen Shot 2016-06-20 at 4.31.58 PM.png Steps to reproduce: 0. hadoop dfs -mkdir /hive_tables/test_path2 hadoop dfs -mkdir /hive_tables/test_path1 1. CREATE TABLE test_export_table4 ( name string ) 2.EXPORT TABLE test_export_table4 to 'hdfs://localhost:9000/hive_tables/test_path2' 3. CREATE EXTERNAL TABLE test_import_table ( name string) location 'hdfs://localhost:9000/test_import_path' 4. IMPORT TABLE test_import_table from 'hdfs://localhost:9000/hive_tables/test_path1' 5. IMPORT TABLE test_import_table from 'hdfs://localhost:9000/hive_tables/test_path2' > Lineage graph is broken when there are multiple paths from same source table > > > Key: ATLAS-920 > URL: https://issues.apache.org/jira/browse/ATLAS-920 > Project: Atlas > Issue Type: Bug > Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-20 at 4.09.02 PM.png, Screen Shot > 2016-06-20 at 4.31.58 PM.png > > > Steps to reproduce > 1. CREATE TABLE test_table(abc string) partitioned by (dt date) > 2. CREATE TABLE test_lineage as SELECT * from test_table > 3. CREATE VIEW test_view as SELECT * from test_table > 4. ALTER VIEW test_view as SELECT * from test_lineage > The lineage is broken as seen in screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-920) Lineage graph is broken when there are multiple paths from same source table
[ https://issues.apache.org/jira/browse/ATLAS-920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-920: --- Attachment: Screen Shot 2016-06-20 at 4.49.24 PM.png > Lineage graph is broken when there are multiple paths from same source table > > > Key: ATLAS-920 > URL: https://issues.apache.org/jira/browse/ATLAS-920 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-20 at 4.09.02 PM.png, Screen Shot > 2016-06-20 at 4.31.58 PM.png, Screen Shot 2016-06-20 at 4.49.24 PM.png > > > Steps to reproduce > 1. CREATE TABLE test_table(abc string) partitioned by (dt date) > 2. CREATE TABLE test_lineage as SELECT * from test_table > 3. CREATE VIEW test_view as SELECT * from test_table > 4. ALTER VIEW test_view as SELECT * from test_lineage > The lineage is broken as seen in screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-822) Type updates - don't allow updating supertypes
[ https://issues.apache.org/jira/browse/ATLAS-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342140#comment-15342140 ] Suma Shivaprasad commented on ATLAS-822: +1 > Type updates - don't allow updating supertypes > -- > > Key: ATLAS-822 > URL: https://issues.apache.org/jira/browse/ATLAS-822 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-822-v2.patch, ATLAS-822.patch > > > In type updates, we allow adding super types. But set of super types is > maintained in every entity, and this type update will require updating all > the entities of that type which is not handled currently. We should either > update all entities or not allow updating super type information -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-922) remove test atlas-application.properties embedded in atlas-typesystem.jar
[ https://issues.apache.org/jira/browse/ATLAS-922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad resolved ATLAS-922. Resolution: Fixed Fix Version/s: 0.7-incubating Committed as part of ATLAS-822. Thanks [~madhan.neethiraj] and [~yhemanth] > remove test atlas-application.properties embedded in atlas-typesystem.jar > - > > Key: ATLAS-922 > URL: https://issues.apache.org/jira/browse/ATLAS-922 > Project: Atlas > Issue Type: Bug >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-922.patch > > > Currently atlas-typesystem.jar embeds atlas-application.properties containing > test properties as a resource. This embedded test properties file can be > loaded in the application/hook, instead of the expected configuration file, > if atlas-typesystem.jar appears earlier in the CLASSPATH. To prevent this, > atlas-application.properties containing test properties should be removed > from atlas-typesystem.jar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-928) UI is not showing the name column for hive tables in the schema tab
[ https://issues.apache.org/jira/browse/ATLAS-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342497#comment-15342497 ] Suma Shivaprasad commented on ATLAS-928: +1. Tested with various hive_table entities > UI is not showing the name column for hive tables in the schema tab > --- > > Key: ATLAS-928 > URL: https://issues.apache.org/jira/browse/ATLAS-928 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Assignee: Hemanth Yamijala >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-928.patch, hive_schema_response.json > > > * Create a hive table from the hive hook. > * Search for it in the UI, and select the schema tab > * There is no name shown for the columns of the table (although all other > attributes like comment, data type are present). > The same issue does not occur for quick_start Tables. > The reason seems to be that the UI is using the attributeDefinitions element > in the return of the schema API to decide what columns to show. With > ATLAS-819, the name attribute was made a common attribute via a supertype for > assets like hive tables. The Schema API does not return the super type > attributes in the attributeDefinitions. Hence, even though the response of > the schema API contains the name attribute, because it is missing from the > attributeDefinitions, it is not picked up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-932) UI: 'create tag' button does not work
[ https://issues.apache.org/jira/browse/ATLAS-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15343012#comment-15343012 ] Suma Shivaprasad commented on ATLAS-932: +1 > UI: 'create tag' button does not work > - > > Key: ATLAS-932 > URL: https://issues.apache.org/jira/browse/ATLAS-932 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj > Fix For: 0.7-incubating > > Attachments: ATLAS-932.patch > > > With the latest Atlas from master, clicking on 'Create Tag' button does not > popup dialog to get details of new tag. Following error is seen in the Java > Console: > Uncaught TypeError: Cannot read property 'get' of undefined > CreateTagLayoutView.js:60 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-920) Lineage graph is broken when there are multiple paths from same source table
[ https://issues.apache.org/jira/browse/ATLAS-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345173#comment-15345173 ] Suma Shivaprasad commented on ATLAS-920: I verified the patch and tested the above scenarios. LGTM. +1 > Lineage graph is broken when there are multiple paths from same source table > > > Key: ATLAS-920 > URL: https://issues.apache.org/jira/browse/ATLAS-920 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Attachments: ATLAS-920.patch, Screen Shot 2016-06-20 at 4.09.02 > PM.png, Screen Shot 2016-06-20 at 4.31.58 PM.png, Screen Shot 2016-06-20 at > 4.49.24 PM.png > > > Steps to reproduce > 1. CREATE TABLE test_table(abc string) partitioned by (dt date) > 2. CREATE TABLE test_lineage as SELECT * from test_table > 3. CREATE VIEW test_view as SELECT * from test_table > 4. ALTER VIEW test_view as SELECT * from test_lineage > The lineage is broken as seen in screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-920) Lineage graph is broken when there are multiple paths from same source table
[ https://issues.apache.org/jira/browse/ATLAS-920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad resolved ATLAS-920. Resolution: Fixed Fix Version/s: 0.8-incubating Committed. Thanks [~kevalbhatt18] > Lineage graph is broken when there are multiple paths from same source table > > > Key: ATLAS-920 > URL: https://issues.apache.org/jira/browse/ATLAS-920 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.8-incubating > > Attachments: ATLAS-920.patch, Screen Shot 2016-06-20 at 4.09.02 > PM.png, Screen Shot 2016-06-20 at 4.31.58 PM.png, Screen Shot 2016-06-20 at > 4.49.24 PM.png > > > Steps to reproduce > 1. CREATE TABLE test_table(abc string) partitioned by (dt date) > 2. CREATE TABLE test_lineage as SELECT * from test_table > 3. CREATE VIEW test_view as SELECT * from test_table > 4. ALTER VIEW test_view as SELECT * from test_lineage > The lineage is broken as seen in screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-919) UI : Deleted references should be shown with deleted status
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-919: --- Summary: UI : Deleted references should be shown with deleted status (was: UI : Deleted references should be filtered out) > UI : Deleted references should be shown with deleted status > --- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, Screen Shot > 2016-06-20 at 3.56.33 PM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-919) UI : Deleted references should be shown in red or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-919: --- Summary: UI : Deleted references should be shown in red or filtered out (was: UI : Deleted references should be shown with deleted status) > UI : Deleted references should be shown in red or filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, Screen Shot > 2016-06-20 at 3.56.33 PM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-919) UI : Deleted references should be shown in red or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340600#comment-15340600 ] Suma Shivaprasad edited comment on ATLAS-919 at 6/23/16 1:20 AM: - [~kevalbhatt18] In general, if the state=DELETED for any of the referenced entities , that could be displayed in red (deleted entity notation) was (Author: suma.shivaprasad): [~kevalbhatt18] In general, if the state=DELETED for any of the referenced entities , filter that out in UI as well. > UI : Deleted references should be shown in red or filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, Screen Shot > 2016-06-20 at 3.56.33 PM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-919) UI : Deleted references should be shown in red or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340607#comment-15340607 ] Suma Shivaprasad edited comment on ATLAS-919 at 6/23/16 1:21 AM: - Refer screenshot which shows that the column is shown as a normal column instead of being shown as deleted was (Author: suma.shivaprasad): Refer screenshot which shows that the column is shown as a normal column instead of being filtered out > UI : Deleted references should be shown in red or filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, Screen Shot > 2016-06-20 at 3.56.33 PM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-918) UI : Deleted entities show status as DELETED and Datasets should be differentiable in lineage
[ https://issues.apache.org/jira/browse/ATLAS-918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345546#comment-15345546 ] Suma Shivaprasad commented on ATLAS-918: [~kevalbhatt18] If I am not mistaken, after calling the lineage API , we get a bunch of guids and we call getEntity(guid) to get the names for each of these ? In that call you will also get the entity status as DELETED, in those cases the entity could be shown as deleted in lineage? > UI : Deleted entities show status as DELETED and Datasets should be > differentiable in lineage > - > > Key: ATLAS-918 > URL: https://issues.apache.org/jira/browse/ATLAS-918 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: Screen Shot 2016-06-20 at 2.20.32 PM.png > > > Keval, > This jira is to track showing deleted entities with a diff color in lineage. > Is there anything missing in the lineage API to implement this? We need to > take care of this since no one looks at lineage via a process and will be > mostly looking at lineage from a table or a dataset. Also a table can be > dropped and recreated in hive, leading to linage showing as attached in > screen shot > Also can we show the property (status)__status for deleted entities as > "DELETED" . We could display that in "properties" section. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-919) UI : Deleted references should be shown in red or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-919: --- Attachment: Screen Shot 2016-06-23 at 11.26.17 AM.png > UI : Deleted references should be shown in red or filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, > ATLAS-919.patch, Screen Shot 2016-06-20 at 3.56.33 PM.png, Screen Shot > 2016-06-23 at 11.26.17 AM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-919) UI : Deleted references should be shown in red or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15346932#comment-15346932 ] Suma Shivaprasad commented on ATLAS-919: [~kevalbhatt18] There were some minor issues while applying. So rebased the patch and applied. However I see that there is another issue after applying the rebased patch . the db, sd and paremeters are appearing blank in the UI after applying this. Attached screenshot . Not sure if it was a problem with the rebase or with the patch itself. Please check and let me know. > UI : Deleted references should be shown in red or filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, > ATLAS-919.patch, Screen Shot 2016-06-20 at 3.56.33 PM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-919) UI : Deleted references should be shown in red or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-919: --- Attachment: ATLAS-919.rebased.patch > UI : Deleted references should be shown in red or filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, > ATLAS-919.patch, ATLAS-919.rebased.patch, Screen Shot 2016-06-20 at 3.56.33 > PM.png, Screen Shot 2016-06-23 at 11.26.17 AM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-918) UI : Deleted entities show status as DELETED and Datasets should be differentiable in lineage
[ https://issues.apache.org/jira/browse/ATLAS-918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15346963#comment-15346963 ] Suma Shivaprasad commented on ATLAS-918: [~kevalbhatt18] Yes it makes sense not to call any extra APIs for getting this information. We will take a look and get back on the API changes. > UI : Deleted entities show status as DELETED and Datasets should be > differentiable in lineage > - > > Key: ATLAS-918 > URL: https://issues.apache.org/jira/browse/ATLAS-918 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: Screen Shot 2016-06-20 at 2.20.32 PM.png > > > Keval, > This jira is to track showing deleted entities with a diff color in lineage. > Is there anything missing in the lineage API to implement this? We need to > take care of this since no one looks at lineage via a process and will be > mostly looking at lineage from a table or a dataset. Also a table can be > dropped and recreated in hive, leading to linage showing as attached in > screen shot > Also can we show the property (status)__status for deleted entities as > "DELETED" . We could display that in "properties" section. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-947) Return state information in inputs and outputs lineage API
Suma Shivaprasad created ATLAS-947: -- Summary: Return state information in inputs and outputs lineage API Key: ATLAS-947 URL: https://issues.apache.org/jira/browse/ATLAS-947 Project: Atlas Issue Type: Bug Affects Versions: 0.7-incubating, 0.8-incubating Reporter: Suma Shivaprasad Fix For: 0.8-incubating Return state of entity along with name in lineage inputs and outputs API -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-948) import-hive should allow an option to continue after failure
[ https://issues.apache.org/jira/browse/ATLAS-948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-948: --- Priority: Critical (was: Major) > import-hive should allow an option to continue after failure > > > Key: ATLAS-948 > URL: https://issues.apache.org/jira/browse/ATLAS-948 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.8-incubating > > > Currently import-hive.sh aborts whenever a failure occurs during import of a > table. It should allow continuation after logging the failure which could be > checked and post processed later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-948) import-hive should allow an option to continue after failure
Suma Shivaprasad created ATLAS-948: -- Summary: import-hive should allow an option to continue after failure Key: ATLAS-948 URL: https://issues.apache.org/jira/browse/ATLAS-948 Project: Atlas Issue Type: Bug Affects Versions: 0.7-incubating, 0.8-incubating Reporter: Suma Shivaprasad Assignee: Suma Shivaprasad Fix For: 0.8-incubating Currently import-hive.sh aborts whenever a failure occurs during import of a table. It should allow continuation after logging the failure which could be checked and post processed later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-947) Return state information in inputs and outputs lineage API
[ https://issues.apache.org/jira/browse/ATLAS-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348971#comment-15348971 ] Suma Shivaprasad commented on ATLAS-947: +1 > Return state information in inputs and outputs lineage API > -- > > Key: ATLAS-947 > URL: https://issues.apache.org/jira/browse/ATLAS-947 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Shwetha G S > Fix For: 0.8-incubating > > Attachments: ATLAS-947.patch > > > Return state of entity along with name in lineage inputs and outputs API -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-919) UI : Deleted references should be shown in red or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348977#comment-15348977 ] Suma Shivaprasad commented on ATLAS-919: When a deleted table is viewed, property "sd" is not displayed . Can you check this? That should also be displayed with a deleted icon > UI : Deleted references should be shown in red or filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, > ATLAS-919.1.patch, ATLAS-919.patch, ATLAS-919.rebased.patch, Screen Shot > 2016-06-20 at 3.56.33 PM.png, Screen Shot 2016-06-23 at 11.26.17 AM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-919) UI : Deleted references should be shown in red or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348977#comment-15348977 ] Suma Shivaprasad edited comment on ATLAS-919 at 6/25/16 1:08 AM: - When a deleted table is viewed, property "sd" is not displayed . Can you check this? That should also be displayed with a deleted icon. Rest looks fine to me was (Author: suma.shivaprasad): When a deleted table is viewed, property "sd" is not displayed . Can you check this? That should also be displayed with a deleted icon > UI : Deleted references should be shown in red or filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, > ATLAS-919.1.patch, ATLAS-919.patch, ATLAS-919.rebased.patch, Screen Shot > 2016-06-20 at 3.56.33 PM.png, Screen Shot 2016-06-23 at 11.26.17 AM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351486#comment-15351486 ] Suma Shivaprasad commented on ATLAS-688: +1 > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-688-v2.patch, ATLAS-688-v3.patch, > ATLAS-688-v4.patch, ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351493#comment-15351493 ] Suma Shivaprasad commented on ATLAS-688: The above change may not work on windows. Can you file a bug to track it? > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-688-v2.patch, ATLAS-688-v3.patch, > ATLAS-688-v4.patch, ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-952) Drill plugin for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad resolved ATLAS-952. Resolution: Invalid Wrong project > Drill plugin for Cassandra > -- > > Key: ATLAS-952 > URL: https://issues.apache.org/jira/browse/ATLAS-952 > Project: Atlas > Issue Type: Improvement >Reporter: Lokesh Ceeba > > Adapter or Drill plugin for Cassandra -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-918) UI : Deleted entities show status as DELETED and Datasets should be differentiable in lineage
[ https://issues.apache.org/jira/browse/ATLAS-918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351829#comment-15351829 ] Suma Shivaprasad commented on ATLAS-918: [~kevalbhatt18] ATLAS-947(State information) is available now on master to finish this jira. Please refer that jira for sample json. > UI : Deleted entities show status as DELETED and Datasets should be > differentiable in lineage > - > > Key: ATLAS-918 > URL: https://issues.apache.org/jira/browse/ATLAS-918 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: Screen Shot 2016-06-20 at 2.20.32 PM.png > > > Keval, > This jira is to track showing deleted entities with a diff color in lineage. > Is there anything missing in the lineage API to implement this? We need to > take care of this since no one looks at lineage via a process and will be > mostly looking at lineage from a table or a dataset. Also a table can be > dropped and recreated in hive, leading to linage showing as attached in > screen shot > Also can we show the property (status)__status for deleted entities as > "DELETED" . We could display that in "properties" section. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-919) UI : Deleted references should be shown in red or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351919#comment-15351919 ] Suma Shivaprasad commented on ATLAS-919: [~kevalbhatt18] Took a look at the patch. Can you use "qualifiedName" if "name" is not available to display the link for references. And if "qualifiedName" is not available, only then use "typeName". Please make these changes . Rest looks fine. > UI : Deleted references should be shown in red or filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, > ATLAS-919.1.patch, ATLAS-919.2.patch, ATLAS-919.patch, > ATLAS-919.rebased.patch, Screen Shot 2016-06-20 at 3.56.33 PM.png, Screen > Shot 2016-06-23 at 11.26.17 AM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-948) import-hive should allow an option to continue after failure
[ https://issues.apache.org/jira/browse/ATLAS-948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-948: --- Attachment: ATLAS-948.patch Handling failures with a flag(failOnError) to abort on failure if specified while running import-hive.sh > import-hive should allow an option to continue after failure > > > Key: ATLAS-948 > URL: https://issues.apache.org/jira/browse/ATLAS-948 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.8-incubating > > Attachments: ATLAS-948.patch > > > Currently import-hive.sh aborts whenever a failure occurs during import of a > table. It should allow continuation after logging the failure which could be > checked and post processed later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-948) import-hive should allow an option to continue after failure
[ https://issues.apache.org/jira/browse/ATLAS-948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351920#comment-15351920 ] Suma Shivaprasad edited comment on ATLAS-948 at 6/27/16 10:00 PM: -- Handling failures with a flag(failOnError) to abort on failure if specified while running import-hive.sh. this option is only for table imports and DB import failures will still continue to fail. was (Author: suma.shivaprasad): Handling failures with a flag(failOnError) to abort on failure if specified while running import-hive.sh > import-hive should allow an option to continue after failure > > > Key: ATLAS-948 > URL: https://issues.apache.org/jira/browse/ATLAS-948 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.8-incubating > > Attachments: ATLAS-948.patch > > > Currently import-hive.sh aborts whenever a failure occurs during import of a > table. It should allow continuation after logging the failure which could be > checked and post processed later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-948) import-hive should allow an option to continue after failure
[ https://issues.apache.org/jira/browse/ATLAS-948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-948: --- Attachment: (was: ATLAS-948.patch) > import-hive should allow an option to continue after failure > > > Key: ATLAS-948 > URL: https://issues.apache.org/jira/browse/ATLAS-948 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.8-incubating > > > Currently import-hive.sh aborts whenever a failure occurs during import of a > table. It should allow continuation after logging the failure which could be > checked and post processed later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-948) import-hive should allow an option to continue after failure
[ https://issues.apache.org/jira/browse/ATLAS-948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-948: --- Attachment: ATLAS-948.patch > import-hive should allow an option to continue after failure > > > Key: ATLAS-948 > URL: https://issues.apache.org/jira/browse/ATLAS-948 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.8-incubating > > Attachments: ATLAS-948.patch > > > Currently import-hive.sh aborts whenever a failure occurs during import of a > table. It should allow continuation after logging the failure which could be > checked and post processed later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-919) UI : Deleted references should be shown in red or filtered out
[ https://issues.apache.org/jira/browse/ATLAS-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-919: --- Affects Version/s: 0.8-incubating Fix Version/s: (was: 0.7-incubating) 0.8-incubating > UI : Deleted references should be shown in red or filtered out > -- > > Key: ATLAS-919 > URL: https://issues.apache.org/jira/browse/ATLAS-919 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.8-incubating > > Attachments: 91529261-d568-40ee-9b2e-b488cff61de1.json, > ATLAS-919.1.patch, ATLAS-919.2.patch, ATLAS-919.3.patch, ATLAS-919.patch, > ATLAS-919.rebased.patch, Screen Shot 2016-06-20 at 3.56.33 PM.png, Screen > Shot 2016-06-23 at 11.26.17 AM.png > > > Foe eg: In a hive_table one of the columns could be dropped in which case the > columns array will have the deleted column as well. This could be completely > filtered out in the UI. The way to identify this is to check if the entity > state is DELETED when going through the references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-954) Get hadoop classpath if command hadoop is in PATH
[ https://issues.apache.org/jira/browse/ATLAS-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-954: --- Affects Version/s: 0.8-incubating > Get hadoop classpath if command hadoop is in PATH > - > > Key: ATLAS-954 > URL: https://issues.apache.org/jira/browse/ATLAS-954 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.8-incubating >Reporter: Vimal Sharma >Assignee: Vimal Sharma > Fix For: 0.8-incubating > > Attachments: ATLAS-954.patch > > > The script import_hive.sh expects Hadoop classpath. > Environment variable HADOOP_HOME or HADOOP_CLASSPATH might not be set but > command hadoop may be in the PATH. > Get hadoop classpath from hadoop command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-917) Add hdfs paths to process qualified name for non-partition based queries
[ https://issues.apache.org/jira/browse/ATLAS-917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-917: --- Attachment: ATLAS-917.patch > Add hdfs paths to process qualified name for non-partition based queries > > > Key: ATLAS-917 > URL: https://issues.apache.org/jira/browse/ATLAS-917 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Attachments: ATLAS-917.patch > > > Add hdfs paths to process qualified name only in case of non-partition based > queries. This needs to be done per HiveOperation type > 1. if HiveOperation = LOAD, IMPORT, EXPORT - detect if the current query > context is dealing with partitions and do not add if it is partition based. > 2. If HiveOperation = INSERT OVERWRITE DFS_PATH/LOCAL_PATH , then detect if > the query context is dealing with a partitioned table in inputs and decide if > we need to add or not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-917) Add hdfs paths to process qualified name for non-partition based queries
[ https://issues.apache.org/jira/browse/ATLAS-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354523#comment-15354523 ] Suma Shivaprasad commented on ATLAS-917: https://reviews.apache.org/r/49350/ > Add hdfs paths to process qualified name for non-partition based queries > > > Key: ATLAS-917 > URL: https://issues.apache.org/jira/browse/ATLAS-917 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-917.patch > > > Add hdfs paths to process qualified name only in case of non-partition based > queries. This needs to be done per HiveOperation type > 1. if HiveOperation = LOAD, IMPORT, EXPORT - detect if the current query > context is dealing with partitions and do not add if it is partition based. > 2. If HiveOperation = INSERT OVERWRITE DFS_PATH/LOCAL_PATH , then detect if > the query context is dealing with a partitioned table in inputs and decide if > we need to add or not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-917) Add hdfs paths to process qualified name for non-partition based queries
[ https://issues.apache.org/jira/browse/ATLAS-917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-917: --- Fix Version/s: 0.8-incubating > Add hdfs paths to process qualified name for non-partition based queries > > > Key: ATLAS-917 > URL: https://issues.apache.org/jira/browse/ATLAS-917 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-917.patch > > > Add hdfs paths to process qualified name only in case of non-partition based > queries. This needs to be done per HiveOperation type > 1. if HiveOperation = LOAD, IMPORT, EXPORT - detect if the current query > context is dealing with partitions and do not add if it is partition based. > 2. If HiveOperation = INSERT OVERWRITE DFS_PATH/LOCAL_PATH , then detect if > the query context is dealing with a partitioned table in inputs and decide if > we need to add or not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-902) Atlas throws exception due to null definition in Hive create table statement
[
https://issues.apache.org/jira/browse/ATLAS-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355307#comment-15355307
]
Suma Shivaprasad commented on ATLAS-902:
+1
> Atlas throws exception due to null definition in Hive create table statement
>
>
> Key: ATLAS-902
> URL: https://issues.apache.org/jira/browse/ATLAS-902
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.5-incubating
> Environment: HDP 2.4 Sandbox
>Reporter: Ryan Templeton
>Assignee: Vimal Sharma
>Priority: Minor
> Attachments: ATLAS-902.patch
>
>
> I was using the following SQL statement to create a Hive table, but Atlas was
> throwing the following exception.
> create external table got_deaths(
> name varchar(64),
> allegiances varchar(32),
> death_year smallint,
> book_of_death tinyint,
> death_chapter tinyint,
> book_intro_chapter tinyint,
> gender tinyint,
> nobility tinyint,
> got tinyint,
> cok tinyint,
> sos tinyint,
> ffc tinyint,
> dwd tinyint
> )
> row format delimited fields terminated by ',' lines terminated by '\n' null
> defined as ''
> stored as TEXTFILE
> location '/user/root/simpleHive/'
> tblproperties ("skip.header.line.count"="1")
> After dropping the "null defined as '' " portion of the create statement,
> Atlas stopped throwing exceptions. I will point out that although Atlas was
> throwing an exception, Hive was able to create the table without issue. So I
> don't think this is semantically incorrect.
> FAILED: Hive Internal Error: org.apache.atlas.AtlasServiceException(Metadata
> service API CREATE_ENTITY failed with status 400(Bad Request) Response Body
> ({"error":"Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32'
> to datatype
> hive_serde","stackTrace":"org.apache.atlas.typesystem.types.ValueConversionException:
> Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32' to
> datatype hive_serde\n\tat
> org.apache.atlas.typesystem.types.TypedStructHandler.convert(TypedStructHandler.java:67)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:168)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:31)\n\tat
>
> org.apache.atlas.typesystem.persistence.StructInstance.set(StructInstance.java:118)\n\tat
>
> org.apache.atlas.typesystem.types.ClassType.convert(ClassType.java:141)\n\tat
> org.apache.atlas.services.DefaultMetadataService.deserializeClassInstance(DefaultMetadataService.java:252)\n\tat
>
> org.apache.atlas.services.DefaultMetadataService.createEntity(DefaultMetadataService.java:230)\n\tat
>
> org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:96)\n\tat
> sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n\tat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
> java.lang.reflect.Method.invoke(Method.java:606)\n\tat
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)\n\tat
>
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)\n\tat
>
> com.sun.jersey.spi.container.servlet.Ser
[jira] [Commented] (ATLAS-959) Exception while writing to audit log. [java.io.IOException: No FileSystem for scheme: hdfs]
[ https://issues.apache.org/jira/browse/ATLAS-959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355308#comment-15355308 ] Suma Shivaprasad commented on ATLAS-959: +1 > Exception while writing to audit log. [java.io.IOException: No FileSystem > for scheme: hdfs] > > > Key: ATLAS-959 > URL: https://issues.apache.org/jira/browse/ATLAS-959 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.8-incubating >Reporter: Saqeeb Shaikh >Assignee: Saqeeb Shaikh >Priority: Critical > Attachments: ATLAS-959.1.patch > > > While writing audit logs in HDFS, Ranger-Atlas plugin gets following error : > [org.apache.ranger.audit.queue.AuditBatchQueue1:] ~ Error writing to log > file. (BaseAuditHandler:329) > java.io.IOException: No FileSystem for scheme: hdfs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-902) Atlas throws exception due to null definition in Hive create table statement
[
https://issues.apache.org/jira/browse/ATLAS-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355422#comment-15355422
]
Suma Shivaprasad commented on ATLAS-902:
1 Test is failing because of this change.
Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 66.991 sec <<<
FAILURE! - in org.apache.atlas.web.resources.EntityJerseyResourceIT
testEntityInvalidValue(org.apache.atlas.web.resources.EntityJerseyResourceIT)
Time elapsed: 0.019 sec <<< FAILURE!
java.lang.AssertionError: Expected AtlasServiceException
at
org.apache.atlas.web.resources.EntityJerseyResourceIT.testEntityInvalidValue(EntityJerseyResourceIT.java:267)
> Atlas throws exception due to null definition in Hive create table statement
>
>
> Key: ATLAS-902
> URL: https://issues.apache.org/jira/browse/ATLAS-902
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.5-incubating
> Environment: HDP 2.4 Sandbox
>Reporter: Ryan Templeton
>Assignee: Vimal Sharma
>Priority: Minor
> Attachments: ATLAS-902.patch
>
>
> I was using the following SQL statement to create a Hive table, but Atlas was
> throwing the following exception.
> create external table got_deaths(
> name varchar(64),
> allegiances varchar(32),
> death_year smallint,
> book_of_death tinyint,
> death_chapter tinyint,
> book_intro_chapter tinyint,
> gender tinyint,
> nobility tinyint,
> got tinyint,
> cok tinyint,
> sos tinyint,
> ffc tinyint,
> dwd tinyint
> )
> row format delimited fields terminated by ',' lines terminated by '\n' null
> defined as ''
> stored as TEXTFILE
> location '/user/root/simpleHive/'
> tblproperties ("skip.header.line.count"="1")
> After dropping the "null defined as '' " portion of the create statement,
> Atlas stopped throwing exceptions. I will point out that although Atlas was
> throwing an exception, Hive was able to create the table without issue. So I
> don't think this is semantically incorrect.
> FAILED: Hive Internal Error: org.apache.atlas.AtlasServiceException(Metadata
> service API CREATE_ENTITY failed with status 400(Bad Request) Response Body
> ({"error":"Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32'
> to datatype
> hive_serde","stackTrace":"org.apache.atlas.typesystem.types.ValueConversionException:
> Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32' to
> datatype hive_serde\n\tat
> org.apache.atlas.typesystem.types.TypedStructHandler.convert(TypedStructHandler.java:67)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:168)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:31)\n\tat
>
> org.apache.atlas.typesystem.persistence.StructInstance.set(StructInstance.java:118)\n\tat
>
> org.apache.atlas.typesystem.types.ClassType.convert(ClassType.java:141)\n\tat
> org.apache.atlas.services.DefaultMetadataService.deserializeClassInstance(DefaultMetadataService.java:252)\n\tat
>
> org.apache.atlas.services.DefaultMetadataService.createEntity(DefaultMetadataService.java:230)\n\tat
>
> org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:96)\n\tat
> sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n\tat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
> java.lang.reflect.Method.invoke(Method.java:606)\n\tat
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)\n\tat
>
> com.sun.jersey
[jira] [Commented] (ATLAS-902) Atlas throws exception due to null definition in Hive create table statement
[
https://issues.apache.org/jira/browse/ATLAS-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355426#comment-15355426
]
Suma Shivaprasad commented on ATLAS-902:
[~svimal2106]
Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 66.991 sec <<<
FAILURE! - in org.apache.atlas.web.resources.EntityJerseyResourceIT
testEntityInvalidValue(org.apache.atlas.web.resources.EntityJerseyResourceIT)
Time elapsed: 0.019 sec <<< FAILURE!
java.lang.AssertionError: Expected AtlasServiceException
at
org.apache.atlas.web.resources.EntityJerseyResourceIT.testEntityInvalidValue(EntityJerseyResourceIT.java:267)
> Atlas throws exception due to null definition in Hive create table statement
>
>
> Key: ATLAS-902
> URL: https://issues.apache.org/jira/browse/ATLAS-902
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.5-incubating
> Environment: HDP 2.4 Sandbox
>Reporter: Ryan Templeton
>Assignee: Vimal Sharma
>Priority: Minor
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-902.patch
>
>
> I was using the following SQL statement to create a Hive table, but Atlas was
> throwing the following exception.
> create external table got_deaths(
> name varchar(64),
> allegiances varchar(32),
> death_year smallint,
> book_of_death tinyint,
> death_chapter tinyint,
> book_intro_chapter tinyint,
> gender tinyint,
> nobility tinyint,
> got tinyint,
> cok tinyint,
> sos tinyint,
> ffc tinyint,
> dwd tinyint
> )
> row format delimited fields terminated by ',' lines terminated by '\n' null
> defined as ''
> stored as TEXTFILE
> location '/user/root/simpleHive/'
> tblproperties ("skip.header.line.count"="1")
> After dropping the "null defined as '' " portion of the create statement,
> Atlas stopped throwing exceptions. I will point out that although Atlas was
> throwing an exception, Hive was able to create the table without issue. So I
> don't think this is semantically incorrect.
> FAILED: Hive Internal Error: org.apache.atlas.AtlasServiceException(Metadata
> service API CREATE_ENTITY failed with status 400(Bad Request) Response Body
> ({"error":"Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32'
> to datatype
> hive_serde","stackTrace":"org.apache.atlas.typesystem.types.ValueConversionException:
> Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32' to
> datatype hive_serde\n\tat
> org.apache.atlas.typesystem.types.TypedStructHandler.convert(TypedStructHandler.java:67)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:168)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:31)\n\tat
>
> org.apache.atlas.typesystem.persistence.StructInstance.set(StructInstance.java:118)\n\tat
>
> org.apache.atlas.typesystem.types.ClassType.convert(ClassType.java:141)\n\tat
> org.apache.atlas.services.DefaultMetadataService.deserializeClassInstance(DefaultMetadataService.java:252)\n\tat
>
> org.apache.atlas.services.DefaultMetadataService.createEntity(DefaultMetadataService.java:230)\n\tat
>
> org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:96)\n\tat
> sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n\tat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
> java.lang.reflect.Method.invoke(Method.java:606)\n\tat
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)\n\ta
[jira] [Commented] (ATLAS-902) Atlas throws exception due to null definition in Hive create table statement
[
https://issues.apache.org/jira/browse/ATLAS-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355428#comment-15355428
]
Suma Shivaprasad commented on ATLAS-902:
[~svimal2106]
Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 66.991 sec <<<
FAILURE! - in org.apache.atlas.web.resources.EntityJerseyResourceIT
testEntityInvalidValue(org.apache.atlas.web.resources.EntityJerseyResourceIT)
Time elapsed: 0.019 sec <<< FAILURE!
java.lang.AssertionError: Expected AtlasServiceException
at
org.apache.atlas.web.resources.EntityJerseyResourceIT.testEntityInvalidValue(EntityJerseyResourceIT.java:267)
> Atlas throws exception due to null definition in Hive create table statement
>
>
> Key: ATLAS-902
> URL: https://issues.apache.org/jira/browse/ATLAS-902
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.5-incubating
> Environment: HDP 2.4 Sandbox
>Reporter: Ryan Templeton
>Assignee: Vimal Sharma
>Priority: Minor
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-902.patch
>
>
> I was using the following SQL statement to create a Hive table, but Atlas was
> throwing the following exception.
> create external table got_deaths(
> name varchar(64),
> allegiances varchar(32),
> death_year smallint,
> book_of_death tinyint,
> death_chapter tinyint,
> book_intro_chapter tinyint,
> gender tinyint,
> nobility tinyint,
> got tinyint,
> cok tinyint,
> sos tinyint,
> ffc tinyint,
> dwd tinyint
> )
> row format delimited fields terminated by ',' lines terminated by '\n' null
> defined as ''
> stored as TEXTFILE
> location '/user/root/simpleHive/'
> tblproperties ("skip.header.line.count"="1")
> After dropping the "null defined as '' " portion of the create statement,
> Atlas stopped throwing exceptions. I will point out that although Atlas was
> throwing an exception, Hive was able to create the table without issue. So I
> don't think this is semantically incorrect.
> FAILED: Hive Internal Error: org.apache.atlas.AtlasServiceException(Metadata
> service API CREATE_ENTITY failed with status 400(Bad Request) Response Body
> ({"error":"Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32'
> to datatype
> hive_serde","stackTrace":"org.apache.atlas.typesystem.types.ValueConversionException:
> Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32' to
> datatype hive_serde\n\tat
> org.apache.atlas.typesystem.types.TypedStructHandler.convert(TypedStructHandler.java:67)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:168)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:31)\n\tat
>
> org.apache.atlas.typesystem.persistence.StructInstance.set(StructInstance.java:118)\n\tat
>
> org.apache.atlas.typesystem.types.ClassType.convert(ClassType.java:141)\n\tat
> org.apache.atlas.services.DefaultMetadataService.deserializeClassInstance(DefaultMetadataService.java:252)\n\tat
>
> org.apache.atlas.services.DefaultMetadataService.createEntity(DefaultMetadataService.java:230)\n\tat
>
> org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:96)\n\tat
> sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n\tat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
> java.lang.reflect.Method.invoke(Method.java:606)\n\tat
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)\n\ta
[jira] [Updated] (ATLAS-917) Add hdfs paths to process qualified name for non-partition based queries
[ https://issues.apache.org/jira/browse/ATLAS-917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-917: --- Attachment: ATLAS-917.1.patch > Add hdfs paths to process qualified name for non-partition based queries > > > Key: ATLAS-917 > URL: https://issues.apache.org/jira/browse/ATLAS-917 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-917.1.patch, ATLAS-917.patch > > > Add hdfs paths to process qualified name only in case of non-partition based > queries. This needs to be done per HiveOperation type > 1. if HiveOperation = LOAD, IMPORT, EXPORT - detect if the current query > context is dealing with partitions and do not add if it is partition based. > 2. If HiveOperation = INSERT OVERWRITE DFS_PATH/LOCAL_PATH , then detect if > the query context is dealing with a partitioned table in inputs and decide if > we need to add or not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-966) Exit execution of import_hive.sh if HIVE_HOME is not set
[ https://issues.apache.org/jira/browse/ATLAS-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357083#comment-15357083 ] Suma Shivaprasad commented on ATLAS-966: Can you fix the echo "+echo Using Hive configuration directory ["$HIVE_CP"]" . Did you mean HIVE_HOME? > Exit execution of import_hive.sh if HIVE_HOME is not set > > > Key: ATLAS-966 > URL: https://issues.apache.org/jira/browse/ATLAS-966 > Project: Atlas > Issue Type: Bug >Reporter: Vimal Sharma >Assignee: Vimal Sharma > Fix For: 0.8-incubating > > Attachments: ATLAS-966.patch > > > The script import_hive.sh should have a check if HIVE_HOME is set. If not, it > should exit with proper message that HIVE_HOME should be set -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-959) Exception while writing to audit log. [java.io.IOException: No FileSystem for scheme: hdfs]
[ https://issues.apache.org/jira/browse/ATLAS-959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-959: --- Affects Version/s: (was: 0.8-incubating) 0.7-incubating > Exception while writing to audit log. [java.io.IOException: No FileSystem > for scheme: hdfs] > > > Key: ATLAS-959 > URL: https://issues.apache.org/jira/browse/ATLAS-959 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Saqeeb Shaikh >Assignee: Saqeeb Shaikh >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-959.1.patch > > > While writing audit logs in HDFS, Ranger-Atlas plugin gets following error : > [org.apache.ranger.audit.queue.AuditBatchQueue1:] ~ Error writing to log > file. (BaseAuditHandler:329) > java.io.IOException: No FileSystem for scheme: hdfs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-966) Exit execution of import_hive.sh if HIVE_HOME is not set
[ https://issues.apache.org/jira/browse/ATLAS-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359376#comment-15359376 ] Suma Shivaprasad commented on ATLAS-966: +1 > Exit execution of import_hive.sh if HIVE_HOME is not set > > > Key: ATLAS-966 > URL: https://issues.apache.org/jira/browse/ATLAS-966 > Project: Atlas > Issue Type: Bug >Reporter: Vimal Sharma >Assignee: Vimal Sharma > Fix For: 0.8-incubating > > Attachments: ATLAS-966.patch > > > The script import_hive.sh should have a check if HIVE_HOME is set. If not, it > should exit with proper message that HIVE_HOME should be set -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-986) Ability to differentiate business catalog terms from traits
Suma Shivaprasad created ATLAS-986: -- Summary: Ability to differentiate business catalog terms from traits Key: ATLAS-986 URL: https://issues.apache.org/jira/browse/ATLAS-986 Project: Atlas Issue Type: Bug Affects Versions: 0.7-incubating Reporter: Suma Shivaprasad Fix For: 0.8-incubating Currently Terms are created a traits and there is no way to differentiate them. Proposal is to add a superType TaxonomyTerm which all Business Catalog terms will inherit from to differentiate them in the model. So , any term type should have TaxononyTerm in its supertypes To differentiate them during notifications, TaxonmoyTerm type will have a "namespace" attribute which will be set to value "atlas.taxonomy" for all terms during their creation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ATLAS-986) Ability to differentiate business catalog terms from traits
[ https://issues.apache.org/jira/browse/ATLAS-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad reassigned ATLAS-986: -- Assignee: Suma Shivaprasad > Ability to differentiate business catalog terms from traits > --- > > Key: ATLAS-986 > URL: https://issues.apache.org/jira/browse/ATLAS-986 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-986.patch > > > Currently Terms are created a traits and there is no way to differentiate > them. > Proposal is to add a superType TaxonomyTerm which all Business Catalog terms > will inherit from to differentiate them in the model. > So , any term type should have TaxononyTerm in its supertypes > To differentiate them during notifications, TaxonmoyTerm type will have a > "namespace" attribute which will be set to value "atlas.taxonomy" for all > terms during their creation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-986) Ability to differentiate business catalog terms from traits
[ https://issues.apache.org/jira/browse/ATLAS-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-986: --- Attachment: ATLAS-986.patch > Ability to differentiate business catalog terms from traits > --- > > Key: ATLAS-986 > URL: https://issues.apache.org/jira/browse/ATLAS-986 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-986.patch > > > Currently Terms are created a traits and there is no way to differentiate > them. > Proposal is to add a superType TaxonomyTerm which all Business Catalog terms > will inherit from to differentiate them in the model. > So , any term type should have TaxononyTerm in its supertypes > To differentiate them during notifications, TaxonmoyTerm type will have a > "namespace" attribute which will be set to value "atlas.taxonomy" for all > terms during their creation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-986) Ability to differentiate business catalog terms from traits
[ https://issues.apache.org/jira/browse/ATLAS-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359513#comment-15359513 ] Suma Shivaprasad commented on ATLAS-986: Deliberately stayed away from adding a "namespace" super type and then adding this to all traits by default since this is then reflected in all trait types which is more invasive than being restricted to business catalog terms (which may undergo remodelling in near future) and namespace change should be well though out and ideally be taken care of as part of ATLAS-51 > Ability to differentiate business catalog terms from traits > --- > > Key: ATLAS-986 > URL: https://issues.apache.org/jira/browse/ATLAS-986 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-986.patch > > > Currently Terms are created a traits and there is no way to differentiate > them. > Proposal is to add a superType TaxonomyTerm which all Business Catalog terms > will inherit from to differentiate them in the model. > So , any term type should have TaxononyTerm in its supertypes > To differentiate them during notifications, TaxonmoyTerm type will have a > "namespace" attribute which will be set to value "atlas.taxonomy" for all > terms during their creation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-986) Ability to differentiate business catalog terms from traits
[ https://issues.apache.org/jira/browse/ATLAS-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-986: --- Description: Currently Terms are created a traits and there is no way to differentiate them. Proposal is to add a superType TaxonomyTerm which all Business Catalog terms will inherit from to differentiate them in the model. So , any term type should have TaxononyTerm in its supertypes To differentiate them during notifications, TaxonmoyTerm type will have a "namespace" attribute which will be set to value "atlas.taxonomy" for all terms during their creation. Normal trait instances which are created and associated with entities will not have this attribute. was: Currently Terms are created a traits and there is no way to differentiate them. Proposal is to add a superType TaxonomyTerm which all Business Catalog terms will inherit from to differentiate them in the model. So , any term type should have TaxononyTerm in its supertypes To differentiate them during notifications, TaxonmoyTerm type will have a "namespace" attribute which will be set to value "atlas.taxonomy" for all terms during their creation. > Ability to differentiate business catalog terms from traits > --- > > Key: ATLAS-986 > URL: https://issues.apache.org/jira/browse/ATLAS-986 > Project: Atlas > Issue Type: Bug > Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-986.patch > > > Currently Terms are created a traits and there is no way to differentiate > them. > Proposal is to add a superType TaxonomyTerm which all Business Catalog terms > will inherit from to differentiate them in the model. > So , any term type should have TaxononyTerm in its supertypes > To differentiate them during notifications, TaxonmoyTerm type will have a > "namespace" attribute which will be set to value "atlas.taxonomy" for all > terms during their creation. Normal trait instances which are created and > associated with entities will not have this attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-986) Ability to differentiate business catalog terms from traits
[ https://issues.apache.org/jira/browse/ATLAS-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359653#comment-15359653 ] Suma Shivaprasad commented on ATLAS-986: A couple of changes from previous patch 1. Made namespace attribute optional 2. Added namespace attribute to Taxonomy entity as well. > Ability to differentiate business catalog terms from traits > --- > > Key: ATLAS-986 > URL: https://issues.apache.org/jira/browse/ATLAS-986 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-986.1.patch, ATLAS-986.patch > > > Currently Terms are created a traits and there is no way to differentiate > them. > Proposal is to add a superType TaxonomyTerm which all Business Catalog terms > will inherit from to differentiate them in the model. > So , any term type should have TaxononyTerm in its supertypes > To differentiate them during notifications, TaxonmoyTerm type will have a > "namespace" attribute which will be set to value "atlas.taxonomy" for all > terms during their creation. Normal trait instances which are created and > associated with entities will not have this attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-986) Ability to differentiate business catalog terms from traits
[ https://issues.apache.org/jira/browse/ATLAS-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-986: --- Attachment: ATLAS-986.1.patch > Ability to differentiate business catalog terms from traits > --- > > Key: ATLAS-986 > URL: https://issues.apache.org/jira/browse/ATLAS-986 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-986.1.patch, ATLAS-986.patch > > > Currently Terms are created a traits and there is no way to differentiate > them. > Proposal is to add a superType TaxonomyTerm which all Business Catalog terms > will inherit from to differentiate them in the model. > So , any term type should have TaxononyTerm in its supertypes > To differentiate them during notifications, TaxonmoyTerm type will have a > "namespace" attribute which will be set to value "atlas.taxonomy" for all > terms during their creation. Normal trait instances which are created and > associated with entities will not have this attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-986) Ability to differentiate business catalog terms from traits
[ https://issues.apache.org/jira/browse/ATLAS-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-986: --- Attachment: (was: ATLAS-986.1.patch) > Ability to differentiate business catalog terms from traits > --- > > Key: ATLAS-986 > URL: https://issues.apache.org/jira/browse/ATLAS-986 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-986.1.patch, ATLAS-986.patch > > > Currently Terms are created a traits and there is no way to differentiate > them. > Proposal is to add a superType TaxonomyTerm which all Business Catalog terms > will inherit from to differentiate them in the model. > So , any term type should have TaxononyTerm in its supertypes > To differentiate them during notifications, TaxonmoyTerm type will have a > "namespace" attribute which will be set to value "atlas.taxonomy" for all > terms during their creation. Normal trait instances which are created and > associated with entities will not have this attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-986) Ability to differentiate business catalog terms from traits
[ https://issues.apache.org/jira/browse/ATLAS-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-986: --- Attachment: ATLAS-986.1.patch > Ability to differentiate business catalog terms from traits > --- > > Key: ATLAS-986 > URL: https://issues.apache.org/jira/browse/ATLAS-986 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-986.1.patch, ATLAS-986.patch > > > Currently Terms are created a traits and there is no way to differentiate > them. > Proposal is to add a superType TaxonomyTerm which all Business Catalog terms > will inherit from to differentiate them in the model. > So , any term type should have TaxononyTerm in its supertypes > To differentiate them during notifications, TaxonmoyTerm type will have a > "namespace" attribute which will be set to value "atlas.taxonomy" for all > terms during their creation. Normal trait instances which are created and > associated with entities will not have this attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-986) Ability to differentiate business catalog terms from traits
[ https://issues.apache.org/jira/browse/ATLAS-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-986: --- Attachment: ATLAS-986.2.patch Fixed test failures in catalog and webapp > Ability to differentiate business catalog terms from traits > --- > > Key: ATLAS-986 > URL: https://issues.apache.org/jira/browse/ATLAS-986 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.8-incubating > > Attachments: ATLAS-986.1.patch, ATLAS-986.2.patch, ATLAS-986.patch > > > Currently Terms are created a traits and there is no way to differentiate > them. > Proposal is to add a superType TaxonomyTerm which all Business Catalog terms > will inherit from to differentiate them in the model. > So , any term type should have TaxononyTerm in its supertypes > To differentiate them during notifications, TaxonmoyTerm type will have a > "namespace" attribute which will be set to value "atlas.taxonomy" for all > terms during their creation. Normal trait instances which are created and > associated with entities will not have this attribute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-1027) Atlas hooks should use properties from atlas-application.properties, instead of component's configuration
[ https://issues.apache.org/jira/browse/ATLAS-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382932#comment-15382932 ] Suma Shivaprasad commented on ATLAS-1027: - Tests are failing in HiveHookIT with the patch . Will need to be fixed before committing. > Atlas hooks should use properties from atlas-application.properties, instead > of component's configuration > - > > Key: ATLAS-1027 > URL: https://issues.apache.org/jira/browse/ATLAS-1027 > Project: Atlas > Issue Type: Bug >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj > Attachments: ATLAS-1027.patch > > > Currently Atlas hook for Hive/Storm/Sqoop read properties from component's > configuration and atlas-application.properties. Having the hooks read the > properties only from atlas-application.properties would make it easier to > manage. Only updates to component's configuration should be to register Atlas > hook with the component. The hook implementation should only read necessary > run time configuration from its own config file - > atlas-application.propertiees. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-1021) Update Atlas architecture wiki
[ https://issues.apache.org/jira/browse/ATLAS-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382988#comment-15382988 ] Suma Shivaprasad commented on ATLAS-1021: - +1 > Update Atlas architecture wiki > -- > > Key: ATLAS-1021 > URL: https://issues.apache.org/jira/browse/ATLAS-1021 > Project: Atlas > Issue Type: Improvement >Reporter: Hemanth Yamijala >Assignee: Hemanth Yamijala > Attachments: ATLAS-1021-1.patch, ATLAS-1021.patch > > > There have been some new additions to the Atlas architecture including new > apps like Business Taxonomy. This JIRA is to update the wiki with this > information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-1027) Atlas hooks should use properties from atlas-application.properties, instead of component's configuration
[ https://issues.apache.org/jira/browse/ATLAS-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383314#comment-15383314 ] Suma Shivaprasad commented on ATLAS-1027: - Fixed failing tests > Atlas hooks should use properties from atlas-application.properties, instead > of component's configuration > - > > Key: ATLAS-1027 > URL: https://issues.apache.org/jira/browse/ATLAS-1027 > Project: Atlas > Issue Type: Bug >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj > Attachments: ATLAS-1027.1.patch, ATLAS-1027.patch > > > Currently Atlas hook for Hive/Storm/Sqoop read properties from component's > configuration and atlas-application.properties. Having the hooks read the > properties only from atlas-application.properties would make it easier to > manage. Only updates to component's configuration should be to register Atlas > hook with the component. The hook implementation should only read necessary > run time configuration from its own config file - > atlas-application.propertiees. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-1027) Atlas hooks should use properties from atlas-application.properties, instead of component's configuration
[ https://issues.apache.org/jira/browse/ATLAS-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-1027: Attachment: ATLAS-1027.1.patch > Atlas hooks should use properties from atlas-application.properties, instead > of component's configuration > - > > Key: ATLAS-1027 > URL: https://issues.apache.org/jira/browse/ATLAS-1027 > Project: Atlas > Issue Type: Bug >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj > Attachments: ATLAS-1027.1.patch, ATLAS-1027.patch > > > Currently Atlas hook for Hive/Storm/Sqoop read properties from component's > configuration and atlas-application.properties. Having the hooks read the > properties only from atlas-application.properties would make it easier to > manage. Only updates to component's configuration should be to register Atlas > hook with the component. The hook implementation should only read necessary > run time configuration from its own config file - > atlas-application.propertiees. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-1027) Atlas hooks should use properties from atlas-application.properties, instead of component's configuration
[ https://issues.apache.org/jira/browse/ATLAS-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383314#comment-15383314 ] Suma Shivaprasad edited comment on ATLAS-1027 at 7/18/16 11:52 PM: --- Fixed failing tests and uploaded a new patch. +1 was (Author: suma.shivaprasad): Fixed failing tests > Atlas hooks should use properties from atlas-application.properties, instead > of component's configuration > - > > Key: ATLAS-1027 > URL: https://issues.apache.org/jira/browse/ATLAS-1027 > Project: Atlas > Issue Type: Bug >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj > Attachments: ATLAS-1027.1.patch, ATLAS-1027.patch > > > Currently Atlas hook for Hive/Storm/Sqoop read properties from component's > configuration and atlas-application.properties. Having the hooks read the > properties only from atlas-application.properties would make it easier to > manage. Only updates to component's configuration should be to register Atlas > hook with the component. The hook implementation should only read necessary > run time configuration from its own config file - > atlas-application.propertiees. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-1022) Update typesystem wiki with details
[ https://issues.apache.org/jira/browse/ATLAS-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15384824#comment-15384824 ] Suma Shivaprasad commented on ATLAS-1022: - +1 > Update typesystem wiki with details > --- > > Key: ATLAS-1022 > URL: https://issues.apache.org/jira/browse/ATLAS-1022 > Project: Atlas > Issue Type: Improvement >Reporter: Hemanth Yamijala >Assignee: Hemanth Yamijala > Attachments: ATLAS-1022.patch > > > The current typesystem wiki page has a few issues: > * The descriptions are a bit too terse and can be expanded from a newbie's > perspective. > * The formatting is messed up. > Since the type system is so central to Atlas, it can be given some more love. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-639) Exception for lineage request
[ https://issues.apache.org/jira/browse/ATLAS-639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-639: --- Attachment: ATLAS-639.1.patch Fixed failing tests > Exception for lineage request > - > > Key: ATLAS-639 > URL: https://issues.apache.org/jira/browse/ATLAS-639 > Project: Atlas > Issue Type: Bug >Affects Versions: trunk >Reporter: Ayub Khan >Assignee: Vimal Sharma >Priority: Critical > Fix For: trunk > > Attachments: ATLAS-639.1.patch, ATLAS-639.patch > > > Exception in log for lineage request > Steps to reproduce: > create table j13 (col4 String); > create table j14 (col5 String); > insert into table j13 select * from j14; > insert into table j14 select * from j13; > Lineage API call i.e > http://localhost:21000/api/atlas/lineage/"guid"/inputs/graph is > non-responsive. The following exception is observed in log file: > 2016-07-18 14:46:20,841 WARN - [ZkClient-EventThread-99-localhost:9026:] ~ > [atlas_hw-f45c89ac3a11.local-1468832155097-fb78fddf], no brokers found when > trying to rebalance. (Logging$class:83) > 2016-07-18 14:46:20,848 WARN - [ZkClient-EventThread-99-localhost:9026:] ~ > [atlas_hw-f45c89ac3a11.local-1468832155097-fb78fddf], Topic for path > /brokers/topics/ATLAS_HOOK gets deleted, which should not happen at this time > (Logging$class:83) > 2016-07-18 14:48:27,697 WARN - > [NIOServerCxn.Factory:localhost/127.0.0.1:9026:] ~ caught end of stream > exception (NIOServerCnxn:357) > EndOfStreamException: Unable to read additional data from client sessionid > 0x155fd3803b20033, likely client has closed socket > at > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) > at > org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) > at java.lang.Thread.run(Thread.java:745) > 2016-07-18 14:48:27,715 WARN - > [NIOServerCxn.Factory:localhost/127.0.0.1:9026:] ~ caught end of stream > exception (NIOServerCnxn:357) > EndOfStreamException: Unable to read additional data from client sessionid > 0x155fd3803b20031, likely client has closed socket > at > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) > at > org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) > at java.lang.Thread.run(Thread.java:745) > 2016-07-18 14:48:34,551 ERROR - [main-EventThread:] ~ Background operation > retry gave up (CuratorFrameworkImpl:537) > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss > at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:708) > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:499) > at > org.apache.curator.framework.imps.BackgroundSyncImpl$1.processResult(BackgroundSyncImpl.java:50) > at > org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:609) > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) > 2016-07-18 14:48:34,561 ERROR - [main-EventThread:] ~ Background operation > retry gave up (CuratorFrameworkImpl:537) > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss > at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:708) > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:499) > at > org.apache.curator.framework.imps.BackgroundSyncImpl$1.processResult(BackgroundSyncImpl.java:50) > at > org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:609) > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) > 2016-07-18 14:48:34,566 WARN - > [NIOServerCxn.Factory:localhost/127.0.0.1:9026:] ~ Exception causing close of > session 0x155fd3803b20032 due to java.nio.channels.AsynchronousCloseException > (NIOServerCnxn:362) > 2016-07-18 14:50:50,513 WARN - [main-EventThread:] ~ Session expired event > received (ConnectionState:288) > 2016-07-18 14:52:52,577 WARN - > [NIOServerCxn.Factory:localhost/127.0.0.1:9026:] ~ caught end of stream > exception (NIOServerCnxn:357) >
[jira] [Commented] (ATLAS-639) Exception for lineage request
[ https://issues.apache.org/jira/browse/ATLAS-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385023#comment-15385023 ] Suma Shivaprasad commented on ATLAS-639: Also added enablePath() for loop gremlin Queries since this is recommended for loop closures - http://gremlindocs.spmallette.documentup.com/#pipeenablepath > Exception for lineage request > - > > Key: ATLAS-639 > URL: https://issues.apache.org/jira/browse/ATLAS-639 > Project: Atlas > Issue Type: Bug >Affects Versions: trunk >Reporter: Ayub Khan >Assignee: Vimal Sharma >Priority: Critical > Fix For: trunk > > Attachments: ATLAS-639.1.patch, ATLAS-639.patch > > > Exception in log for lineage request > Steps to reproduce: > create table j13 (col4 String); > create table j14 (col5 String); > insert into table j13 select * from j14; > insert into table j14 select * from j13; > Lineage API call i.e > http://localhost:21000/api/atlas/lineage/"guid"/inputs/graph is > non-responsive. The following exception is observed in log file: > 2016-07-18 14:46:20,841 WARN - [ZkClient-EventThread-99-localhost:9026:] ~ > [atlas_hw-f45c89ac3a11.local-1468832155097-fb78fddf], no brokers found when > trying to rebalance. (Logging$class:83) > 2016-07-18 14:46:20,848 WARN - [ZkClient-EventThread-99-localhost:9026:] ~ > [atlas_hw-f45c89ac3a11.local-1468832155097-fb78fddf], Topic for path > /brokers/topics/ATLAS_HOOK gets deleted, which should not happen at this time > (Logging$class:83) > 2016-07-18 14:48:27,697 WARN - > [NIOServerCxn.Factory:localhost/127.0.0.1:9026:] ~ caught end of stream > exception (NIOServerCnxn:357) > EndOfStreamException: Unable to read additional data from client sessionid > 0x155fd3803b20033, likely client has closed socket > at > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) > at > org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) > at java.lang.Thread.run(Thread.java:745) > 2016-07-18 14:48:27,715 WARN - > [NIOServerCxn.Factory:localhost/127.0.0.1:9026:] ~ caught end of stream > exception (NIOServerCnxn:357) > EndOfStreamException: Unable to read additional data from client sessionid > 0x155fd3803b20031, likely client has closed socket > at > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) > at > org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) > at java.lang.Thread.run(Thread.java:745) > 2016-07-18 14:48:34,551 ERROR - [main-EventThread:] ~ Background operation > retry gave up (CuratorFrameworkImpl:537) > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss > at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:708) > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:499) > at > org.apache.curator.framework.imps.BackgroundSyncImpl$1.processResult(BackgroundSyncImpl.java:50) > at > org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:609) > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) > 2016-07-18 14:48:34,561 ERROR - [main-EventThread:] ~ Background operation > retry gave up (CuratorFrameworkImpl:537) > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss > at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:708) > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:499) > at > org.apache.curator.framework.imps.BackgroundSyncImpl$1.processResult(BackgroundSyncImpl.java:50) > at > org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:609) > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) > 2016-07-18 14:48:34,566 WARN - > [NIOServerCxn.Factory:localhost/127.0.0.1:9026:] ~ Exception causing close of > session 0x155fd3803b20032 due to java.nio.channels.AsynchronousCloseException > (NIOServerCnxn:362) > 2016-07-18 14:50:50,513 WARN - [main-EventThread:] ~ Session expired event > received (ConnectionState:288)
[jira] [Commented] (ATLAS-1036) Compilation error on java 1.8 - GraphBackedDiscoveryService
[
https://issues.apache.org/jira/browse/ATLAS-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386114#comment-15386114
]
Suma Shivaprasad commented on ATLAS-1036:
-
+1
> Compilation error on java 1.8 - GraphBackedDiscoveryService
> ---
>
> Key: ATLAS-1036
> URL: https://issues.apache.org/jira/browse/ATLAS-1036
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-1036.patch
>
>
> Fails on java 1.8:
> {noformat}
> [INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @
> atlas-repository ---
> [WARNING] Zinc server is not available at port 3030 - reverting to normal
> incremental compile
> [INFO] Using incremental compilation
> [INFO] Compiling 9 Scala sources and 50 Java sources to
> /Users/sshivalingamurthy/git/atlas-clone/repository/target/classes...
> [ERROR] warning: [options] bootstrap class path not set in conjunction with
> -source 1.7
> [ERROR]
> /Users/sshivalingamurthy/git/atlas-clone/repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java:143:
> error: incomparable types: Object and int
> [ERROR] && ((Expressions.LimitExpression)
> validatedExpression).limit().rawValue() == 0) {
> [ERROR]
> ^
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (ATLAS-1038) Multiple instances of AtlasPluginClassloader getting initialized
Suma Shivaprasad created ATLAS-1038:
---
Summary: Multiple instances of AtlasPluginClassloader getting
initialized
Key: ATLAS-1038
URL: https://issues.apache.org/jira/browse/ATLAS-1038
Project: Atlas
Issue Type: Bug
Affects Versions: 0.8-incubating
Reporter: Suma Shivaprasad
Assignee: Suma Shivaprasad
Fix For: 0.8-incubating
Current code doesnt not synchronize the instance creation and hence results in
multiple instances getting created
AtlasPluginClassLoader ret =
AccessController.doPrivileged(new
PrivilegedExceptionAction() {
public AtlasPluginClassLoader run() throws Exception {
return new AtlasPluginClassLoader(pluginType,
pluginClass);
}
});
return ret;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-1038) Multiple instances of AtlasPluginClassloader getting initialized
[
https://issues.apache.org/jira/browse/ATLAS-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386830#comment-15386830
]
Suma Shivaprasad commented on ATLAS-1038:
-
Thanks [~madhan.neethiraj] for help with the debugging
> Multiple instances of AtlasPluginClassloader getting initialized
>
>
> Key: ATLAS-1038
> URL: https://issues.apache.org/jira/browse/ATLAS-1038
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
> Fix For: 0.8-incubating
>
>
> Current code doesnt not synchronize the instance creation and hence results
> in multiple instances getting created
> AtlasPluginClassLoader ret =
> AccessController.doPrivileged(new
> PrivilegedExceptionAction() {
> public AtlasPluginClassLoader run() throws Exception {
> return new AtlasPluginClassLoader(pluginType,
> pluginClass);
> }
> });
> return ret;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-1038) Multiple instances of AtlasPluginClassloader getting initialized
[
https://issues.apache.org/jira/browse/ATLAS-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-1038:
Attachment: ATLAS-1038.patch
Fixed with double locked check
> Multiple instances of AtlasPluginClassloader getting initialized
>
>
> Key: ATLAS-1038
> URL: https://issues.apache.org/jira/browse/ATLAS-1038
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-1038.patch
>
>
> Current code doesnt not synchronize the instance creation and hence results
> in multiple instances getting created
> AtlasPluginClassLoader ret =
> AccessController.doPrivileged(new
> PrivilegedExceptionAction() {
> public AtlasPluginClassLoader run() throws Exception {
> return new AtlasPluginClassLoader(pluginType,
> pluginClass);
> }
> });
> return ret;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-1038) Multiple instances of AtlasPluginClassloader getting initialized
[
https://issues.apache.org/jira/browse/ATLAS-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-1038:
Priority: Blocker (was: Major)
> Multiple instances of AtlasPluginClassloader getting initialized
>
>
> Key: ATLAS-1038
> URL: https://issues.apache.org/jira/browse/ATLAS-1038
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-1038.patch
>
>
> Current code doesnt not synchronize the instance creation and hence results
> in multiple instances getting created
> AtlasPluginClassLoader ret =
> AccessController.doPrivileged(new
> PrivilegedExceptionAction() {
> public AtlasPluginClassLoader run() throws Exception {
> return new AtlasPluginClassLoader(pluginType,
> pluginClass);
> }
> });
> return ret;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (ATLAS-1041) Add Mixed Index for Entity State property
Suma Shivaprasad created ATLAS-1041: --- Summary: Add Mixed Index for Entity State property Key: ATLAS-1041 URL: https://issues.apache.org/jira/browse/ATLAS-1041 Project: Atlas Issue Type: Bug Reporter: Suma Shivaprasad Assignee: Suma Shivaprasad Priority: Critical We are using Entity State to query both during writes for unique key checks and DSL searches. Need to index this to improve performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-1041) Add Mixed Index for Entity State property
[ https://issues.apache.org/jira/browse/ATLAS-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-1041: Attachment: ATLAS-1041.patch > Add Mixed Index for Entity State property > - > > Key: ATLAS-1041 > URL: https://issues.apache.org/jira/browse/ATLAS-1041 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.8-incubating > > Attachments: ATLAS-1041.patch > > > We are using Entity State to query both during writes for unique key checks > and DSL searches. Need to index this to improve performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-1041) Add Mixed Index for Entity State property
[ https://issues.apache.org/jira/browse/ATLAS-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15388548#comment-15388548 ] Suma Shivaprasad commented on ATLAS-1041: - Should not create a composite index for Entity State since it has issues with Titan 0.5.4 - https://groups.google.com/forum/#!searchin/aureliusgraphs/hemanth/aureliusgraphs/bx7T843mzXU/fjAsclx7GAAJ. hence creating only a mixed index in GraphBackedSearchIndexer. > Add Mixed Index for Entity State property > - > > Key: ATLAS-1041 > URL: https://issues.apache.org/jira/browse/ATLAS-1041 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.8-incubating > > Attachments: ATLAS-1041.patch > > > We are using Entity State to query both during writes for unique key checks > and DSL searches. Need to index this to improve performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-1041) Add Mixed Index for Entity State property
[ https://issues.apache.org/jira/browse/ATLAS-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-1041: Attachment: (was: ATLAS-1041.patch) > Add Mixed Index for Entity State property > - > > Key: ATLAS-1041 > URL: https://issues.apache.org/jira/browse/ATLAS-1041 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.8-incubating > > > We are using Entity State to query both during writes for unique key checks > and DSL searches. Need to index this to improve performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-1041) Add Mixed Index for Entity State property
[ https://issues.apache.org/jira/browse/ATLAS-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-1041: Attachment: ATLAS-1041.patch > Add Mixed Index for Entity State property > - > > Key: ATLAS-1041 > URL: https://issues.apache.org/jira/browse/ATLAS-1041 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.8-incubating > > Attachments: ATLAS-1041.patch > > > We are using Entity State to query both during writes for unique key checks > and DSL searches. Need to index this to improve performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-1041) Add Mixed Index for Entity State property
[ https://issues.apache.org/jira/browse/ATLAS-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-1041: Attachment: ATLAS-1041.1.patch Fixed typo in comment > Add Mixed Index for Entity State property > - > > Key: ATLAS-1041 > URL: https://issues.apache.org/jira/browse/ATLAS-1041 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating, 0.8-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.8-incubating > > Attachments: ATLAS-1041.1.patch, ATLAS-1041.patch > > > We are using Entity State to query both during writes for unique key checks > and DSL searches. Need to index this to improve performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-1041) Add Mixed Index for Entity State property
[ https://issues.apache.org/jira/browse/ATLAS-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15388578#comment-15388578 ] Suma Shivaprasad commented on ATLAS-1041: - Verified from logs that only mixed indexes are created for __state 2016-07-21 15:47:10,557 INFO - [main:] ~ Indexes do not exist, Creating indexes for titanGraph. (GraphBackedSearchIndexer:98) 2016-07-21 15:47:11,595 DEBUG - [main:] ~ Creating backing index for property __guid of type java.lang.String (GraphBackedSearchIndexer:356) 2016-07-21 15:47:11,653 DEBUG - [main:] ~ Created backing index for property __guid of type java.lang.String (GraphBackedSearchIndexer:359) 2016-07-21 15:47:11,653 DEBUG - [main:] ~ Creating composite index for property __guid of type java.lang.String (GraphBackedSearchIndexer:341) 2016-07-21 15:47:11,657 DEBUG - [main:] ~ Created composite index for property __guid of type java.lang.String (GraphBackedSearchIndexer:349) 2016-07-21 15:47:11,659 DEBUG - [main:] ~ Creating backing index for property __timestamp of type java.lang.Long (GraphBackedSearchIndexer:356) 2016-07-21 15:47:11,668 DEBUG - [main:] ~ Created backing index for property __timestamp of type java.lang.Long (GraphBackedSearchIndexer:359) 2016-07-21 15:47:11,668 DEBUG - [main:] ~ Creating composite index for property __timestamp of type java.lang.Long (GraphBackedSearchIndexer:341) 2016-07-21 15:47:11,672 DEBUG - [main:] ~ Created composite index for property __timestamp of type java.lang.Long (GraphBackedSearchIndexer:349) 2016-07-21 15:47:11,674 DEBUG - [main:] ~ Creating backing index for property __state of type java.lang.String (GraphBackedSearchIndexer:356) 2016-07-21 15:47:11,680 DEBUG - [main:] ~ Created backing index for property __state of type java.lang.String (GraphBackedSearchIndexer:359) 2016-07-21 15:47:11,683 DEBUG - [main:] ~ Creating backing index for property __modificationTimestamp of type java.lang.Long (GraphBackedSearchIndexer:356) 2016-07-21 15:47:11,690 DEBUG - [main:] ~ Created backing index for property __modificationTimestamp of type java.lang.Long (GraphBackedSearchIndexer:359) 2016-07-21 15:47:11,690 DEBUG - [main:] ~ Creating composite index for property __modificationTimestamp of type java.lang.Long (GraphBackedSearchIndexer:341) 2016-07-21 15:47:11,693 DEBUG - [main:] ~ Created composite index for property __modificationTimestamp of type java.lang.Long (GraphBackedSearchIndexer:349) 2016-07-21 15:47:11,695 DEBUG - [main:] ~ Creating backing index for property __typeName of type java.lang.String (GraphBackedSearchIndexer:356) 2016-07-21 15:47:11,702 DEBUG - [main:] ~ Created backing index for property __typeName of type java.lang.String (GraphBackedSearchIndexer:359) 2016-07-21 15:47:11,702 DEBUG - [main:] ~ Creating composite index for property __typeName of type java.lang.String (GraphBackedSearchIndexer:341) 2016-07-21 15:47:11,706 DEBUG - [main:] ~ Created composite index for property __typeName of type java.lang.String (GraphBackedSearchIndexer:349) 2016-07-21 15:47:11,709 DEBUG - [main:] ~ Creating composite index for property __superTypeNames of type java.lang.String (GraphBackedSearchIndexer:341) 2016-07-21 15:47:11,712 DEBUG - [main:] ~ Created composite index for property __superTypeNames of type java.lang.String (GraphBackedSearchIndexer:349) 2016-07-21 15:47:11,715 DEBUG - [main:] ~ Creating composite index for property __traitNames of type java.lang.String (GraphBackedSearchIndexer:341) 2016-07-21 15:47:11,718 DEBUG - [main:] ~ Created composite index for property __traitNames of type java.lang.String (GraphBackedSearchIndexer:349) 2016-07-21 15:47:11,728 INFO - [main:] ~ Created mixed index for entityText (GraphBackedSearchIndexer:152) 2016-07-21 15:47:11,732 DEBUG - [main:] ~ Creating backing index for property __type.name of type java.lang.String (GraphBackedSearchIndexer:356) 2016-07-21 15:47:11,738 DEBUG - [main:] ~ Created backing index for property __type.name of type java.lang.String (GraphBackedSearchIndexer:359) 2016-07-21 15:47:11,738 DEBUG - [main:] ~ Creating composite index for property __type.name of type java.lang.String (GraphBackedSearchIndexer:341) 2016-07-21 15:47:11,742 DEBUG - [main:] ~ Created composite index for property __type.name of type java.lang.String (GraphBackedSearchIndexer:349) 2016-07-21 15:47:11,744 DEBUG - [main:] ~ Creating backing index for property __type of type java.lang.String (GraphBackedSearchIndexer:356) 2016-07-21 15:47:11,751 DEBUG - [main:] ~ Created backing index for property __type of type java.lang.String (GraphBackedSearchIndexer:359) 2016-07-21 15:47:11,752 DEBUG - [main:] ~ Creating composite index for property __type of type java.lang.String (GraphBackedSearchIndexer:341) > Add Mixed Index for Entity State property > - > >
[jira] [Created] (ATLAS-1042) Performance improvement changes for propertykey+typeName based queries
Suma Shivaprasad created ATLAS-1042: --- Summary: Performance improvement changes for propertykey+typeName based queries Key: ATLAS-1042 URL: https://issues.apache.org/jira/browse/ATLAS-1042 Project: Atlas Issue Type: Bug Reporter: Suma Shivaprasad Assignee: Suma Shivaprasad The current entity writes which run with qualifiedName + typeName + state seem to be running slow since composite indexes do not exist for qualifiedName+typeName which seems to explain the regression caused in ATLAS-911 which added typeName for unique checks. This jira is to address that and possibly explore any other optimizations that could be done for propertyKey+typeName based queries -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-1042) Performance improvement changes for propertykey+typeName based queries
[ https://issues.apache.org/jira/browse/ATLAS-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-1042: Attachment: ATLAS-1042.patch The patch currently has changes also done in ATLAS-1041 > Performance improvement changes for propertykey+typeName based queries > -- > > Key: ATLAS-1042 > URL: https://issues.apache.org/jira/browse/ATLAS-1042 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > Attachments: ATLAS-1042.patch > > > The current entity writes which run with qualifiedName + typeName + state > seem to be running slow since composite indexes do not exist for > qualifiedName+typeName which seems to explain the regression caused in > ATLAS-911 which added typeName for unique checks. > This jira is to address that and possibly explore any other optimizations > that could be done for propertyKey+typeName based queries -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-1042) Performance improvement changes for propertykey+typeName based queries
[ https://issues.apache.org/jira/browse/ATLAS-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15388859#comment-15388859 ] Suma Shivaprasad commented on ATLAS-1042: - This is an initial draft and needs to be cleaned/tested. > Performance improvement changes for propertykey+typeName based queries > -- > > Key: ATLAS-1042 > URL: https://issues.apache.org/jira/browse/ATLAS-1042 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > Attachments: ATLAS-1042.patch > > > The current entity writes which run with qualifiedName + typeName + state > seem to be running slow since composite indexes do not exist for > qualifiedName+typeName which seems to explain the regression caused in > ATLAS-911 which added typeName for unique checks. > This jira is to address that and possibly explore any other optimizations > that could be done for propertyKey+typeName based queries -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-1042) Performance improvement changes for propertykey+typeName based queries
[ https://issues.apache.org/jira/browse/ATLAS-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-1042: Description: The current entity writes which run with qualifiedName + typeName + state seem to be running slow since composite indexes do not exist for qualifiedName+typeName which seems to explain the regression caused in ATLAS-911 which added typeName for unique checks. This jira is to address that and possibly explore any other optimizations that could be done for propertyKey+typeName based queries. was: The current entity writes which run with qualifiedName + typeName + state seem to be running slow since composite indexes do not exist for qualifiedName+typeName which seems to explain the regression caused in ATLAS-911 which added typeName for unique checks. This jira is to address that and possibly explore any other optimizations that could be done for propertyKey+typeName based queries > Performance improvement changes for propertykey+typeName based queries > -- > > Key: ATLAS-1042 > URL: https://issues.apache.org/jira/browse/ATLAS-1042 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > Attachments: ATLAS-1042.patch > > > The current entity writes which run with qualifiedName + typeName + state > seem to be running slow since composite indexes do not exist for > qualifiedName+typeName which seems to explain the regression caused in > ATLAS-911 which added typeName for unique checks. > This jira is to address that and possibly explore any other optimizations > that could be done for propertyKey+typeName based queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
