[jira] [Commented] (ATLAS-2120) Inconsistency in Importing already existing types on backup cluster with new definition.

2017-09-09 Thread Ashutosh Mestry (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159973#comment-16159973
 ] 

Ashutosh Mestry commented on ATLAS-2120:


[~madhan.neethiraj] As of now, we add _attrib2_ since it is absent. We don't 
modify _attrib1_ since it is already present.

I have updated unit test (attached) to reflect this.

Should I change behavior so that we don't update if there is any conflict?

> Inconsistency in Importing already existing types on backup cluster with new 
> definition.
> 
>
> Key: ATLAS-2120
> URL: https://issues.apache.org/jira/browse/ATLAS-2120
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Ashutosh Mestry
>Priority: Critical
> Attachments: hdfs_path1.zip
>
>
> 1.Created a tag tag1 on cluster1 with attributes : 
> * attrib1 : string
> * attrib2 : integer
> 2.Created a tag with same name on cluster2 with attributes:
> *attrib1: date
> *attrib3: integer
> (Note the tag names are same , and attrib1 is same but datatypes of attrib1 
> are different in both the clusters)
> 3. On cluster1 , created an entity and associated the tag1 to the entity with 
> attribute values
> *attrib1: "randstr"
> *attrib2: 5
>   and exported the entity into zip file .
> 4.Tried to import the entity into cluster2. 
> Import failed with 500 Internal server error and with following exception :
> {code}
> {"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
>  org.apache.atlas.repository.graphdb.AtlasSchemaViolationException: 
> com.thinkaurelius.titan.core.SchemaViolationException: Value [rand_str] is 
> not an instance of the expected data type for property key [tag1.attrib1] and 
> cannot be converted. Expected: class java.lang.Long, found: class 
> java.lang.String"}
> {code}
> Following is the inconsistency observed :
> Entity is not imported into the cluster2 , but the type definition of tag1 in 
> cluster2 had 3 attributes now (attrib1 : date , attrib2:Integer , 
> attrib3:Integer) and 500 Internal server error is thrown.
> Normally, when a datatype of an attribute is attempted to be updated , Atlas 
> throws the following exception and the type is not updated.
> {code}
> {"errorCode":"ATLAS-400-00-029","errorMessage":"Data type update for 
> attribute is not supported"}
> {code}
> Expected the same to happen while importing (i.e) Import failing with Bad 
> request with the proper error message.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2120) Inconsistency in Importing already existing types on backup cluster with new definition.

2017-09-09 Thread Ashutosh Mestry (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2120:
---
Attachment: ATLAS-2120-Import-Failure-Test.patch

> Inconsistency in Importing already existing types on backup cluster with new 
> definition.
> 
>
> Key: ATLAS-2120
> URL: https://issues.apache.org/jira/browse/ATLAS-2120
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Ashutosh Mestry
>Priority: Critical
> Attachments: ATLAS-2120-Import-Failure-Test.patch, hdfs_path1.zip
>
>
> 1.Created a tag tag1 on cluster1 with attributes : 
> * attrib1 : string
> * attrib2 : integer
> 2.Created a tag with same name on cluster2 with attributes:
> *attrib1: date
> *attrib3: integer
> (Note the tag names are same , and attrib1 is same but datatypes of attrib1 
> are different in both the clusters)
> 3. On cluster1 , created an entity and associated the tag1 to the entity with 
> attribute values
> *attrib1: "randstr"
> *attrib2: 5
>   and exported the entity into zip file .
> 4.Tried to import the entity into cluster2. 
> Import failed with 500 Internal server error and with following exception :
> {code}
> {"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
>  org.apache.atlas.repository.graphdb.AtlasSchemaViolationException: 
> com.thinkaurelius.titan.core.SchemaViolationException: Value [rand_str] is 
> not an instance of the expected data type for property key [tag1.attrib1] and 
> cannot be converted. Expected: class java.lang.Long, found: class 
> java.lang.String"}
> {code}
> Following is the inconsistency observed :
> Entity is not imported into the cluster2 , but the type definition of tag1 in 
> cluster2 had 3 attributes now (attrib1 : date , attrib2:Integer , 
> attrib3:Integer) and 500 Internal server error is thrown.
> Normally, when a datatype of an attribute is attempted to be updated , Atlas 
> throws the following exception and the type is not updated.
> {code}
> {"errorCode":"ATLAS-400-00-029","errorMessage":"Data type update for 
> attribute is not supported"}
> {code}
> Expected the same to happen while importing (i.e) Import failing with Bad 
> request with the proper error message.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2120) Inconsistency in Importing already existing types on backup cluster with new definition.

2017-09-09 Thread Ashutosh Mestry (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2120:
---
Attachment: (was: ATLAS-2120-Import-Failure-Test.patch)

> Inconsistency in Importing already existing types on backup cluster with new 
> definition.
> 
>
> Key: ATLAS-2120
> URL: https://issues.apache.org/jira/browse/ATLAS-2120
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Ashutosh Mestry
>Priority: Critical
> Attachments: ATLAS-2120-Import-Failure-Test.patch, hdfs_path1.zip
>
>
> 1.Created a tag tag1 on cluster1 with attributes : 
> * attrib1 : string
> * attrib2 : integer
> 2.Created a tag with same name on cluster2 with attributes:
> *attrib1: date
> *attrib3: integer
> (Note the tag names are same , and attrib1 is same but datatypes of attrib1 
> are different in both the clusters)
> 3. On cluster1 , created an entity and associated the tag1 to the entity with 
> attribute values
> *attrib1: "randstr"
> *attrib2: 5
>   and exported the entity into zip file .
> 4.Tried to import the entity into cluster2. 
> Import failed with 500 Internal server error and with following exception :
> {code}
> {"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
>  org.apache.atlas.repository.graphdb.AtlasSchemaViolationException: 
> com.thinkaurelius.titan.core.SchemaViolationException: Value [rand_str] is 
> not an instance of the expected data type for property key [tag1.attrib1] and 
> cannot be converted. Expected: class java.lang.Long, found: class 
> java.lang.String"}
> {code}
> Following is the inconsistency observed :
> Entity is not imported into the cluster2 , but the type definition of tag1 in 
> cluster2 had 3 attributes now (attrib1 : date , attrib2:Integer , 
> attrib3:Integer) and 500 Internal server error is thrown.
> Normally, when a datatype of an attribute is attempted to be updated , Atlas 
> throws the following exception and the type is not updated.
> {code}
> {"errorCode":"ATLAS-400-00-029","errorMessage":"Data type update for 
> attribute is not supported"}
> {code}
> Expected the same to happen while importing (i.e) Import failing with Bad 
> request with the proper error message.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 59821: ATLAS-1805:Provide an Atlas hook to send Hbase Namespace/Table/column family metadata to Atlas

2017-09-09 Thread Ramesh Mani 


> On Sept. 1, 2017, 5:45 p.m., Madhan Neethiraj wrote:
> > addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/model/HBaseContext.java
> > Lines 38 (patched)
> > 
> >
> > Few fields seem redundant. Please review and remove fields that are not 
> > needed.
> > 
> > nameSpace ==> namespaceDescriptor should have this info
> > tableName ==> hTableDescriptor should have this info
> > hColumnDescriptors, hColumnDescriptor: is one for column-family and 
> > other for column?

nameSpace is String object, namespaceDescriptor is object with namespace and 
addition info. I need to have this in HbaseOperationContext as different 
operation from Hbase sends either namespace or namespaceDescriptor and hence 
having this in hbaseOperationCOntext made it simple to refer. Also this allowed 
me to have a same method to be used, when message are created for 
namespace,table,columnfamily


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59821/#review184324
---


On Aug. 31, 2017, 10:52 p.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59821/
> ---
> 
> (Updated Aug. 31, 2017, 10:52 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1805
> https://issues.apache.org/jira/browse/ATLAS-1805
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1805: Provide an Atlas hook to send Hbase Namespace/Table/column family 
> metadata to Atlas
> 
> 
> Diffs
> -
> 
>   addons/hbase-bridge-shim/pom.xml PRE-CREATION 
>   
> addons/hbase-bridge-shim/src/main/java/org/apache/atlas/hbase/hook/HBaseAtlasCoprocessor.java
>  PRE-CREATION 
>   addons/hbase-bridge/pom.xml PRE-CREATION 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
>  PRE-CREATION 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/event/HBaseEvent.java
>  PRE-CREATION 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/hook/HBaseAtlasCoprocessor.java
>  PRE-CREATION 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/hook/HBaseAtlasCoprocessorBase.java
>  PRE-CREATION 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/model/HBaseContext.java
>  PRE-CREATION 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/model/HBaseDataTypes.java
>  PRE-CREATION 
>   addons/hbase-bridge/src/main/resources/atlas-hbase-import-log4j.xml 
> PRE-CREATION 
>   
> addons/hbase-bridge/src/test/java/org/apache/atlas/hbase/HBaseAtlasHookTest.java
>  PRE-CREATION 
>   addons/models/0060-hbase_model.json 3e46e06 
>   distro/src/main/assemblies/standalone-package.xml 215cb23 
>   pom.xml 8b9eee6 
> 
> 
> Diff: https://reviews.apache.org/r/59821/diff/3/
> 
> 
> Testing
> ---
> 
> * Review comments fixed with 2 exceptions whcih I have commented on 
> * Added an IT test
> * Testing done in LOCAL VM
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>