Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-12-02 Thread Aadarsh Jajodia


> On Dec. 2, 2019, 5:06 a.m., Madhan Neethiraj wrote:
> > intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java
> > Lines 126 (patched)
> > 
> >
> > Having AtlasNamespaceAttribute extend from AtlasAttribute can be 
> > helpful in many cases, like:
> >   - write atrribute values in EntityGraphMapper
> >   - read attribute values in EntityGraphRetriever
> >   - SolrIndexHelper
> >   - perhaps in search as well (SearchProcessor)
> > 
> > However, this would require:
> >   - AtlasNamespace extend from AtlasStructType (similr to 
> > AtlasClassificationType, AtlasEntityType)
> >   - AtlasNamespaceDef extend from AtlasStructDef
> > 
> > Please review the design for above.

Hi Madhan. Thanks for reviewing. The problem with AtlasNamespaceDef extending 
from AtlasStructDef, is that the attributes in a structDef are not tied to a 
specific entity type, like we want for the attributes in a Namespace Def. 
AtlasStructDef. AtlasStructDef has a list of List but those 
attributes are independent of any entity type, hence I was not able to extend 
AtlasNamespaceDef from AtlasStructDef.


- Aadarsh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218870
---


On Nov. 22, 2019, 2:03 a.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Nov. 22, 2019, 2:03 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes and also updates the type registry to include the applicable 
> namespace attributes for every entity type
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
>   intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
> f06f64f450f407e3f9a0e742726ff4dd12ccc695 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 
> 3634fdfd313639eb97b3c4698e091487b0e44a80 
>   intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 
> 4ee68a936f99bb4c819b5335da2cc8bf7d539397 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
> 557ef74a95c2a939b4b89cd1db8fa4c73d52dd51 
>   intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 
> b071dc9d664cee9e1ffc54726ffbf15f4f602d30 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
> 0883d54f490e22c6510e6fc0cb804b87713a7ecb 
>   intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java 
> dba2d88146eff314191ae6bb24ad7337b0ea10ae 
>   intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
> 02613b5f7250b14324ed294c22de079b74d55b08 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
> ff79994c519702e90b2e478d00cae0008889f956 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  7c551304b2b65b90302f6e5fa9fc5b9f1b8e2c12 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
>  2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
>  2cb2b47898ded4d6c5c84800ff93fa58b2c480da 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java
>  a5ccfb5b2055c88f596312f4033bc0034d3d165c 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java
>  PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 
> 6cd0ee331b7ae24757b58e76ec47bf556106846a 
>   webapp/src/main/java/org/apache/atlas/web/rest/TypesREST.java 
> fb56fad6412079e20dd3e345b81a08d9e5ace657 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/8/
> 
> 
> Testing
> ---
> 

Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-12-01 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218870
---




intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java
Lines 126 (patched)


Having AtlasNamespaceAttribute extend from AtlasAttribute can be helpful in 
many cases, like:
  - write atrribute values in EntityGraphMapper
  - read attribute values in EntityGraphRetriever
  - SolrIndexHelper
  - perhaps in search as well (SearchProcessor)

However, this would require:
  - AtlasNamespace extend from AtlasStructType (similr to 
AtlasClassificationType, AtlasEntityType)
  - AtlasNamespaceDef extend from AtlasStructDef

Please review the design for above.


- Madhan Neethiraj


On Nov. 22, 2019, 2:03 a.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Nov. 22, 2019, 2:03 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes and also updates the type registry to include the applicable 
> namespace attributes for every entity type
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
>   intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
> f06f64f450f407e3f9a0e742726ff4dd12ccc695 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 
> 3634fdfd313639eb97b3c4698e091487b0e44a80 
>   intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 
> 4ee68a936f99bb4c819b5335da2cc8bf7d539397 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
> 557ef74a95c2a939b4b89cd1db8fa4c73d52dd51 
>   intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 
> b071dc9d664cee9e1ffc54726ffbf15f4f602d30 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
> 0883d54f490e22c6510e6fc0cb804b87713a7ecb 
>   intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java 
> dba2d88146eff314191ae6bb24ad7337b0ea10ae 
>   intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
> 02613b5f7250b14324ed294c22de079b74d55b08 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
> ff79994c519702e90b2e478d00cae0008889f956 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  7c551304b2b65b90302f6e5fa9fc5b9f1b8e2c12 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
>  2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
>  2cb2b47898ded4d6c5c84800ff93fa58b2c480da 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java
>  a5ccfb5b2055c88f596312f4033bc0034d3d165c 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java
>  PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 
> 6cd0ee331b7ae24757b58e76ec47bf556106846a 
>   webapp/src/main/java/org/apache/atlas/web/rest/TypesREST.java 
> fb56fad6412079e20dd3e345b81a08d9e5ace657 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/8/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-12-01 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218867
---




intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 75 (patched)


perhaps you meant to check if "this.attributeDefs" is null? Consider 
replacing lines #75 - #79 with:

  if (CollectionUtils.isNotEmpty(attributeDefs)) {
if (this.attributeDefs == null) {
  this.attributeDefs = new ArrayList<>();
}

this.attributeDefs.addAll(attributeDefs);
  }



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 198 (patched)


Move this method to #236, after all get/set methods



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 219 (patched)


Add '@Override' annotation



intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java
Lines 243 (patched)


insert following line here:
  sb.append("}");



intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 990 (patched)


Consider replacing populateNamespaceAttributes() by having 
AtlasNamespaceType.resolveReferences() call 
AtlasEntityType.addNamespaceAttribute() - as 
AtlasNamespaceType.resolveReferences() already iterates through namespace 
attributes and validates if entity-types specified are valid.



intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java
Lines 83 (patched)


Comment in #83 should either be removed or moved to #114 below.



intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java
Lines 103 (patched)


Only primitive or enum type attributes should be allowed in namespaces. 
Entity/Struct/Classification/Namespace types should not be allowed here.


- Madhan Neethiraj


On Nov. 22, 2019, 2:03 a.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Nov. 22, 2019, 2:03 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes and also updates the type registry to include the applicable 
> namespace attributes for every entity type
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
>   intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
> f06f64f450f407e3f9a0e742726ff4dd12ccc695 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 
> 3634fdfd313639eb97b3c4698e091487b0e44a80 
>   intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 
> 4ee68a936f99bb4c819b5335da2cc8bf7d539397 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
> 557ef74a95c2a939b4b89cd1db8fa4c73d52dd51 
>   intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 
> b071dc9d664cee9e1ffc54726ffbf15f4f602d30 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
> 0883d54f490e22c6510e6fc0cb804b87713a7ecb 
>   intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java 
> dba2d88146eff314191ae6bb24ad7337b0ea10ae 
>   intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
> 02613b5f7250b14324ed294c22de079b74d55b08 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
> ff79994c519702e90b2e478d00cae0008889f956 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  7c551304b2b65b90302f6e5fa9fc5b9f1b8e2c12 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
>  2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f 
>   
> 

Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-12-01 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218869
---




intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 75 (patched)


this check is incorrect - adding null/empty 'attributeDefs' will reset the 
attributeDefs to emplty list. (old entries are removed). Please review

if (CollectionUtils.isNotEmpty(attributeDefs)) {
   this.attributeDefs.addAll(attributeDefs);
}



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 162 (patched)


'maxStrLength' and 'validPattern' are string attribute specific 
constraints, consider adding this constraints in AtlasAttributeDef.options map 
(define allowed string constraint enums). This will be extensible to add any 
condition in the future.



intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 49 (patched)


nit: remove new line



intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 1000 (patched)


consider refactoring this method:

private void addNamespaceAttribute(AtlasNamespaceAttribute nsAttribute) {
   String nsName = nsAttribute.getDefinedInType().getTypeName();

   if (!namespaceAttributes.containsKey(nsName)) {
  namespaceAttributes.put(nsName, new ArrayList<>());
   }

   namespaceAttributes.get(nsName).add(nsAttribute);
}



intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java
Lines 31 (patched)


nit: remove unused import.



repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
Lines 586 (patched)


this method is similar to method in line 505. consider refactoring to a 
single method and change behavior for AtlasNamespaceDef and AtlasStructDef



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
Lines 67 (patched)


consider refactoring line 67-81 to avoid duplicates:

if (typeDef instanceof AtlasStructDef) {
   for (AtlasStructDef.AtlasAttributeDef attrDef : ((AtlasStructDef) 
typeDef).getAttributeDefs()) {
  validateAttributeName(typeDef, attrDef.getName());
   }
} else if (typeDef instanceof AtlasNamespaceDef) {
   for (AtlasNamespaceDef.AtlasNamespaceAttributeDef attrDef : 
((AtlasNamespaceDef) typeDef).getAttributeDefs()) {
  validateAttributeName(typeDef, attrDef.getName());
   }
}

private void validateAttributeName(AtlasBaseTypeDef typeDef, String 
attributeName) throws AtlasBaseException {
   if (AtlasDSL.Parser.isKeyword(attributeName)) {
  throw new AtlasBaseException(AtlasErrorCode.ATTRIBUTE_NAME_INVALID, 
attributeName, typeDef.getCategory().name());
   }
}



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
Lines 93 (patched)


nit: remove new line



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 370 (patched)


encodedtypeNamePropertyKey => encodedTypeNamePropertyKey



repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java
Lines 32 (patched)


nit: remove unused imports in line 32-33


- Sarath Subramanian


On Nov. 21, 2019, 6:03 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Nov. 21, 2019, 6:03 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes and also updates the type registry to include the applicable 
> namespace attributes for every entity type
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
>   intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
> 

Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-12-01 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218868
---



Patch fails to apply, can you please rebase.

- Sarath Subramanian


On Nov. 21, 2019, 6:03 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Nov. 21, 2019, 6:03 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes and also updates the type registry to include the applicable 
> namespace attributes for every entity type
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
>   intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
> f06f64f450f407e3f9a0e742726ff4dd12ccc695 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 
> 3634fdfd313639eb97b3c4698e091487b0e44a80 
>   intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 
> 4ee68a936f99bb4c819b5335da2cc8bf7d539397 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
> 557ef74a95c2a939b4b89cd1db8fa4c73d52dd51 
>   intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 
> b071dc9d664cee9e1ffc54726ffbf15f4f602d30 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
> 0883d54f490e22c6510e6fc0cb804b87713a7ecb 
>   intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java 
> dba2d88146eff314191ae6bb24ad7337b0ea10ae 
>   intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
> 02613b5f7250b14324ed294c22de079b74d55b08 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
> ff79994c519702e90b2e478d00cae0008889f956 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  7c551304b2b65b90302f6e5fa9fc5b9f1b8e2c12 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
>  2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
>  2cb2b47898ded4d6c5c84800ff93fa58b2c480da 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java
>  a5ccfb5b2055c88f596312f4033bc0034d3d165c 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java
>  PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 
> 6cd0ee331b7ae24757b58e76ec47bf556106846a 
>   webapp/src/main/java/org/apache/atlas/web/rest/TypesREST.java 
> fb56fad6412079e20dd3e345b81a08d9e5ace657 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/8/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-11-21 Thread Aadarsh Jajodia

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/
---

(Updated Nov. 22, 2019, 2:03 a.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, 
and Sarath Subramanian.


Changes
---

Addressed Review comments


Bugs: ATLAS-3486
https://issues.apache.org/jira/browse/ATLAS-3486


Repository: atlas


Description
---

This change is the first part of the bigger task defined as part of ATLAS-3485.
This adds the data model needed for supporting namespaces and namespace 
attributes and also updates the type registry to include the applicable 
namespace attributes for every entity type


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
  intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
f06f64f450f407e3f9a0e742726ff4dd12ccc695 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
PRE-CREATION 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 
3634fdfd313639eb97b3c4698e091487b0e44a80 
  intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 
4ee68a936f99bb4c819b5335da2cc8bf7d539397 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
557ef74a95c2a939b4b89cd1db8fa4c73d52dd51 
  intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 
b071dc9d664cee9e1ffc54726ffbf15f4f602d30 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
0883d54f490e22c6510e6fc0cb804b87713a7ecb 
  intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java 
dba2d88146eff314191ae6bb24ad7337b0ea10ae 
  intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
02613b5f7250b14324ed294c22de079b74d55b08 
  intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
ff79994c519702e90b2e478d00cae0008889f956 
  intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
 7c551304b2b65b90302f6e5fa9fc5b9f1b8e2c12 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
 2cb2b47898ded4d6c5c84800ff93fa58b2c480da 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java
 a5ccfb5b2055c88f596312f4033bc0034d3d165c 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java
 PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 
6cd0ee331b7ae24757b58e76ec47bf556106846a 
  webapp/src/main/java/org/apache/atlas/web/rest/TypesREST.java 
fb56fad6412079e20dd3e345b81a08d9e5ace657 


Diff: https://reviews.apache.org/r/71649/diff/8/

Changes: https://reviews.apache.org/r/71649/diff/7-8/


Testing
---

Added unit tests


Thanks,

Aadarsh Jajodia



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-11-21 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218727
---



I don't see namespace attribuites indexed. This should be indexed. 

Check GraphBackSearchIndexer.addIndexForType() method


intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 174 (patched)


removed unused constructors in line 174, 179, 193 and 203



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 162 (patched)


'maxStrLength' and 'validPattern' are string specific attribute options. 
Consider adding a enum - namespaceAttributeOptions {MAX_STRING_LENGTH, 
VALID_PATTERN} and use 'options' map in AtlasAttributeDef to set the namespace 
attribute options. This will be easier to extend for any int, date options as 
well.



intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java
Lines 83 (patched)


add REST endpoints in TypesREST for:

/namespacedef/name/{name}
/namespacedef/guid/{guid}



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 22 (patched)


nit: remove unused imports in line 22,42



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 59 (patched)


validateType() checks for invalid types only for structDef attruibutes, add 
check in validateType() to check for namespace attributes as well.



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 133 (patched)


is this line needed? please review.



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 226 (patched)


"update classification-def " => "update namespace-def "



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 260 (patched)


"delete struct-def " => "delete namespace-def "



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 297 (patched)


AtlasBaseException is never thrown



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 349 (patched)


'if' condition in line 349 and 353 is duplicate. please review.



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 357 (patched)


check for null as well for 'applicableEntityTypes'. => 
CollectionUtils.isNotEmpty(attributeDef.getApplicableEntityTypes())



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 382 (patched)


=> CollectionUtils.isNotEmpty(attributeDef.getApplicableEntityTypes())



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 470 (patched)


refactor to:

if (!currAttrNames.containsAll(attrNames)) {
 //throw exception
}



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 496 (patched)


check for null attributeDef.getApplicableEntityTypes() as well ?



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 515 (patched)


=> 
if(!updatedApplicableEntityTypes.containsAll(existingAttribute.getApplicableEntityTypes()))
 {
// throw exception
}



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 526 (patched)


set using encoded property key here:

AtlasGraphUtilsV2.encodePropertyKey(propertyKey);



repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
Line 694 (original), 694 (patched)


contains only whitespace changes. revert changes to this file


- Sarath Subramanian


On Nov. 20, 2019, 7:03 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is 

Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-11-20 Thread Aadarsh Jajodia

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/
---

(Updated Nov. 21, 2019, 3:03 a.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, 
and Sarath Subramanian.


Changes
---

Rebased


Bugs: ATLAS-3486
https://issues.apache.org/jira/browse/ATLAS-3486


Repository: atlas


Description
---

This change is the first part of the bigger task defined as part of ATLAS-3485.
This adds the data model needed for supporting namespaces and namespace 
attributes and also updates the type registry to include the applicable 
namespace attributes for every entity type


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
  intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
f06f64f450f407e3f9a0e742726ff4dd12ccc695 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
PRE-CREATION 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 
3634fdfd313639eb97b3c4698e091487b0e44a80 
  intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 
4ee68a936f99bb4c819b5335da2cc8bf7d539397 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
557ef74a95c2a939b4b89cd1db8fa4c73d52dd51 
  intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 
b071dc9d664cee9e1ffc54726ffbf15f4f602d30 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
0883d54f490e22c6510e6fc0cb804b87713a7ecb 
  intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java 
dba2d88146eff314191ae6bb24ad7337b0ea10ae 
  intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
02613b5f7250b14324ed294c22de079b74d55b08 
  intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
ff79994c519702e90b2e478d00cae0008889f956 
  intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java
 a5ccfb5b2055c88f596312f4033bc0034d3d165c 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
 51dd16b8518c9a16088547f3e95c0ef401695895 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java
 PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 
6cd0ee331b7ae24757b58e76ec47bf556106846a 


Diff: https://reviews.apache.org/r/71649/diff/7/

Changes: https://reviews.apache.org/r/71649/diff/6-7/


Testing
---

Added unit tests


Thanks,

Aadarsh Jajodia



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-11-19 Thread Aadarsh Jajodia

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/
---

(Updated Nov. 20, 2019, 2:24 a.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, 
and Sarath Subramanian.


Changes
---

Updated patch to handle cases when applicable entity types cannot be removed as 
part of a namespaceDef update call


Bugs: ATLAS-3486
https://issues.apache.org/jira/browse/ATLAS-3486


Repository: atlas


Description
---

This change is the first part of the bigger task defined as part of ATLAS-3485.
This adds the data model needed for supporting namespaces and namespace 
attributes and also updates the type registry to include the applicable 
namespace attributes for every entity type


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
  intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
f06f64f450f407e3f9a0e742726ff4dd12ccc695 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
PRE-CREATION 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
e10965b870c4a300b41e93ee046b5f6d6b722728 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 
3634fdfd313639eb97b3c4698e091487b0e44a80 
  intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 
4ee68a936f99bb4c819b5335da2cc8bf7d539397 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
884447f81c57faf917a8d0565fc0a0c7ebbd99f0 
  intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 
8b4fd1c3b9005d0a8852f2828475b4ad6a806822 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
0883d54f490e22c6510e6fc0cb804b87713a7ecb 
  intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java 
dba2d88146eff314191ae6bb24ad7337b0ea10ae 
  intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
02613b5f7250b14324ed294c22de079b74d55b08 
  intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
530d5cda44b7d3746f5cb8dd5a23b3c68b254cac 
  intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java
 a5ccfb5b2055c88f596312f4033bc0034d3d165c 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
 51dd16b8518c9a16088547f3e95c0ef401695895 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java
 PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 
6cd0ee331b7ae24757b58e76ec47bf556106846a 


Diff: https://reviews.apache.org/r/71649/diff/6/

Changes: https://reviews.apache.org/r/71649/diff/5-6/


Testing
---

Added unit tests


Thanks,

Aadarsh Jajodia



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-11-19 Thread Aadarsh Jajodia

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/
---

(Updated Nov. 19, 2019, 11:07 p.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, 
and Sarath Subramanian.


Changes
---

Addressed review comments. Also added unit tests to validate applicable entity 
types


Bugs: ATLAS-3486
https://issues.apache.org/jira/browse/ATLAS-3486


Repository: atlas


Description
---

This change is the first part of the bigger task defined as part of ATLAS-3485.
This adds the data model needed for supporting namespaces and namespace 
attributes and also updates the type registry to include the applicable 
namespace attributes for every entity type


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
  intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
f06f64f450f407e3f9a0e742726ff4dd12ccc695 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
PRE-CREATION 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
e10965b870c4a300b41e93ee046b5f6d6b722728 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 
3634fdfd313639eb97b3c4698e091487b0e44a80 
  intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 
4ee68a936f99bb4c819b5335da2cc8bf7d539397 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
884447f81c57faf917a8d0565fc0a0c7ebbd99f0 
  intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 
8b4fd1c3b9005d0a8852f2828475b4ad6a806822 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
0883d54f490e22c6510e6fc0cb804b87713a7ecb 
  intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java 
dba2d88146eff314191ae6bb24ad7337b0ea10ae 
  intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
02613b5f7250b14324ed294c22de079b74d55b08 
  intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
530d5cda44b7d3746f5cb8dd5a23b3c68b254cac 
  intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java
 a5ccfb5b2055c88f596312f4033bc0034d3d165c 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
 51dd16b8518c9a16088547f3e95c0ef401695895 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java
 PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 
6cd0ee331b7ae24757b58e76ec47bf556106846a 


Diff: https://reviews.apache.org/r/71649/diff/5/

Changes: https://reviews.apache.org/r/71649/diff/4-5/


Testing
---

Added unit tests


Thanks,

Aadarsh Jajodia



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-11-19 Thread Aadarsh Jajodia


> On Nov. 15, 2019, 1:03 a.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
> > Lines 264 (patched)
> > 
> >
> > this is not valid. NamespaceDef should be deleted only when no entity 
> > has reference to that namespace name.
> > 
> > Maybe check if there are any entity referring this namespace (using 
> > index query?) before deleting.
> > 
> > Same for line #294

So i can remove this check from this patch and when i submit the patch for the 
entity changes I can make this change for the delete method?


- Aadarsh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218640
---


On Nov. 14, 2019, 12:08 a.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Nov. 14, 2019, 12:08 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes and also updates the type registry to include the applicable 
> namespace attributes for every entity type
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
>   intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
> f06f64f450f407e3f9a0e742726ff4dd12ccc695 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> e10965b870c4a300b41e93ee046b5f6d6b722728 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 
> 3634fdfd313639eb97b3c4698e091487b0e44a80 
>   intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 
> 4ee68a936f99bb4c819b5335da2cc8bf7d539397 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
> 884447f81c57faf917a8d0565fc0a0c7ebbd99f0 
>   intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 
> 8b4fd1c3b9005d0a8852f2828475b4ad6a806822 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
> 0883d54f490e22c6510e6fc0cb804b87713a7ecb 
>   intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java 
> dba2d88146eff314191ae6bb24ad7337b0ea10ae 
>   intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
> 02613b5f7250b14324ed294c22de079b74d55b08 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
> 530d5cda44b7d3746f5cb8dd5a23b3c68b254cac 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
>  2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java
>  a5ccfb5b2055c88f596312f4033bc0034d3d165c 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
>  51dd16b8518c9a16088547f3e95c0ef401695895 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java
>  PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 
> 6cd0ee331b7ae24757b58e76ec47bf556106846a 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/4/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-11-14 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218640
---




intg/src/main/java/org/apache/atlas/AtlasErrorCode.java
Lines 163 (patched)


it will be useful to include namespace name and attribute name in the error 
message.



intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 27 (patched)


nit: unused import



intg/src/test/java/org/apache/atlas/TestUtilsV2.java
Lines 1466 (patched)


defineNamespaceTypesDef() is never used. please review



repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
Lines 559 (patched)


preDeleteByGuid() and preDeleteByName() may not be needed. Please review.



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 253 (patched)


preDeleteByName() and preDeleteByGuid() may not be needed. We will not be 
deleting any reference edges from the namespaceDef vertex (since no edges 
exists).



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 264 (patched)


this is not valid. NamespaceDef should be deleted only when no entity has 
reference to that namespace name.

Maybe check if there are any entity referring this namespace (using index 
query?) before deleting.

Same for line #294



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 311 (patched)


toNamespaceDef => toAtlasNamespaceDef



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 321 (patched)


toNamespaceDef => toAtlasNamespaceDef



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 382 (patched)


toJsonFromAttribute => toJsonFromNamespaceAttribute



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 420 (patched)


toAttributeDefFromJson => toNamespaceAttributeDefFromJson



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 421 (patched)


AtlasBaseException is never thrown



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 524 (patched)


we may need to check for NamespaceAttributeDef with empty/null 
'applicableTypes' => 
CollectionUtils.isEmpty(attributeDef.getApplicableEntityTypes())



repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
Lines 697 (patched)


empty test?



webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java
Lines 22 (patched)


nit: review unused imports


- Sarath Subramanian


On Nov. 13, 2019, 4:08 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Nov. 13, 2019, 4:08 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes and also updates the type registry to include the applicable 
> namespace attributes for every entity type
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
>   intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
> f06f64f450f407e3f9a0e742726ff4dd12ccc695 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> e10965b870c4a300b41e93ee046b5f6d6b722728 
>   

Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-11-13 Thread Aadarsh Jajodia

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/
---

(Updated Nov. 14, 2019, 12:08 a.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, 
and Sarath Subramanian.


Changes
---

Addressed review comments


Bugs: ATLAS-3486
https://issues.apache.org/jira/browse/ATLAS-3486


Repository: atlas


Description
---

This change is the first part of the bigger task defined as part of ATLAS-3485.
This adds the data model needed for supporting namespaces and namespace 
attributes and also updates the type registry to include the applicable 
namespace attributes for every entity type


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 
  intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
f06f64f450f407e3f9a0e742726ff4dd12ccc695 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
PRE-CREATION 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
e10965b870c4a300b41e93ee046b5f6d6b722728 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 
3634fdfd313639eb97b3c4698e091487b0e44a80 
  intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 
4ee68a936f99bb4c819b5335da2cc8bf7d539397 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
884447f81c57faf917a8d0565fc0a0c7ebbd99f0 
  intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 
8b4fd1c3b9005d0a8852f2828475b4ad6a806822 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
0883d54f490e22c6510e6fc0cb804b87713a7ecb 
  intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java 
dba2d88146eff314191ae6bb24ad7337b0ea10ae 
  intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
02613b5f7250b14324ed294c22de079b74d55b08 
  intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
530d5cda44b7d3746f5cb8dd5a23b3c68b254cac 
  intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java
 a5ccfb5b2055c88f596312f4033bc0034d3d165c 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
 51dd16b8518c9a16088547f3e95c0ef401695895 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java
 PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 
6cd0ee331b7ae24757b58e76ec47bf556106846a 


Diff: https://reviews.apache.org/r/71649/diff/4/

Changes: https://reviews.apache.org/r/71649/diff/3-4/


Testing
---

Added unit tests


Thanks,

Aadarsh Jajodia



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-11-09 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218589
---




intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 46 (patched)


I suggest to use List instead  of Set, to be consistent with other 
attributeDef lists - in AtlasStructDef 
(AtlasEntityDef/AtlasClassificationDef/AtlasRelationshipDef).



intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java
Line 83 (original), 86 (patched)


Consider adding a constructor, instead of updating existing one.



intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 271 (patched)


Consider marking populateNamespaceAttributes() method as private, as this 
is called only from this class.  Also, move this impl to later in the class, 
after all public and protected methods.



intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 272 (patched)


Instead of iteraring namespaceDef, consider iterating through 
namespaceTypes:

  private void populateNamespaceAttributes(AtlasTypeRegistry typeRegistry) {
for (AtlasNamespaceType nsType : typeRegistry.getAllNamespaceTypes()) {
  for (AtlasNamespaceAttribute nsAttribute : 
namespaceType.getAttributes()) {
if 
(nsAttribute.getApplicableEntityTypes().contains(this.getTypeName())) {
  addNamespaceAttribute(nsAttribute);
}
  }
}
  }

  private void addNamespaceAttribute(AtlasNamespaceAttribute nsAttribute) {
StringnsName = 
nsAttribute.getDefinedInType().getTypeName();
List attributes = 
namespaceAttributes.get(nsName);

if (attributes == null) {
  attributes = new ArrayList<>();

  namespaceAttributes.put(nsName, attributes);
}

attributes.add(nsAttribute);
  }



intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 272 (patched)


Instead of iteraring namespaceDef, consider iterating through 
namespaceTypes:

  private void populateNamespaceAttributes(AtlasTypeRegistry typeRegistry) {
for (AtlasNamespaceType nsType : typeRegistry.getAllNamespaceTypes()) {
  for (AtlasNamespaceAttribute nsAttribute : 
namespaceType.getAttributes()) {
if 
(nsAttribute.getApplicableEntityTypes().contains(this.getTypeName())) {
  addNamespaceAttribute(nsAttribute);
}
  }
}
  }

  private void addNamespaceAttribute(AtlasNamespaceAttribute nsAttribute) {
StringnsName = 
nsAttribute.getDefinedInType().getTypeName();
List attributes = 
namespaceAttributes.get(nsName);

if (attributes == null) {
  attributes = new ArrayList<>();

  namespaceAttributes.put(nsName, attributes);
}

attributes.add(nsAttribute);
  }



intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java
Line 310 (original), 302 (patched)


retain existing getTypesDef() method, to avoid breaking exising callers of 
this method. Add a new method that takes 'namespaces' parameter.



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
Lines 191 (patched)


"struct-def" => "namespace-def"



repository/src/test/java/org/apache/atlas/repository/tagpropagation/ClassificationPropagationTest.java
Line 619 (original), 619 (patched)


existing constructor for AtlasTypesDef should be retained; changes to this 
file can be reverted.



tools/classification-updater/src/main/java/org/apache/atlas/tools/BulkFetchAndUpdate.java
Line 524 (original), 524 (patched)


existing constructor for AtlasTypesDef should be retained; changes to this 
file can be reverted.


- Madhan Neethiraj


On Nov. 8, 2019, 9:42 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Nov. 8, 2019, 9:42 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 

Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same

2019-11-08 Thread Aadarsh Jajodia

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/
---

(Updated Nov. 8, 2019, 9:42 p.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, 
and Sarath Subramanian.


Summary (updated)
-

ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas 
and also the corresponding type registry changes for the same


Bugs: ATLAS-3486
https://issues.apache.org/jira/browse/ATLAS-3486


Repository: atlas


Description (updated)
---

This change is the first part of the bigger task defined as part of ATLAS-3485.
This adds the data model needed for supporting namespaces and namespace 
attributes and also updates the type registry to include the applicable 
namespace attributes for every entity type


Diffs
-

  intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
f06f64f450f407e3f9a0e742726ff4dd12ccc695 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
PRE-CREATION 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
e10965b870c4a300b41e93ee046b5f6d6b722728 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 
3634fdfd313639eb97b3c4698e091487b0e44a80 
  intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 
4ee68a936f99bb4c819b5335da2cc8bf7d539397 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
884447f81c57faf917a8d0565fc0a0c7ebbd99f0 
  intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 
8b4fd1c3b9005d0a8852f2828475b4ad6a806822 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
0883d54f490e22c6510e6fc0cb804b87713a7ecb 
  intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java 
dba2d88146eff314191ae6bb24ad7337b0ea10ae 
  intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
02613b5f7250b14324ed294c22de079b74d55b08 
  intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
530d5cda44b7d3746f5cb8dd5a23b3c68b254cac 
  intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java
 a5ccfb5b2055c88f596312f4033bc0034d3d165c 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
 51dd16b8518c9a16088547f3e95c0ef401695895 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/tagpropagation/ClassificationPropagationTest.java
 6f9c05e7a7a30c678a0a376d25402c6026d0b391 
  
tools/classification-updater/src/main/java/org/apache/atlas/tools/BulkFetchAndUpdate.java
 91ff89a00d779af9437ba5f6d129bd4595e5f036 
  webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 
6cd0ee331b7ae24757b58e76ec47bf556106846a 


Diff: https://reviews.apache.org/r/71649/diff/3/


Testing
---

Added unit tests


Thanks,

Aadarsh Jajodia



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-30 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218462
---


Ship it!




Ship It!

- Sarath Subramanian


On Oct. 24, 2019, 4:32 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Oct. 24, 2019, 4:32 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
> f06f64f450f407e3f9a0e742726ff4dd12ccc695 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> e10965b870c4a300b41e93ee046b5f6d6b722728 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/2/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-28 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218418
---



Aadarsh - the patch looks good. Thanks!

- Madhan Neethiraj


On Oct. 24, 2019, 11:32 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Oct. 24, 2019, 11:32 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
> f06f64f450f407e3f9a0e742726ff4dd12ccc695 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> e10965b870c4a300b41e93ee046b5f6d6b722728 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/2/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-24 Thread Aadarsh Jajodia


> On Oct. 23, 2019, 8:20 p.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
> > Lines 155 (patched)
> > 
> >
> > displayName is nto used; consider removing it.

Setting the displayName in the constructor for AtlasNamespaceAttributeDef


- Aadarsh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218367
---


On Oct. 22, 2019, 9:08 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Oct. 22, 2019, 9:08 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> e10965b870c4a300b41e93ee046b5f6d6b722728 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/1/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-24 Thread Aadarsh Jajodia

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/
---

(Updated Oct. 24, 2019, 11:32 p.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, 
and Sarath Subramanian.


Changes
---

Addressed Review Comments


Bugs: ATLAS-3486
https://issues.apache.org/jira/browse/ATLAS-3486


Repository: atlas


Description
---

This change is the first part of the bigger task defined as part of ATLAS-3485.
This adds the data model needed for supporting namespaces and namespace 
attributes


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/model/TypeCategory.java 
f06f64f450f407e3f9a0e742726ff4dd12ccc695 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
PRE-CREATION 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
e10965b870c4a300b41e93ee046b5f6d6b722728 
  intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/71649/diff/2/

Changes: https://reviews.apache.org/r/71649/diff/1-2/


Testing
---

Added unit tests


Thanks,

Aadarsh Jajodia



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-24 Thread Aadarsh Jajodia


> On Oct. 22, 2019, 9:38 p.m., Le Ma wrote:
> > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
> > Lines 106 (patched)
> > 
> >
> > instead of using a variable i, Why not using for (int i = 0; i < 
> > xx.size(); i++), it would be cleaner and more intitive, only when i is the 
> > laste index, you can stop appending ","

so since attributeDefs is a Set, cannot iterate over a set using this kind of 
construct for (int i = 0; i < xx.size(); i++). We will have to use an iterator 
for that. Hence I am dropping this comment.


> On Oct. 22, 2019, 9:38 p.m., Le Ma wrote:
> > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
> > Lines 207 (patched)
> > 
> >
> > Consider using the result of toString() to get hashcode.

updated the hashcode to include all the attibutes of the object


> On Oct. 22, 2019, 9:38 p.m., Le Ma wrote:
> > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
> > Lines 211 (patched)
> > 
> >
> > Why not merging two toString() functions? create the stringbuilder 
> > inside, and return sb.toString() ?

So we want to have both methods because the class AtlasBaseTypeDef only 
overrides toString(StringBuilder)


- Aadarsh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218343
---


On Oct. 22, 2019, 9:08 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Oct. 22, 2019, 9:08 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> e10965b870c4a300b41e93ee046b5f6d6b722728 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/1/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-24 Thread Aadarsh Jajodia


> On Oct. 23, 2019, 12:30 a.m., Madhan Neethiraj wrote:
> > - please update AtlasTypesDef to add attribute: private 
> > List   namespaceDefs;

Will be adding this in the next update of this patch


- Aadarsh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218345
---


On Oct. 22, 2019, 9:08 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Oct. 22, 2019, 9:08 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> e10965b870c4a300b41e93ee046b5f6d6b722728 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/1/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-23 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218367
---




intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 155 (patched)


displayName is nto used; consider removing it.


- Sarath Subramanian


On Oct. 22, 2019, 2:08 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Oct. 22, 2019, 2:08 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> e10965b870c4a300b41e93ee046b5f6d6b722728 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/1/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-23 Thread Ashutosh Mestry via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218358
---




intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java
Lines 27 (patched)


New test: Reads JSON.



intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java
Lines 28 (patched)


Refactor: Rename all methods without prefixing with test to avoid 
redundancy.


- Ashutosh Mestry


On Oct. 22, 2019, 9:08 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Oct. 22, 2019, 9:08 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> e10965b870c4a300b41e93ee046b5f6d6b722728 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/1/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-22 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218345
---



- please update AtlasTypesDef to add attribute: private List 
  namespaceDefs;


intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 43 (patched)


namespaceAttributeDefs => attributeDefs



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 50 (patched)


Instead of passing 'null' for TypeCategory, please add NAMESPACE to 
TypeCategory and pass TypeCategory.NAMESPACE



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 60 (patched)


Le - use of "==" is correct here - this is to a simple optimization when 
the same object (this.namespaceAttributeDefs) is passed as the argument.



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 81 (patched)


insert this before #81:
  if (!super.equals(0)) {
return false;
  }



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 83 (patched)


'name' attribute will be handled in super.equals(). Replace #83 with the 
following:
  return Objects.equals(attributeDefs, that.attributeDefs);



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 88 (patched)


Objects.hash(getName()); => Objects.hash(super.hashCode(), attributeDefs);



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 131 (patched)


maxTextLength => maxStrLength
patternText   => validPattern



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 200 (patched)


insert the following before #200:
  if (!super.equals(o)) {
return false;
  }



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 202 (patched)


replace #202 with:
  return Objects.equals(this.applicableEntityTypes, 
that.applicableEntityTypes) &&
 Objects.equals(this.maxStrLength, that.maxStrLength) &&
 Objects.equals(this.validPattern, that.validPattern);



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 232 (patched)


To be consistent with other classes, please move all getters/setters 
immediately after constructors.


- Madhan Neethiraj


On Oct. 22, 2019, 9:08 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Oct. 22, 2019, 9:08 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> e10965b870c4a300b41e93ee046b5f6d6b722728 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/1/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-22 Thread Le Ma

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/#review218343
---




intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 60 (patched)


"==" will comapre the reference of real objects in Java, please consider 
using "equals"



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 106 (patched)


instead of using a variable i, Why not using for (int i = 0; i < xx.size(); 
i++), it would be cleaner and more intitive, only when i is the laste index, 
you can stop appending ","



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 180 (patched)


same commet as above.



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 195 (patched)


Should we consider compare its attributes inside this equal funciton? Use 
name as the only comparison might cause issues.



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 207 (patched)


Consider using the result of toString() to get hashcode.



intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java
Lines 211 (patched)


Why not merging two toString() functions? create the stringbuilder inside, 
and return sb.toString() ?


- Le Ma


On Oct. 22, 2019, 9:08 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71649/
> ---
> 
> (Updated Oct. 22, 2019, 9:08 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan 
> Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3486
> https://issues.apache.org/jira/browse/ATLAS-3486
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This change is the first part of the bigger task defined as part of 
> ATLAS-3485.
> This adds the data model needed for supporting namespaces and namespace 
> attributes
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> e10965b870c4a300b41e93ee046b5f6d6b722728 
>   
> intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71649/diff/1/
> 
> 
> Testing
> ---
> 
> Added unit tests
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>



Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-22 Thread Aadarsh Jajodia

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/
---

Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, 
and Sarath Subramanian.


Repository: atlas


Description
---

This change is the first part of the bigger task defined as part of ATLAS-3485.
This adds the data model needed for supporting namespaces and namespace 
attributes

Testing done:
Added unit tests


Diffs
-

  intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
PRE-CREATION 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
e10965b870c4a300b41e93ee046b5f6d6b722728 
  intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/71649/diff/1/


Testing
---


Thanks,

Aadarsh Jajodia



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-22 Thread Aadarsh Jajodia

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/
---

(Updated Oct. 22, 2019, 9:08 p.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, 
and Sarath Subramanian.


Bugs: ATLAS-3486
https://issues.apache.org/jira/browse/ATLAS-3486


Repository: atlas


Description (updated)
---

This change is the first part of the bigger task defined as part of ATLAS-3485.
This adds the data model needed for supporting namespaces and namespace 
attributes


Diffs
-

  intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
PRE-CREATION 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
e10965b870c4a300b41e93ee046b5f6d6b722728 
  intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/71649/diff/1/


Testing (updated)
---

Added unit tests


Thanks,

Aadarsh Jajodia



Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas

2019-10-22 Thread Aadarsh Jajodia

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71649/
---

(Updated Oct. 22, 2019, 8:58 p.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, 
and Sarath Subramanian.


Bugs: ATLAS-3486
https://issues.apache.org/jira/browse/ATLAS-3486


Repository: atlas


Description
---

This change is the first part of the bigger task defined as part of ATLAS-3485.
This adds the data model needed for supporting namespaces and namespace 
attributes

Testing done:
Added unit tests


Diffs
-

  intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java 
PRE-CREATION 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
e10965b870c4a300b41e93ee046b5f6d6b722728 
  intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/71649/diff/1/


Testing
---


Thanks,

Aadarsh Jajodia