date:20160517

Re: Review Request 47382: ATLAS-491 : Business Catalog API

2016-05-17 Thread Hemanth Yamijala


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47382/#review133702
---


Ship it!




My comments which we agreed to address in this patch are in. Testing seems to 
go fine as well. We need to raise follow-up JIRAs for other items which I will 
do. One weird issue I am seeing is that quick_start is throwing an exception if 
taxonomy and term are added and queried, before running quickstart. However, 
Hive hook and import continue to work fine even in that scenario. Will try to 
get some more data on this ASAP.

- Hemanth Yamijala


On May 17, 2016, 10:41 p.m., John Speidel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47382/
> ---
> 
> (Updated May 17, 2016, 10:41 p.m.)
> 
> 
> Review request for atlas and Hemanth Yamijala.
> 
> 
> Bugs: ATLAS-491
> https://issues.apache.org/jira/browse/ATLAS-491
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Initial implementation of Catalog API.
> 
> Known Issues/Limitations:
> - No update/delete support for taxonomies/terms
>   -- this will be added next week
> - Some gaps in test coverage
>   - Need more in-depth test coverage that exercise the generated gremlin 
> pipeline
> - some refactoring should still be done espcecially in the 'query' package
> - need to add more logging
> - need to finish some javadoc
> 
> 
> Diffs
> -
> 
>   catalog/pom.xml PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/AtlasTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/CollectionRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultDateFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultPropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/EntityResourceProvider.java 
> PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/EntityTagResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/InstanceRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/JsonSerializer.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyValueFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Request.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceComparator.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Result.java PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/TaxonomyResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermPath.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermVertexWrapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/VertexWrapper.java 
> PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/BaseResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/EntityResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/EntityTagResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/ResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/TaxonomyResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/TermResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/CatalogException.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/CatalogRuntimeException.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/InvalidPayloadException.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/InvalidQueryException.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/ResourceAlreadyExistsException.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/ResourceNotFoundException.java
>  PRE-CREATION 
>   
>

[jira] [Created] (ATLAS-802) New look UI to show Business Catalog functionalities

2016-05-17 Thread Keval Bhatt (JIRA)

Keval Bhatt created ATLAS-802:
-

 Summary: New look UI to show Business Catalog functionalities
 Key: ATLAS-802
 URL: https://issues.apache.org/jira/browse/ATLAS-802
 Project: Atlas
  Issue Type: Improvement
Affects Versions: 0.7-incubating
Reporter: Keval Bhatt
Assignee: Keval Bhatt
 Fix For: 0.7-incubating


Update the user interface that integrates latest APIs of Business Catalog. 

* Handle CRUD for Terms and provide feature to  create Taxonomy. 
* Feature to create child-terms.

 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 47382: ATLAS-491 : Business Catalog API

2016-05-17 Thread Hemanth Yamijala



> On May 16, 2016, 5:40 a.m., Hemanth Yamijala wrote:
> > catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java, line 
> > 57
> > 
> >
> > This would be a problem if multiple threads are trying to define the 
> > type at the same time (as could happen if multiple clients are trying to 
> > create taxonomies / terms at the same time). We should move type 
> > registration to a one time activity as the rest of the framework.
> 
> John Speidel wrote:
> ok, I will investigate this but I didn't see a way to do this without 
> making changes to Atlas proper, which was a desing goal based on team input.
> 
> Hemanth Yamijala wrote:
> John, we do one time model registration of other models like Hive, Sqoop, 
> Storm etc. These are done one time when the (active) server starts up. The 
> method was introduced in ATLAS-512. Broadly it involves generating a model 
> definition and writing it out as a json file as part of the build process. 
> This is then packaged under the models directory and is picked up 
> automatically by the server at start up. If you could go through the patch in 
> ATLAS-512 and follow the same steps we should be good for fixing this.
> 
> John Speidel wrote:
> I would prefer to handle this in another patch if that is ok with you.

Yes, this is fine. Will file a JIRA to track this.


- Hemanth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47382/#review133328
---


On May 17, 2016, 10:41 p.m., John Speidel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47382/
> ---
> 
> (Updated May 17, 2016, 10:41 p.m.)
> 
> 
> Review request for atlas and Hemanth Yamijala.
> 
> 
> Bugs: ATLAS-491
> https://issues.apache.org/jira/browse/ATLAS-491
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Initial implementation of Catalog API.
> 
> Known Issues/Limitations:
> - No update/delete support for taxonomies/terms
>   -- this will be added next week
> - Some gaps in test coverage
>   - Need more in-depth test coverage that exercise the generated gremlin 
> pipeline
> - some refactoring should still be done espcecially in the 'query' package
> - need to add more logging
> - need to finish some javadoc
> 
> 
> Diffs
> -
> 
>   catalog/pom.xml PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/AtlasTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/CollectionRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultDateFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultPropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/EntityResourceProvider.java 
> PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/EntityTagResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/InstanceRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/JsonSerializer.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyValueFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Request.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceComparator.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Result.java PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/TaxonomyResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermPath.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermVertexWrapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/VertexWrapper.java 
> PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/BaseResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/EntityResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/EntityTagResourceDefinition.java
>  PRE-CREATION 
>   
>

[jira] [Updated] (ATLAS-491) Business Catalog / Taxonomy

2016-05-17 Thread John Speidel (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Speidel updated ATLAS-491:
---
Attachment: ATLAS-491.patch

> Business Catalog / Taxonomy
> ---
>
> Key: ATLAS-491
> URL: https://issues.apache.org/jira/browse/ATLAS-491
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.7-incubating
>Reporter: Erik Bergenholtz
>Assignee: John Speidel
> Fix For: 0.7-incubating
>
> Attachments: 
> 0001-ATLAS-491-Business-Catalog-API-changes-by-John-Speid.patch, 
> ATLAS-491.patch
>
>
> s a Atlas UX user, I need to:
> Browse business taxonomy hieracharlically through tree
> Search by Tags
> Search by combination of keyword, tag, free text in search field
> All search from view context of taxonomy. If 3 levels deep, then the search 
> should be narrow to that scope.
> expose search ordering capabilities available through SOLR.
> As a Data Steward, I need to:
> Manage the taxonomy of objects including
> Objects Management:
> Creating new objects
> Modify the object details - annotations, external links, tags, acceptable use 
> policy, etc
> Delete objects (in UI - changing status in backend,and versioning changes)
> Relationship management:
> creating relationship of objects as a parent or child
> assigning tagging to taxonomy
> verify tag inheritance
> Verify tag impact on tag based policy - ranger API call
> Tag Management:
> List
> Search
> Edit
> Delete (change state to archive)
> All user should be able to comment on any business catalog entity.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-491) Business Catalog / Taxonomy

2016-05-17 Thread John Speidel (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287815#comment-15287815
 ] 

John Speidel commented on ATLAS-491:


I am going to try to attach the latest patch to this Jira.  If you don't see 
it, then the upload failed again.
Attempting to upload the previous patch failed and it appears that it caused 
your network to be blacklisted.

> Business Catalog / Taxonomy
> ---
>
> Key: ATLAS-491
> URL: https://issues.apache.org/jira/browse/ATLAS-491
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.7-incubating
>Reporter: Erik Bergenholtz
>Assignee: John Speidel
> Fix For: 0.7-incubating
>
> Attachments: 
> 0001-ATLAS-491-Business-Catalog-API-changes-by-John-Speid.patch
>
>
> s a Atlas UX user, I need to:
> Browse business taxonomy hieracharlically through tree
> Search by Tags
> Search by combination of keyword, tag, free text in search field
> All search from view context of taxonomy. If 3 levels deep, then the search 
> should be narrow to that scope.
> expose search ordering capabilities available through SOLR.
> As a Data Steward, I need to:
> Manage the taxonomy of objects including
> Objects Management:
> Creating new objects
> Modify the object details - annotations, external links, tags, acceptable use 
> policy, etc
> Delete objects (in UI - changing status in backend,and versioning changes)
> Relationship management:
> creating relationship of objects as a parent or child
> assigning tagging to taxonomy
> verify tag inheritance
> Verify tag impact on tag based policy - ranger API call
> Tag Management:
> List
> Search
> Edit
> Delete (change state to archive)
> All user should be able to comment on any business catalog entity.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-491) Business Catalog / Taxonomy

2016-05-17 Thread John Speidel (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287810#comment-15287810
 ] 

John Speidel commented on ATLAS-491:


Regarding the question about Taxonomy results being returned with entity 
queries.  
To be clear, this observation is with DSL queries and not "v1/entities" queries 
which won't return taxonomy results.
The reason for this is that taxonomies are implemented as entities under the 
covers.
Terms are modeled as traits under the covers.
To provide a 'singleton' term instance, each term(trait) is associated with its 
owning taxonomy.


> Business Catalog / Taxonomy
> ---
>
> Key: ATLAS-491
> URL: https://issues.apache.org/jira/browse/ATLAS-491
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.7-incubating
>Reporter: Erik Bergenholtz
>Assignee: John Speidel
> Fix For: 0.7-incubating
>
> Attachments: 
> 0001-ATLAS-491-Business-Catalog-API-changes-by-John-Speid.patch
>
>
> s a Atlas UX user, I need to:
> Browse business taxonomy hieracharlically through tree
> Search by Tags
> Search by combination of keyword, tag, free text in search field
> All search from view context of taxonomy. If 3 levels deep, then the search 
> should be narrow to that scope.
> expose search ordering capabilities available through SOLR.
> As a Data Steward, I need to:
> Manage the taxonomy of objects including
> Objects Management:
> Creating new objects
> Modify the object details - annotations, external links, tags, acceptable use 
> policy, etc
> Delete objects (in UI - changing status in backend,and versioning changes)
> Relationship management:
> creating relationship of objects as a parent or child
> assigning tagging to taxonomy
> verify tag inheritance
> Verify tag impact on tag based policy - ranger API call
> Tag Management:
> List
> Search
> Edit
> Delete (change state to archive)
> All user should be able to comment on any business catalog entity.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 47382: ATLAS-491 : Business Catalog API

2016-05-17 Thread John Speidel


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47382/
---

(Updated May 17, 2016, 10:41 p.m.)


Review request for atlas and Hemanth Yamijala.


Changes
---

patch which addresses all issues unless otherwise noted


Bugs: ATLAS-491
https://issues.apache.org/jira/browse/ATLAS-491


Repository: atlas


Description
---

Initial implementation of Catalog API.

Known Issues/Limitations:
- No update/delete support for taxonomies/terms
  -- this will be added next week
- Some gaps in test coverage
  - Need more in-depth test coverage that exercise the generated gremlin 
pipeline
- some refactoring should still be done espcecially in the 'query' package
- need to add more logging
- need to finish some javadoc


Diffs (updated)
-

  catalog/pom.xml PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/AtlasTypeSystem.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/BaseRequest.java PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/BaseResourceProvider.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/CollectionRequest.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/DefaultDateFormatter.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/DefaultPropertyMapper.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/EntityResourceProvider.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/EntityTagResourceProvider.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/InstanceRequest.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/JsonSerializer.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/PropertyMapper.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/PropertyValueFormatter.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/Request.java PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/ResourceComparator.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/ResourceProvider.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/Result.java PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/TaxonomyResourceProvider.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/TermPath.java PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/TermResourceProvider.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/TermVertexWrapper.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/VertexWrapper.java 
PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/definition/BaseResourceDefinition.java
 PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/definition/EntityResourceDefinition.java
 PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/definition/EntityTagResourceDefinition.java
 PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/definition/ResourceDefinition.java
 PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/definition/TaxonomyResourceDefinition.java
 PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/definition/TermResourceDefinition.java
 PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/exception/CatalogException.java 
PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/exception/CatalogRuntimeException.java
 PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/exception/InvalidPayloadException.java
 PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/exception/InvalidQueryException.java
 PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/exception/ResourceAlreadyExistsException.java
 PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/exception/ResourceNotFoundException.java
 PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/projection/GenericRelation.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/projection/Projection.java 
PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/projection/ProjectionResult.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/projection/Relation.java 
PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/projection/RelationProjection.java
 PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/projection/RelationSet.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/projection/TagRelation.java 
PRE-CREATION 
  catalog/src/main/java/org/apache/atlas/catalog/projection/TraitRelation.java 
PRE-CREATION 
  
catalog/src/main/java/org/apache/atlas/catalog/query/AlwaysQueryExpression.java 
PRE-CREATION

Re: Review Request 47382: ATLAS-491 : Business Catalog API

2016-05-17 Thread John Speidel



> On May 16, 2016, 5:40 a.m., Hemanth Yamijala wrote:
> > catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java, line 
> > 57
> > 
> >
> > This would be a problem if multiple threads are trying to define the 
> > type at the same time (as could happen if multiple clients are trying to 
> > create taxonomies / terms at the same time). We should move type 
> > registration to a one time activity as the rest of the framework.
> 
> John Speidel wrote:
> ok, I will investigate this but I didn't see a way to do this without 
> making changes to Atlas proper, which was a desing goal based on team input.
> 
> Hemanth Yamijala wrote:
> John, we do one time model registration of other models like Hive, Sqoop, 
> Storm etc. These are done one time when the (active) server starts up. The 
> method was introduced in ATLAS-512. Broadly it involves generating a model 
> definition and writing it out as a json file as part of the build process. 
> This is then packaged under the models directory and is picked up 
> automatically by the server at start up. If you could go through the patch in 
> ATLAS-512 and follow the same steps we should be good for fixing this.

I would prefer to handle this in another patch if that is ok with you.


- John


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47382/#review133328
---


On May 14, 2016, 12:58 a.m., John Speidel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47382/
> ---
> 
> (Updated May 14, 2016, 12:58 a.m.)
> 
> 
> Review request for atlas and Hemanth Yamijala.
> 
> 
> Bugs: ATLAS-491
> https://issues.apache.org/jira/browse/ATLAS-491
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Initial implementation of Catalog API.
> 
> Known Issues/Limitations:
> - No update/delete support for taxonomies/terms
>   -- this will be added next week
> - Some gaps in test coverage
>   - Need more in-depth test coverage that exercise the generated gremlin 
> pipeline
> - some refactoring should still be done espcecially in the 'query' package
> - need to add more logging
> - need to finish some javadoc
> 
> 
> Diffs
> -
> 
>   catalog/pom.xml PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/AtlasTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/CollectionRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultDateFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultPropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/EntityResourceProvider.java 
> PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/EntityTagResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/InstanceRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/JsonSerializer.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyValueFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Request.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceComparator.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Result.java PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/TaxonomyResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermPath.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermVertexWrapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/VertexWrapper.java 
> PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/BaseResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/EntityResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/EntityTagResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/ResourceDefinition.java
>  PRE-CREATION 
>

[jira] [Assigned] (ATLAS-774) Better error handling from login.jsp

2016-05-17 Thread Nixon Rodrigues (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues reassigned ATLAS-774:
-

Assignee: Nixon Rodrigues  (was: Keval Bhatt)

> Better error handling from login.jsp
> 
>
> Key: ATLAS-774
> URL: https://issues.apache.org/jira/browse/ATLAS-774
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-774.patch
>
>
> To post login form via javascript function and to do better error handling 
> pre and post submit of login request.
> This will help redirecting users to accurate pages based on permissions that 
> they have to. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 47458: ATLAS-774 - Error-Handling-from-LoginPage

2016-05-17 Thread Nixon Rodrigues


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47458/
---

Review request for atlas, Erik Bergenholtz, Gautam Borad, Madhan Neethiraj, 
Shwetha GS, and Hemanth Yamijala.


Repository: atlas


Description
---

To post login form via javascript function and to do better error handling pre 
and post submit of login request.
This will help redirecting users to accurate pages based on permissions that 
they have to.

Changes also includes fix for

* favicon 404 issue.
* (ATLAS-780) UI: Atlas UI shows an API response page after login sometimes.


Diffs
-

  dashboardv2/public/js/modules/atlasLogin.js PRE-CREATION 
  
webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationFailureHandler.java
 PRE-CREATION 
  
webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationSuccessHandler.java
 PRE-CREATION 
  webapp/src/main/resources/spring-security.xml 4590eef 
  webapp/src/main/webapp/login.jsp 2990477 

Diff: https://reviews.apache.org/r/47458/diff/


Testing
---

Manual Testing for login related scenerios.
Run mvn clean install.


Thanks,

Nixon Rodrigues

[jira] [Resolved] (ATLAS-790) persist method on JavaRDDLike

2016-05-17 Thread Shwetha G S (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S resolved ATLAS-790.
---
Resolution: Invalid

> persist method on JavaRDDLike
> -
>
> Key: ATLAS-790
> URL: https://issues.apache.org/jira/browse/ATLAS-790
> Project: Atlas
>  Issue Type: Improvement
>Reporter: sumit rastogi
>
> All SubClass of JavaRDDLike has persist function but JavaRDDLike does not 
> have it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-683) Refactor local type-system cache with cache provider interface

2016-05-17 Thread venkata madugundu (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

venkata madugundu updated ATLAS-683:

Attachment: ATLAS-683-2.patch

Addressed first set of review comments from Shwetha. Attaching the patch.

> Refactor local type-system cache with cache provider interface
> --
>
> Key: ATLAS-683
> URL: https://issues.apache.org/jira/browse/ATLAS-683
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 0.7-incubating
>Reporter: venkata madugundu
>Assignee: venkata madugundu
>Priority: Critical
>  Labels: high-availability, performance, scalability
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-683-1.patch, ATLAS-683-2.patch, ATLAS-683.patch, 
> RedisTypeCacheProvider.java
>
>
> As noted in ATLAS-488, local type-system cache makes Atlas runtime stateful 
> and prevents multiple Atlas instances to be active in a cluster. Either the 
> type-cache should be synched across Atlas instances (on all type 
> create/update requests) or the type-cache should be moved out of Atlas to 
> something like a distributed cache. 
> 1. As a first step, the local type-cache code in TypeSystem.java can be 
> refactored to be carved out as an interface like TypeCacheProvider (whose 
> default implementation for a standalone Atlas server would just use 
> in-process local cache). The cache provider implementation itself could be 
> specified as an optional configuration property. Expert users of Atlas can 
> choose to inject a custom cache provider which can likely hit a distributed 
> cache. We are evaluating the use of a distributed cache. 
> 2. As a second step, some more refactoring can be done to minimize/optimize 
> the calls made to TypeSystem for type lookup queries. Essentially, in a given 
> transaction/request, once a type lookup is done, it should not be requeried 
> again. A request scoped variable (guice would probably help with that 
> scoping) can hold all the lookups made in a request. This might sound like a 
> cache of a cache, but I think it should help in reducing the hits to cache 
> provider if the provider is hitting a remote cache.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 46330: Refactor local type-system cache with cache provider interface

2016-05-17 Thread venkata madugundu

> On May 16, 2016, 11:53 a.m., Shwetha GS wrote:
> > common/src/main/java/org/apache/atlas/AtlasRuntimeException.java, line 23
> > 
> >
> > Why throw unchecked exception? Should be checked exception so that 
> > usages can handle it

Agreed. The reason I chose to initially to go with a subclass of 
RuntimeException is to avoid refactoring a whole bunch of methods making calls 
to TypeSystem API, and further call stack. If you feel strongly about having to 
use checked exception, I can make the changes.

> On May 16, 2016, 11:53 a.m., Shwetha GS wrote:
> > repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java, 
> > line 135
> > 
> >
> > Use ApplicationProperties.getClass()

The initialization of cache provider has now been moved from TypeSystemProvider 
to the constructor of TypeSystem and the ApplicationProperties.getClass() has 
been used to get hold of the cache provider.

> On May 16, 2016, 11:53 a.m., Shwetha GS wrote:
> > typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java, 
> > line 167
> > 
> >
> > should also check among core types

Good find. Done.

> On May 16, 2016, 11:53 a.m., Shwetha GS wrote:
> > typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java, 
> > line 328
> > 
> >
> > use isRegistered()

Good one. All internal type lookup with typeCache.has has been modified to use 
isRegistered.

> On May 16, 2016, 11:53 a.m., Shwetha GS wrote:
> > typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java, 
> > line 431
> > 
> >
> > Use isRegistered() wherever typeCache.has() is used

Done.

> On May 16, 2016, 11:53 a.m., Shwetha GS wrote:
> > typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystemProvider.java,
> >  line 44
> > 
> >
> > It doesn't make sense to create an instance of TypeSystem without 
> > setCacheProvider(). Can you move initialising cache provider to TypeSystem 
> > constructor?

Completely agreed. This was something which I struggled to make a good choice 
between using Guice versus and not using it, especially considering the 
TypeSystem bypasses the Guice by way of being directly accessed in all the 
dependencies with TypeSystem.getInstance. Also, once a user calls 
TypeSystem.getInstance, a completely ready instance of TypeSystem should be 
returned. But the instance of TypeSystem is created privately within TypeSystem 
to honor singleton pattern (infact ideally the constructor should have been 
declared private, but has been made public probably for testing purposes).

So, I finally moved the cache provider initialization to the 
TypeSystem::initialize.

> On May 16, 2016, 11:53 a.m., Shwetha GS wrote:
> > typesystem/src/main/java/org/apache/atlas/typesystem/types/cache/DefaultTypeCacheProvider.java,
> >  line 64
> > 
> >
> > I don't see any usages of this except in tests, can you remove this 
> > interface?

This was added to later address to further tweak the calls to cache provider 
from TypeSystem. In the sense, an implementer can chose to provide a better 
performing cache provider if the types can be partitioned based on their 
category. TypeSystem code, at many places, has the type category info, so there 
is a chance for further refactoring those call points and the 
DefaultTypeCacheProvider.

I know it is not needed to code for future, but we plan to do that soon after 
this JIRA goes through.

> On May 16, 2016, 11:53 a.m., Shwetha GS wrote:
> > typesystem/src/main/java/org/apache/atlas/typesystem/types/cache/DefaultTypeCacheProvider.java,
> >  line 110
> > 
> >
> > Not required?

Please see the comment above.

> On May 16, 2016, 11:53 a.m., Shwetha GS wrote:
> > typesystem/src/main/java/org/apache/atlas/typesystem/types/cache/DefaultTypeCacheProvider.java,
> >  line 124
> > 
> >
> > Faster to maintain another map like earlier?

Agreed. Please see my earlier comment (2 comments above). The plan is to 
further refactor calls made to type cache to lookup more narrowly based on 
category. As part, this comment should get addressed.

> On May 16, 2016, 11:53 a.m., Shwetha GS wrote:
> > typesystem/src/main/java/org/apache/atlas/typesystem/types/cache/DefaultTypeCacheProvider.java,
> >  line 125
> >

Build failed in Jenkins: apache-atlas-nightly #276

2016-05-17 Thread Apache Jenkins Server

See 

Changes:

[hyamijala] ATLAS-736 UI - BUG :: displaying timestamp values for hive_db

--
[...truncated 4830 lines...]
[INFO] --- keytool-maven-plugin:1.5:generateKeyPair (generateKeyPair) @ 
atlas-webapp ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @ 
atlas-webapp ---
[WARNING] Zinc server is not available at port 3030 - reverting to normal 
incremental compile
[INFO] Using incremental compilation
[INFO] Compiling 60 Java sources to 

[WARNING] Note: 

 uses or overrides a deprecated API.
[WARNING] Note: Recompile with -Xlint:deprecation for details.
[WARNING] Note: 

 uses unchecked or unsafe operations.
[WARNING] Note: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ atlas-webapp 
---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 60 source files to 

[WARNING] 
:
 

 uses or overrides a deprecated API.
[WARNING] 
:
 Recompile with -Xlint:deprecation for details.
[WARNING] 
:
 

 uses unchecked or unsafe operations.
[WARNING] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- scala-maven-plugin:3.2.0:testCompile (scala-test-compile-first) @ 
atlas-webapp ---
[WARNING] Zinc server is not available at port 3030 - reverting to normal 
incremental compile
[INFO] Using incremental compilation
[INFO] Compiling 42 Java sources to 

[WARNING] Note: Some input files use unchecked or unsafe operations.
[WARNING] Note: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ 
atlas-webapp ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 42 source files to 

[WARNING] 
:
 Some input files use unchecked or unsafe operations.
[WARNING] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-war-plugin:2.6:war (default-war) @ atlas-webapp ---
[INFO] Packaging webapp
[INFO] Assembling webapp [atlas-webapp] in 
[
[INFO] Processing overlay [ id org.apache.atlas:atlas-dashboardv2]
[INFO] Processing war project
[INFO] Copying webapp resources 
[
[INFO] Building jar: 

[INFO] Webapp assembled in [1629 msecs]
[INFO] Building war: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @

Re: Review Request 47225: ATLAS-713 Atlas UI is still showing the deleted column of a hive table (Entity response of hive table does not have it). - with browser cache disabled.

2016-05-17 Thread Suma Shivaprasad


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47225/#review133548
---


Ship it!




Ship It!

- Suma Shivaprasad


On May 17, 2016, 8:15 a.m., Shwetha GS wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47225/
> ---
> 
> (Updated May 17, 2016, 8:15 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-713
> https://issues.apache.org/jira/browse/ATLAS-713
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Current lineage APIs work on the dataset name. With soft delete, there can be 
> more than one entities for the given dataset name. 
> 1. Added new lineage APIs(input graph, output graph, schema) which work on 
> entity id
> 2. Modified current lineage APIs to work on the active entity for the given 
> dataset name
> 3. Modified UI to use the new APIs
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/atlas/AtlasClient.java b3ec95c 
>   dashboardv2/public/js/models/VLineage.js e33488a 
>   dashboardv2/public/js/models/VSchema.js 1f8e0bb 
>   dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js 87adec0 
>   dashboardv2/public/js/views/graph/LineageLayoutView.js 973d091 
>   dashboardv2/public/js/views/schema/SchemaLayoutView.js 35487a5 
>   distro/src/conf/atlas-application.properties 68a0021 
>   repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 
> 8dae968 
>   
> repository/src/main/java/org/apache/atlas/discovery/DataSetLineageService.java
>  PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/discovery/HiveLineageService.java 
> 00905d7 
>   repository/src/main/scala/org/apache/atlas/query/ClosureQuery.scala 05dc6a4 
>   repository/src/test/java/org/apache/atlas/BaseHiveRepositoryTest.java 
> 66e1365 
>   repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 
> PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/discovery/DataSetLineageServiceTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
>  3b50dfb 
>   
> repository/src/test/java/org/apache/atlas/discovery/HiveLineageServiceTest.java
>  6d5a15a 
>   repository/src/test/scala/org/apache/atlas/query/GremlinTest2.scala 2fd8bb9 
>   server-api/src/main/java/org/apache/atlas/discovery/LineageService.java 
> 8dc36cd 
>   typesystem/src/main/resources/atlas-application.properties aafad0f 
>   
> webapp/src/main/java/org/apache/atlas/web/resources/DataSetLineageResource.java
>  PRE-CREATION 
>   
> webapp/src/main/java/org/apache/atlas/web/resources/HiveLineageResource.java 
> 9b3fbc9 
>   webapp/src/main/java/org/apache/atlas/web/resources/LineageResource.java 
> PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/DataSetLineageJerseyResourceIT.java
>  PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/HiveLineageJerseyResourceIT.java
>  bc02f90 
> 
> Diff: https://reviews.apache.org/r/47225/diff/
> 
> 
> Testing
> ---
> 
> UTs, ITs and end to end test with hive cli
> 
> 
> Thanks,
> 
> Shwetha GS
> 
>

Re: Review Request 46330: Refactor local type-system cache with cache provider interface

2016-05-17 Thread venkata madugundu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46330/
---

(Updated May 17, 2016, 2:55 p.m.)


Review request for atlas, David Kantor and Hemanth Yamijala.


Changes
---

Attaching latest patch after addressing some of the review comments given by 
Shwetha.


Bugs: ATLAS-683
https://issues.apache.org/jira/browse/ATLAS-683


Repository: atlas


Description
---

The local type-cache has been carved out as an interface ITypeCacheProvider and 
the TypeSystem code has been refactored to use an instance of 
ITypeCacheProvider injected through Guice. An instance of ITypeCacheProvider is 
created by looking up the implementation class specified in 
atlas-application.properties (property - atlas.typesystem.cache.provider). The 
default implementation is a local cache implemented by DefaultTypeCacheProvider.


Diffs (updated)
-

  .gitignore f5899e2069d29d71378cabe51900c120db19eef9 
  common/src/main/java/org/apache/atlas/AtlasRuntimeException.java PRE-CREATION 
  distro/src/conf/atlas-application.properties 
68a002115bdda99fba4dcace5ab97399c59289d9 
  typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 
7763ebc39593d2ba747bd53e861f1d35657c8ea1 
  
typesystem/src/main/java/org/apache/atlas/typesystem/types/cache/DefaultTypeCacheProvider.java
 PRE-CREATION 
  
typesystem/src/main/java/org/apache/atlas/typesystem/types/cache/ITypeCacheProvider.java
 PRE-CREATION 
  
typesystem/src/test/java/org/apache/atlas/typesystem/types/cache/DefaultTypeCacheProviderTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/46330/diff/


Testing
---

Unit tests for the local cache DefaultTypeCacheProvider have been authored and 
all other affected unit tests of TypeSystem interface are modified to function 
in the presence of a type cache provider. Ran all tests of Atlas.


Thanks,

venkata madugundu

Re: Review Request 46197: ATLAS-658 : Improve Lineage with Backbone porting

2016-05-17 Thread keval bhatt


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46197/
---

(Updated May 17, 2016, 2:20 p.m.)


Review request for atlas, Erik Bergenholtz, Gautam Borad, Mehul Parikh, Nixon 
Rodrigues, and Hemanth Yamijala.


Changes
---

* Tried with your given query and its working fine.
* Removed hardcoded "Table" check for icons.


Bugs: ATLAS-658
https://issues.apache.org/jira/browse/ATLAS-658


Repository: atlas


Description
---

Improved display of vertices and edges in lineage graph that gets generated.


Diffs (updated)
-

  dashboardv2/public/css/style.css d1d23cc 
  dashboardv2/public/img/icon-gear-active.png PRE-CREATION 
  dashboardv2/public/img/icon-table-active.png PRE-CREATION 
  dashboardv2/public/js/collection/BaseCollection.js 2018019 
  dashboardv2/public/js/collection/VCommonList.js d9e1f52 
  dashboardv2/public/js/collection/VSchemaList.js 6a5c9d6 
  dashboardv2/public/js/collection/VTagList.js 9fa48bf 
  dashboardv2/public/js/modules/Helpers.js 4deb19b 
  dashboardv2/public/js/templates/common/TableLayout_tmpl.html 4264035 
  dashboardv2/public/js/templates/common/modal.html 0f64f5a 
  dashboardv2/public/js/templates/detail_page/DetailPageLayoutView_tmpl.html 
3a8eb44 
  dashboardv2/public/js/templates/graph/LineageLayoutView_tmpl.html 0c32a48 
  dashboardv2/public/js/templates/schema/SchemaTableLayoutView_tmpl.html 
cbc0e8a 
  dashboardv2/public/js/templates/tag/addTagModalView_tmpl.html acae4e1 
  dashboardv2/public/js/utils/Overrides.js 3b598e3 
  dashboardv2/public/js/utils/Utils.js 3892c72 
  dashboardv2/public/js/views/asset/AssetPageLayoutView.js 6334790 
  dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js 87adec0 
  dashboardv2/public/js/views/graph/LineageLayoutView.js 973d091 
  dashboardv2/public/js/views/schema/SchemaLayoutView.js de558a7 
  dashboardv2/public/js/views/tag/TagLayoutView.js 42791b9 
  dashboardv2/public/js/views/tag/addTagModalView.js d7e7c67 
  dashboardv2/public/js/views/tag/createTagsLayoutView.js 5d93eee 

Diff: https://reviews.apache.org/r/46197/diff/


Testing
---

1) mvn tests
2) Loaded the new UI in local sandbox and verified Lineage graph generation 
with proper vcertices and edges shown based on input and output of API calls.


Thanks,

keval bhatt

[jira] [Updated] (ATLAS-658) Improve Lineage with Backbone porting

2016-05-17 Thread Keval Bhatt (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-658:
--
Attachment: ATLAS-658.2.patch

> Improve Lineage with Backbone porting 
> --
>
> Key: ATLAS-658
> URL: https://issues.apache.org/jira/browse/ATLAS-658
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-658.2.patch, ATLAS-658.patch, 
> atlas_658_hive_lineage_issues.png
>
>
> Improved display of vertices and edges in lineage graph that gets generated. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (ATLAS-630) Network partition of active instance before first time initialization completes could cause a split-brain scenario

2016-05-17 Thread Hemanth Yamijala (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hemanth Yamijala reassigned ATLAS-630:
--

Assignee: Hemanth Yamijala

> Network partition of active instance before first time initialization 
> completes could cause a split-brain scenario
> --
>
> Key: ATLAS-630
> URL: https://issues.apache.org/jira/browse/ATLAS-630
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Hemanth Yamijala
>Assignee: Hemanth Yamijala
> Fix For: 0.7-incubating
>
>
> Start HA in active mode for the first time - when all dependent Graph stores 
> are being initialized the first time.
> Before the initialization completes, create a network partition such that the 
> active instance loses connection to Zookeeper. This could cause a split brain 
> scenario where the active instance does not stop the initialization and 
> another instance becomes active and starts initialization in parallel.
> This is only an issue for the very first initialization and the probability 
> of it happening is possibly very low.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (ATLAS-654) Storm hook downloads dependencies repeatedly

2016-05-17 Thread Hemanth Yamijala (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hemanth Yamijala resolved ATLAS-654.

Resolution: Fixed

With ATLAS-689 in, I don't see this issue anymore. Closing it as fixed / 
contained by. Please feel free to reopen if this is not the case.

> Storm hook downloads dependencies repeatedly
> 
>
> Key: ATLAS-654
> URL: https://issues.apache.org/jira/browse/ATLAS-654
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>
> It appears the Storm hook is downloading dependencies repeatedly, thus 
> increasing build time. Filing this to see if we can avoid this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-801) Atlas hooks would lose messages if Kafka is down for extended period of time

2016-05-17 Thread Hemanth Yamijala (JIRA)

Hemanth Yamijala created ATLAS-801:
--

 Summary: Atlas hooks would lose messages if Kafka is down for 
extended period of time
 Key: ATLAS-801
 URL: https://issues.apache.org/jira/browse/ATLAS-801
 Project: Atlas
  Issue Type: Improvement
Reporter: Hemanth Yamijala


All integration hooks in Atlas write messages to Kafka which are picked up by 
the Atlas server. This can be mitigated to some extent using multiple replicas 
for Kafka topics (see ATLAS-515). This JIRA is to see if we can make this even 
more robust and have some form of store and forward mechanism for increased 
fault tolerance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-800) Embedded Solr5 does not start on Windows7

2016-05-17 Thread venkata madugundu (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

venkata madugundu updated ATLAS-800:

Attachment: application.log

The generated application.log

> Embedded Solr5 does not start on Windows7
> -
>
> Key: ATLAS-800
> URL: https://issues.apache.org/jira/browse/ATLAS-800
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
> Environment: Windows7
>Reporter: venkata madugundu
>  Labels: startup
> Fix For: 0.7-incubating
>
> Attachments: application.log
>
>
> In latest Atlas, embedded Solr5 does not start. Tried disabling it and 
> enabling Elasticsearch, and Atlas started working fine.
> 
> java.lang.ExceptionInInitializerError
>   at java.lang.J9VMInternals.ensureError(J9VMInternals.java:137)
>   at 
> java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:126)
>   at 
> org.apache.atlas.repository.graph.DeleteHandler.(DeleteHandler.java:50)
>   at java.lang.Class.forNameImpl(Native Method)
>   at java.lang.Class.forName(Class.java:273)
>   at 
> org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:99)
>   at 
> org.apache.atlas.RepositoryMetadataModule.getDeleteHandler(RepositoryMetadataModule.java:114)
>   at 
> org.apache.atlas.RepositoryMetadataModule.configure(RepositoryMetadataModule.java:90)
>   
>   ...
>   at org.apache.atlas.Atlas.main(Atlas.java:113)
>   
>   ...
>   ...
>   REST OF THE EXCEPTION STACK
>   ...
>   ...
> Caused by: org.apache.solr.common.SolrException: Cannot connect to cluster at 
> localhost:2181: cluster not found/not ready
>   at 
> org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:344)
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:458)
>   at 
> com.thinkaurelius.titan.diskstorage.solr.Solr5Index.(Solr5Index.java:211)
>   ... 48 more
> 
> Attaching the log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-800) Embedded Solr5 does not start on Windows7

2016-05-17 Thread venkata madugundu (JIRA)

venkata madugundu created ATLAS-800:
---

 Summary: Embedded Solr5 does not start on Windows7
 Key: ATLAS-800
 URL: https://issues.apache.org/jira/browse/ATLAS-800
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.7-incubating
 Environment: Windows7
Reporter: venkata madugundu
 Fix For: 0.7-incubating


In latest Atlas, embedded Solr5 does not start. Tried disabling it and enabling 
Elasticsearch, and Atlas started working fine.


java.lang.ExceptionInInitializerError
at java.lang.J9VMInternals.ensureError(J9VMInternals.java:137)
at 
java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:126)
at 
org.apache.atlas.repository.graph.DeleteHandler.(DeleteHandler.java:50)
at java.lang.Class.forNameImpl(Native Method)
at java.lang.Class.forName(Class.java:273)
at 
org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:99)
at 
org.apache.atlas.RepositoryMetadataModule.getDeleteHandler(RepositoryMetadataModule.java:114)
at 
org.apache.atlas.RepositoryMetadataModule.configure(RepositoryMetadataModule.java:90)

...
at org.apache.atlas.Atlas.main(Atlas.java:113)

...
...
REST OF THE EXCEPTION STACK
...
...

Caused by: org.apache.solr.common.SolrException: Cannot connect to cluster at 
localhost:2181: cluster not found/not ready
at 
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:344)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:458)
at 
com.thinkaurelius.titan.diskstorage.solr.Solr5Index.(Solr5Index.java:211)
... 48 more


Attaching the log.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-799) UI: Searching for a tag should never be a full text search.

2016-05-17 Thread Hemanth Yamijala (JIRA)

Hemanth Yamijala created ATLAS-799:
--

 Summary: UI: Searching for a tag should never be a full text 
search.
 Key: ATLAS-799
 URL: https://issues.apache.org/jira/browse/ATLAS-799
 Project: Atlas
  Issue Type: Bug
Reporter: Hemanth Yamijala


* Create a tag.
* Click on it from the assets page.
* This is firing a full text search with the tag, which doesn't make sense. 

Expected result is that this should always result in a DSL ISA search.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 47293: ATLAS-736 :: UI - BUG :: displaying timestamp values for hive_db description

2016-05-17 Thread Hemanth Yamijala


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47293/#review133531
---


Ship it!




Ship It!

- Hemanth Yamijala


On May 13, 2016, 10:39 a.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47293/
> ---
> 
> (Updated May 13, 2016, 10:39 a.m.)
> 
> 
> Review request for atlas, Ayub Pathan, Gautam Borad, Shwetha GS, Selvamohan 
> Neethiraj, and Hemanth Yamijala.
> 
> 
> Bugs: ATLAS-736
> https://issues.apache.org/jira/browse/ATLAS-736
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Bug Description : Showing date in description if description is null same for 
> other colums
> 
> Changed: * Validation added to handle null value and showing empty value.
>  * Removed Tools column for add tag and add tag button displayed 
> after tags.
>  * Search table loading effect improvements.
>  * Header Menu fix.
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/css/override.css 4c452cc 
>   dashboardv2/public/index.html c2f15c9 
>   dashboardv2/public/js/utils/CommonViewFunction.js PRE-CREATION 
>   dashboardv2/public/js/views/asset/AssetPageLayoutView.js c86fef4 
>   dashboardv2/public/js/views/schema/SchemaLayoutView.js 35487a5 
>   dashboardv2/public/js/views/site/Header.js 500bd91 
>   dashboardv2/public/js/views/tag/TagDetailTableLayoutView.js c5e1ace 
>   dashboardv2/public/js/views/tag/addTagModalView.js d663ae2 
> 
> Diff: https://reviews.apache.org/r/47293/diff/
> 
> 
> Testing
> ---
> 
> Manually tested the changes.
> 
> 
> Thanks,
> 
> keval bhatt
> 
>

[jira] [Created] (ATLAS-798) UI: Search of a tag with empty results is resulting in a stuck spinner.

2016-05-17 Thread Hemanth Yamijala (JIRA)

Hemanth Yamijala created ATLAS-798:
--

 Summary: UI: Search of a tag with empty results is resulting in a 
stuck spinner.
 Key: ATLAS-798
 URL: https://issues.apache.org/jira/browse/ATLAS-798
 Project: Atlas
  Issue Type: Bug
Reporter: Hemanth Yamijala


* Create a tag.
* With search type as Text, select the Tag from the UI screen. (Same happens 
with DSL search as well, if the result set is empty)
* The results from the API comes back with empty results as below.
{code}
{
"requestId": "qtp142805395-1502 - a6928cc0-8f79-49e1-a1e5-354ddedae094",
"query": "tag2",
"queryType": "full-text",
"count": 0,
"results": []
}
{code}

The result is a stuck spinner on the UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-797) Tags are not disassociating correctly from entities

2016-05-17 Thread Hemanth Yamijala (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286606#comment-15286606
 ] 

Hemanth Yamijala commented on ATLAS-797:


json-response.txt has the response showing two same entities associated to the 
same tag.

> Tags are not disassociating correctly from entities
> ---
>
> Key: ATLAS-797
> URL: https://issues.apache.org/jira/browse/ATLAS-797
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>Priority: Blocker
> Attachments: json-response.txt
>
>
> * Create a tag
> * Attach it to an entity
> * Delete the tag from an entity page. The JSON response seems to be OK and 
> the entry gets removed.
> * Go to the assets page and do an ISA query again for the tag
> Expected result: Should get not results back.
> Actual result: See that the entity is being returned back in the result set.
> Further, adding the tag again to the same entity and doing a DSL query is 
> showing it up twice. Will attach JSON response returned.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-797) Tags are not disassociating correctly from entities

2016-05-17 Thread Hemanth Yamijala (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hemanth Yamijala updated ATLAS-797:
---
Attachment: json-response.txt

> Tags are not disassociating correctly from entities
> ---
>
> Key: ATLAS-797
> URL: https://issues.apache.org/jira/browse/ATLAS-797
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>Priority: Blocker
> Attachments: json-response.txt
>
>
> * Create a tag
> * Attach it to an entity
> * Delete the tag from an entity page. The JSON response seems to be OK and 
> the entry gets removed.
> * Go to the assets page and do an ISA query again for the tag
> Expected result: Should get not results back.
> Actual result: See that the entity is being returned back in the result set.
> Further, adding the tag again to the same entity and doing a DSL query is 
> showing it up twice. Will attach JSON response returned.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 47382: ATLAS-491 : Business Catalog API

2016-05-17 Thread Hemanth Yamijala



> On May 16, 2016, 6:41 a.m., Hemanth Yamijala wrote:
> >
> 
> John Speidel wrote:
> ok, I will use the other constructor along with calling of the necessary 
> setters.
> But to be honest, why this constructor should be marked this way makes 
> absolutely no sense to me.

I agree and we could just remove the annotation I think, if the usage seems 
harmless.


- Hemanth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47382/#review11
---


On May 14, 2016, 12:58 a.m., John Speidel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47382/
> ---
> 
> (Updated May 14, 2016, 12:58 a.m.)
> 
> 
> Review request for atlas and Hemanth Yamijala.
> 
> 
> Bugs: ATLAS-491
> https://issues.apache.org/jira/browse/ATLAS-491
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Initial implementation of Catalog API.
> 
> Known Issues/Limitations:
> - No update/delete support for taxonomies/terms
>   -- this will be added next week
> - Some gaps in test coverage
>   - Need more in-depth test coverage that exercise the generated gremlin 
> pipeline
> - some refactoring should still be done espcecially in the 'query' package
> - need to add more logging
> - need to finish some javadoc
> 
> 
> Diffs
> -
> 
>   catalog/pom.xml PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/AtlasTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/CollectionRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultDateFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultPropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/EntityResourceProvider.java 
> PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/EntityTagResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/InstanceRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/JsonSerializer.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyValueFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Request.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceComparator.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Result.java PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/TaxonomyResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermPath.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermVertexWrapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/VertexWrapper.java 
> PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/BaseResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/EntityResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/EntityTagResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/ResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/TaxonomyResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/TermResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/CatalogException.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/CatalogRuntimeException.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/InvalidPayloadException.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/InvalidQueryException.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/ResourceAlreadyExistsException.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/exception/ResourceNotFoundException.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/projection/GenericRelation.java
>

Re: Review Request 47382: ATLAS-491 : Business Catalog API

2016-05-17 Thread Hemanth Yamijala



> On May 16, 2016, 6:59 a.m., Hemanth Yamijala wrote:
> > catalog/src/main/java/org/apache/atlas/catalog/query/AtlasEntityQuery.java, 
> > line 37
> > 
> >
> > If I understood this correctly, the basepipe for the Gremlin search for 
> > entities begins by trying to identify all entity vertices (by virtue of 
> > using the full text search property that is common to all vertices). This 
> > would still be a huge result set to begin with, no?
> 
> John Speidel wrote:
> This is currently the only way to identify an entity vertex with the 
> current database model.  It is a requirement to be able to search all entity 
> types and to be able to apply tags to entities across types. So, we need to 
> be able to get all of the entity vertices, which is the set that would be 
> returned from the base query. The biggest issue with using this field to 
> determine the entity set is that it isn't indexed.  Also, the query starts 
> with V(), which is all vertices in the graph but this doesn't result in all 
> vertices being returned but instead adds them to the pipeline interator.  
> 
> That being said, there are lots of improvements that can be made to 
> improve query performance.  This would include, changes to the database model 
> so that we knew the propery names ahead of time so that we could use an index 
> instead of a filter.  Also, one optimization that I was going to add which 
> didn't make it in was to use the type if it was provided so that indexes 
> could be used as mentioned above.  For now, I don't think that we can improve 
> on this.

I thought we do index the ENTITY_TEXT_PROPERTY_KEY. This is done in 
GraphBackedSearchIndexer.createFullTextIndex. Please see comment below on 
search and we can probably get this in and open follow-up JIRAs to fix.


- Hemanth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47382/#review12
---


On May 14, 2016, 12:58 a.m., John Speidel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47382/
> ---
> 
> (Updated May 14, 2016, 12:58 a.m.)
> 
> 
> Review request for atlas and Hemanth Yamijala.
> 
> 
> Bugs: ATLAS-491
> https://issues.apache.org/jira/browse/ATLAS-491
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Initial implementation of Catalog API.
> 
> Known Issues/Limitations:
> - No update/delete support for taxonomies/terms
>   -- this will be added next week
> - Some gaps in test coverage
>   - Need more in-depth test coverage that exercise the generated gremlin 
> pipeline
> - some refactoring should still be done espcecially in the 'query' package
> - need to add more logging
> - need to finish some javadoc
> 
> 
> Diffs
> -
> 
>   catalog/pom.xml PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/AtlasTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/CollectionRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultDateFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultPropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/EntityResourceProvider.java 
> PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/EntityTagResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/InstanceRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/JsonSerializer.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyValueFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Request.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceComparator.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Result.java PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/TaxonomyResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermPath.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermVertexWrapper.java 
>

Re: Review Request 47382: ATLAS-491 : Business Catalog API

2016-05-17 Thread Hemanth Yamijala



> On May 16, 2016, 5:40 a.m., Hemanth Yamijala wrote:
> > catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java, line 
> > 57
> > 
> >
> > This would be a problem if multiple threads are trying to define the 
> > type at the same time (as could happen if multiple clients are trying to 
> > create taxonomies / terms at the same time). We should move type 
> > registration to a one time activity as the rest of the framework.
> 
> John Speidel wrote:
> ok, I will investigate this but I didn't see a way to do this without 
> making changes to Atlas proper, which was a desing goal based on team input.

John, we do one time model registration of other models like Hive, Sqoop, Storm 
etc. These are done one time when the (active) server starts up. The method was 
introduced in ATLAS-512. Broadly it involves generating a model definition and 
writing it out as a json file as part of the build process. This is then 
packaged under the models directory and is picked up automatically by the 
server at start up. If you could go through the patch in ATLAS-512 and follow 
the same steps we should be good for fixing this.


- Hemanth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47382/#review133328
---


On May 14, 2016, 12:58 a.m., John Speidel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47382/
> ---
> 
> (Updated May 14, 2016, 12:58 a.m.)
> 
> 
> Review request for atlas and Hemanth Yamijala.
> 
> 
> Bugs: ATLAS-491
> https://issues.apache.org/jira/browse/ATLAS-491
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Initial implementation of Catalog API.
> 
> Known Issues/Limitations:
> - No update/delete support for taxonomies/terms
>   -- this will be added next week
> - Some gaps in test coverage
>   - Need more in-depth test coverage that exercise the generated gremlin 
> pipeline
> - some refactoring should still be done espcecially in the 'query' package
> - need to add more logging
> - need to finish some javadoc
> 
> 
> Diffs
> -
> 
>   catalog/pom.xml PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/AtlasTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/BaseResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/CollectionRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultDateFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultPropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/EntityResourceProvider.java 
> PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/EntityTagResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/InstanceRequest.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/JsonSerializer.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyMapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/PropertyValueFormatter.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Request.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceComparator.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/ResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/Result.java PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/TaxonomyResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermPath.java PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermResourceProvider.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/TermVertexWrapper.java 
> PRE-CREATION 
>   catalog/src/main/java/org/apache/atlas/catalog/VertexWrapper.java 
> PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/BaseResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/EntityResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/EntityTagResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/ResourceDefinition.java
>  PRE-CREATION 
>   
> catalog/src/main/java/org/apache/atlas/catalog/definition/TaxonomyResourceDefinition.java
>  PRE-CREATION 
>

Re: Review Request 46330: Refactor local type-system cache with cache provider interface

2016-05-17 Thread venkata madugundu



> On May 17, 2016, 5:58 a.m., Shwetha GS wrote:
> > While addressing the comments, can you also re-base the patch to trunk. 
> > Sorry about that. Will review as soon as the patch is updated and will 
> > close it soon

Thanks Shwetha for the review. I have started addressing the review comments. 
Will update the review board once I am done with the changes.


- venkata


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46330/#review133497
---


On April 28, 2016, 12:59 p.m., venkata madugundu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46330/
> ---
> 
> (Updated April 28, 2016, 12:59 p.m.)
> 
> 
> Review request for atlas, David Kantor and Hemanth Yamijala.
> 
> 
> Bugs: ATLAS-683
> https://issues.apache.org/jira/browse/ATLAS-683
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> The local type-cache has been carved out as an interface ITypeCacheProvider 
> and the TypeSystem code has been refactored to use an instance of 
> ITypeCacheProvider injected through Guice. An instance of ITypeCacheProvider 
> is created by looking up the implementation class specified in 
> atlas-application.properties (property - atlas.typesystem.cache.provider). 
> The default implementation is a local cache implemented by 
> DefaultTypeCacheProvider.
> 
> 
> Diffs
> -
> 
>   .gitignore ff7c32e9280a2b1fdafc99fe8113c4d82452318e 
>   common/src/main/java/org/apache/atlas/AtlasRuntimeException.java 
> PRE-CREATION 
>   distro/src/conf/atlas-application.properties 
> 290105f2374c3c2e7fadf601d26993081e344974 
>   repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 
> 8dae9686d0cb698577e6028758ae8eb682823995 
>   repository/src/test/java/org/apache/atlas/repository/BaseTest.java 
> 7e1df676c1502d14df2af4fc90513009fdafc632 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  ae215f99080047c6fe45b55ae0c5ad7c1ba600e2 
>   
> repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java
>  961442bbcaff86b472a876e6c348aa9e65c1efca 
>   repository/src/test/scala/org/apache/atlas/query/GremlinTest.scala 
> b23c0f60ace59fa15886c94f35e35d3113872783 
>   repository/src/test/scala/org/apache/atlas/query/LineageQueryTest.scala 
> 0e0ac8633a256dc7c8966c2b8d52eb942b52e1c6 
>   typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 
> 7763ebc39593d2ba747bd53e861f1d35657c8ea1 
>   
> typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystemProvider.java
>  4e1cd36bc1ccf6e770bca3d649b9203dbbc179a8 
>   
> typesystem/src/main/java/org/apache/atlas/typesystem/types/cache/DefaultTypeCacheProvider.java
>  PRE-CREATION 
>   
> typesystem/src/main/java/org/apache/atlas/typesystem/types/cache/ITypeCacheProvider.java
>  PRE-CREATION 
>   typesystem/src/test/java/org/apache/atlas/typesystem/types/BaseTest.java 
> 64dc7511a756f8f68519b48a38228696a866a7f8 
>   
> typesystem/src/test/java/org/apache/atlas/typesystem/types/HierarchicalTypeDependencySorterTest.java
>  19bdccfc6baf503f3144fce8fe0b85914693d054 
>   
> typesystem/src/test/java/org/apache/atlas/typesystem/types/TypeInheritanceTest.java
>  c13ef3a10d7efa2c6a8fa6c261a3402d633f08d4 
>   
> typesystem/src/test/java/org/apache/atlas/typesystem/types/cache/DefaultTypeCacheProviderTest.java
>  PRE-CREATION 
>   
> typesystem/src/test/scala/org/apache/atlas/typesystem/builders/BuilderTest.scala
>  9d1d00f1dc649101119a3eebfca72e9bb3378a3d 
>   
> typesystem/src/test/scala/org/apache/atlas/typesystem/builders/MultiplicityTest.scala
>  91e72c7d78e44d65f8fe8e2516d38c0da00ddd00 
>   
> typesystem/src/test/scala/org/apache/atlas/typesystem/json/InstanceSerializationTest.scala
>  98a97398cfc232a985e7dc7c83c57f00791c3cdb 
> 
> Diff: https://reviews.apache.org/r/46330/diff/
> 
> 
> Testing
> ---
> 
> Unit tests for the local cache DefaultTypeCacheProvider have been authored 
> and all other affected unit tests of TypeSystem interface are modified to 
> function in the presence of a type cache provider. Ran all tests of Atlas.
> 
> 
> Thanks,
> 
> venkata madugundu
> 
>

Re: Review Request 47225: ATLAS-713 Atlas UI is still showing the deleted column of a hive table (Entity response of hive table does not have it). - with browser cache disabled.

2016-05-17 Thread Shwetha GS


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47225/
---

(Updated May 17, 2016, 8:15 a.m.)


Review request for atlas.


Bugs: ATLAS-713
https://issues.apache.org/jira/browse/ATLAS-713


Repository: atlas


Description
---

Current lineage APIs work on the dataset name. With soft delete, there can be 
more than one entities for the given dataset name. 
1. Added new lineage APIs(input graph, output graph, schema) which work on 
entity id
2. Modified current lineage APIs to work on the active entity for the given 
dataset name
3. Modified UI to use the new APIs


Diffs
-

  client/src/main/java/org/apache/atlas/AtlasClient.java b3ec95c 
  dashboardv2/public/js/models/VLineage.js e33488a 
  dashboardv2/public/js/models/VSchema.js 1f8e0bb 
  dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js 87adec0 
  dashboardv2/public/js/views/graph/LineageLayoutView.js 973d091 
  dashboardv2/public/js/views/schema/SchemaLayoutView.js 35487a5 
  distro/src/conf/atlas-application.properties 68a0021 
  repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 
8dae968 
  
repository/src/main/java/org/apache/atlas/discovery/DataSetLineageService.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/discovery/HiveLineageService.java 
00905d7 
  repository/src/main/scala/org/apache/atlas/query/ClosureQuery.scala 05dc6a4 
  repository/src/test/java/org/apache/atlas/BaseHiveRepositoryTest.java 66e1365 
  repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 
PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/DataSetLineageServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
 3b50dfb 
  
repository/src/test/java/org/apache/atlas/discovery/HiveLineageServiceTest.java 
6d5a15a 
  repository/src/test/scala/org/apache/atlas/query/GremlinTest2.scala 2fd8bb9 
  server-api/src/main/java/org/apache/atlas/discovery/LineageService.java 
8dc36cd 
  typesystem/src/main/resources/atlas-application.properties aafad0f 
  
webapp/src/main/java/org/apache/atlas/web/resources/DataSetLineageResource.java 
PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/resources/HiveLineageResource.java 
9b3fbc9 
  webapp/src/main/java/org/apache/atlas/web/resources/LineageResource.java 
PRE-CREATION 
  
webapp/src/test/java/org/apache/atlas/web/resources/DataSetLineageJerseyResourceIT.java
 PRE-CREATION 
  
webapp/src/test/java/org/apache/atlas/web/resources/HiveLineageJerseyResourceIT.java
 bc02f90 

Diff: https://reviews.apache.org/r/47225/diff/


Testing
---

UTs, ITs and end to end test with hive cli


Thanks,

Shwetha GS

Re: Review Request 47382: ATLAS-491 : Business Catalog API

[jira] [Created] (ATLAS-802) New look UI to show Business Catalog functionalities

Re: Review Request 47382: ATLAS-491 : Business Catalog API

[jira] [Updated] (ATLAS-491) Business Catalog / Taxonomy

[jira] [Commented] (ATLAS-491) Business Catalog / Taxonomy

[jira] [Commented] (ATLAS-491) Business Catalog / Taxonomy

Re: Review Request 47382: ATLAS-491 : Business Catalog API

Re: Review Request 47382: ATLAS-491 : Business Catalog API

[jira] [Assigned] (ATLAS-774) Better error handling from login.jsp

Review Request 47458: ATLAS-774 - Error-Handling-from-LoginPage

[jira] [Resolved] (ATLAS-790) persist method on JavaRDDLike

[jira] [Updated] (ATLAS-683) Refactor local type-system cache with cache provider interface

Re: Review Request 46330: Refactor local type-system cache with cache provider interface

Build failed in Jenkins: apache-atlas-nightly #276

Re: Review Request 47225: ATLAS-713 Atlas UI is still showing the deleted column of a hive table (Entity response of hive table does not have it). - with browser cache disabled.

Re: Review Request 46330: Refactor local type-system cache with cache provider interface

Re: Review Request 46197: ATLAS-658 : Improve Lineage with Backbone porting

[jira] [Updated] (ATLAS-658) Improve Lineage with Backbone porting

[jira] [Assigned] (ATLAS-630) Network partition of active instance before first time initialization completes could cause a split-brain scenario

[jira] [Resolved] (ATLAS-654) Storm hook downloads dependencies repeatedly

[jira] [Created] (ATLAS-801) Atlas hooks would lose messages if Kafka is down for extended period of time

[jira] [Updated] (ATLAS-800) Embedded Solr5 does not start on Windows7

[jira] [Created] (ATLAS-800) Embedded Solr5 does not start on Windows7

[jira] [Created] (ATLAS-799) UI: Searching for a tag should never be a full text search.

Re: Review Request 47293: ATLAS-736 :: UI - BUG :: displaying timestamp values for hive_db description

[jira] [Created] (ATLAS-798) UI: Search of a tag with empty results is resulting in a stuck spinner.

[jira] [Commented] (ATLAS-797) Tags are not disassociating correctly from entities

[jira] [Updated] (ATLAS-797) Tags are not disassociating correctly from entities

Re: Review Request 47382: ATLAS-491 : Business Catalog API

Re: Review Request 47382: ATLAS-491 : Business Catalog API

Re: Review Request 47382: ATLAS-491 : Business Catalog API

Re: Review Request 46330: Refactor local type-system cache with cache provider interface

Re: Review Request 47225: ATLAS-713 Atlas UI is still showing the deleted column of a hive table (Entity response of hive table does not have it). - with browser cache disabled.

33 matches

Site Navigation

Mail list logo

Footer information