[jira] [Commented] (ATLAS-503) Not all Hive tables are not imported into Atlas when interrupted with search queries while importing.

[Hemanth Yamijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299499#comment-15299499
 ] 

Hemanth Yamijala commented on ATLAS-503:


This is just new tag creation. Not associating to any entity. One thing if you 
could help with: I tried to add a print to 
{{HBaseKeyColumnValueStore.acquireLock}} to print the key and column values as 
strings. It gets unprintable stuff like {{^HDq^Prt%__type.test}}. Do you know 
how to convert that byte array to something printable , if at all possible. I 
tried to do lots of tricks yesterday including String encodings / using HBase 
utilities etc. But none seemed to work. The only thing that's coming out is 
__type.test which could mean anything?

> Not all Hive tables are not imported into Atlas when interrupted with search 
> queries while importing.  
> ---
>
> Key: ATLAS-503
> URL: https://issues.apache.org/jira/browse/ATLAS-503
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sharmadha Sainath
>Assignee: Hemanth Yamijala
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: hiv2atlaslogs.rtf
>
>
> On running a file containing 100 table creation commands using beeline -f , 
> all hive tables are created. But only 81 of them are imported into Atlas 
> (HiveHook enabled) when queries like "hive_table" is searched frequently 
> while the import process for the table is going on.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-819) All user defined types should have a set of common attributes

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-819:
---
Assignee: Shwetha G S

> All user defined types should have a set of common attributes
> -
>
> Key: ATLAS-819
> URL: https://issues.apache.org/jira/browse/ATLAS-819
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Shwetha G S
>
> It would be very convenient if all user defined types have a conventional set 
> of common attributes including:
> * name
> * description
> * owner
> * created at
> * modified at



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-503) Not all Hive tables are not imported into Atlas when interrupted with search queries while importing.

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299451#comment-15299451
 ] 

Suma Shivaprasad commented on ATLAS-503:


[~yhemanth] Is it trying to create multiple tags on the same entity? If so, 
could it possibly be the lock being held onTRAIT_NAMES_PROPERTY_KEY  and/or the 
vertexId on which tags are being created?

> Not all Hive tables are not imported into Atlas when interrupted with search 
> queries while importing.  
> ---
>
> Key: ATLAS-503
> URL: https://issues.apache.org/jira/browse/ATLAS-503
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sharmadha Sainath
>Assignee: Hemanth Yamijala
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: hiv2atlaslogs.rtf
>
>
> On running a file containing 100 table creation commands using beeline -f , 
> all hive tables are created. But only 81 of them are imported into Atlas 
> (HiveHook enabled) when queries like "hive_table" is searched frequently 
> while the import process for the table is going on.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-824) isis security module, create local user

[Steve Cameron (JIRA)]

Steve Cameron created ATLAS-824:
---

 Summary: isis security module, create local user
 Key: ATLAS-824
 URL: https://issues.apache.org/jira/browse/ATLAS-824
 Project: Atlas
  Issue Type: Bug
Reporter: Steve Cameron






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-819) All user defined types should have a set of common attributes

[CASSIO DOS SANTOS (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299053#comment-15299053
 ] 

CASSIO DOS SANTOS commented on ATLAS-819:
-

Can you confirm that this is not going to be the default for "all user defined 
types" as stated in the description, but instead an optional base class, or 
rather a couple of optional base classes, one with the read-only system 
attributes and one with 'name' and 'description', as per Dave comments, with 
which I fully agree? 

We have such classes in our application and other applications are likely to 
have something similar, so making it optional would give more flexibility to 
application developers. 

Based on my experience, it's not uncommon to have classes for which even those 
read-only system attributes are not required or desired (unnecessary overhead), 
as they represent objects that may need to be referenced from multiple objects 
but are to be otherwise handled as "lightweight" sub-objects of another 
root/parent object. 

> All user defined types should have a set of common attributes
> -
>
> Key: ATLAS-819
> URL: https://issues.apache.org/jira/browse/ATLAS-819
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>
> It would be very convenient if all user defined types have a conventional set 
> of common attributes including:
> * name
> * description
> * owner
> * created at
> * modified at



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-693) Add titan 0.5.4 implemenation of graph database abstraction layer

[ATLAS QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298785#comment-15298785
 ] 

ATLAS QA commented on ATLAS-693:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12805945/rb47466.patch
  against master revision 74d9b3f.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified test files.

  {color:red}-1 javac{color}.  The applied patch generated 140 javac 
compiler warnings (more than the master's current 136 warnings).

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

+1 checkstyle.  The patch generated 0 code style errors.

{color:red}-1 findbugs{color}.  The patch appears to introduce 380 new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The test build failed in  

Test results: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningsrepository.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningscommon.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningsnotification.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningswebapp.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningsclient.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningstitan0.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningstypesystem.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningsfalcon-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningssqoop-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningshive-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningsstorm-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningshdfs-model.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningsauthorization.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/newPatchFindbugsWarningscatalog.html
Javac warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/249//artifact/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-ATLAS-Build/249//console

This message is automatically generated.

> Add titan 0.5.4 implemenation of graph database abstraction layer
> -
>
> Key: ATLAS-693
> URL: https://issues.apache.org/jira/browse/ATLAS-693
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 0.6-incubating
>Reporter: Jeffrey Hagelberg
>Assignee: Jeffrey Hagelberg
> Fix For: trunk
>
> Attachments: rb47466.patch
>
>
> In this task, we put in place the titan 0.5.4 implementation of the graph 
> database abstraction layer.  As part of this effort, the existing "titan" 
> maven project will be merged into this new project, since it is specific to 
> titan 0.5.4.  The maven projects in Atlas that depend on the "titan" will be 
> changed to depend on the new project.  Direct dependencies on titan 0.5.4 
> will be removed from all of the atlas projects.  These projects will be 
> changed to depend on this new project instead.  They will transitively obtain 
> the dependencies on titan 0.5.4 that they need.  For this task, very little 
> of the existing Atlas java / scala code will be changed.  The classes from in 
> the titan project will be moved verbatim into the new project.  The only 
> other changes will be the addition of the new classes that implement the 
> abstraction layer interfaces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 46731: ATLAS-716 Entity update/delete notifications

[Suma Shivaprasad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46731/#review134619
---


Ship it!




Ship It!

- Suma Shivaprasad


On May 24, 2016, 7:34 a.m., Shwetha GS wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46731/
> ---
> 
> (Updated May 24, 2016, 7:34 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-716
> https://issues.apache.org/jira/browse/ATLAS-716
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1. Update/delete entity APIs return list of entities created, updated and 
> deleted
> 2. Listeners are called for all the affected entities
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  254e150 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> 70100f1 
>   client/src/main/java/org/apache/atlas/AtlasClient.java 7e32cc2 
>   client/src/main/java/org/apache/atlas/EntityAuditEvent.java 460f708 
>   client/src/main/java/org/apache/atlas/SerDe.java PRE-CREATION 
>   client/src/test/java/org/apache/atlas/AtlasClientTest.java 0e80573 
>   
> notification/src/main/java/org/apache/atlas/notification/MessageVersion.java 
> 3f16a9a 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java
>  384f383 
>   
> notification/src/main/java/org/apache/atlas/notification/VersionedMessageDeserializer.java
>  290be59 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/NotificationEntityChangeListener.java
>  300cbb5 
>   
> notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
>  e8b55ef 
>   
> repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 
> 540c308 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
>  5b4bdbf 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  8f11322 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> a9e4f39 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  3604277 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> cccafc2 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
>  df28ab3 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/HardDeleteHandler.java
>  f8bbf73 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/SoftDeleteHandler.java
>  aa78582 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  a017536 
>   
> repository/src/main/java/org/apache/atlas/repository/typestore/GraphBackedTypeStore.java
>  3fb128c 
>   
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
>  7cd83f8 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  be407a5 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  1aeedb5 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositoryHardDeleteTest.java
>  d2109d3 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositorySoftDeleteTest.java
>  d9e3ec9 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  1f906ed 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java b1d87ea 
>   server-api/src/main/java/org/apache/atlas/services/MetadataService.java 
> c8c1067 
>   typesystem/src/main/java/org/apache/atlas/typesystem/Referenceable.java 
> 31f157e 
>   webapp/src/main/java/org/apache/atlas/LocalAtlasClient.java c6ed85d 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationEntityChangeListener.java
>  PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
> 709fec5 
>   webapp/src/main/java/org/apache/atlas/web/service/ServiceModule.java 
> 0f8bcb1 
>   webapp/src/test/java/org/apache/atlas/LocalAtlasClientTest.java 7f20652 
>   
> webapp/src/test/java/org/apache/atlas/notification/EntityNotificationIT.java 
> 6985152 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationEntityChangeListenerTest.java
>  PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java
>  aa92bc0 
>   webapp/src/test/java/org/apache/atlas/web/service/CuratorFactoryTest.java 
> 2d510a0 
> 
> Diff: https://reviews.apache.org/r/46731/diff/
> 
> 
> Testing
> ---
> 
> UTs
> 
> 
> Thanks,
> 
> Shwetha GS
> 
>

Re: Review Request 47638: ATLAS-752 Column rename in Hive should retain traits/tags associated with it

[Suma Shivaprasad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47638/#review134618
---


Ship it!




Ship It!

- Suma Shivaprasad


On May 23, 2016, 6:52 a.m., Vimal Sharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47638/
> ---
> 
> (Updated May 23, 2016, 6:52 a.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-752
> https://issues.apache.org/jira/browse/ATLAS-752
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-752 When a column is renamed, the traits associted with it are lost. 
> The traits should be retained with the new column
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
> 418e755 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> 70100f1 
> 
> Diff: https://reviews.apache.org/r/47638/diff/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> Revised Patch
>   
> https://reviews.apache.org/media/uploaded/files/2016/05/20/a3ab9b8c-4ffc-4ec6-92d7-1f84db246c6a__ATLAS-752-v3.patch
> 
> 
> Thanks,
> 
> Vimal Sharma
> 
>

Re: Review Request 47638: ATLAS-752 Column rename in Hive should retain traits/tags associated with it

[Suma Shivaprasad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47638/#review134617
---




addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
(line 742)


can you pls add a test with column position changed as well


- Suma Shivaprasad


On May 23, 2016, 6:52 a.m., Vimal Sharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47638/
> ---
> 
> (Updated May 23, 2016, 6:52 a.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-752
> https://issues.apache.org/jira/browse/ATLAS-752
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-752 When a column is renamed, the traits associted with it are lost. 
> The traits should be retained with the new column
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
> 418e755 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> 70100f1 
> 
> Diff: https://reviews.apache.org/r/47638/diff/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> Revised Patch
>   
> https://reviews.apache.org/media/uploaded/files/2016/05/20/a3ab9b8c-4ffc-4ec6-92d7-1f84db246c6a__ATLAS-752-v3.patch
> 
> 
> Thanks,
> 
> Vimal Sharma
> 
>

Re: Review Request 47638: ATLAS-752 Column rename in Hive should retain traits/tags associated with it

[Suma Shivaprasad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47638/#review134616
---




addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java (line 
381)


Can we overload this function instead of making it null?


- Suma Shivaprasad


On May 23, 2016, 6:52 a.m., Vimal Sharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47638/
> ---
> 
> (Updated May 23, 2016, 6:52 a.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-752
> https://issues.apache.org/jira/browse/ATLAS-752
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-752 When a column is renamed, the traits associted with it are lost. 
> The traits should be retained with the new column
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
> 418e755 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> 70100f1 
> 
> Diff: https://reviews.apache.org/r/47638/diff/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> Revised Patch
>   
> https://reviews.apache.org/media/uploaded/files/2016/05/20/a3ab9b8c-4ffc-4ec6-92d7-1f84db246c6a__ATLAS-752-v3.patch
> 
> 
> Thanks,
> 
> Vimal Sharma
> 
>

[jira] [Updated] (ATLAS-693) Add titan 0.5.4 implemenation of graph database abstraction layer

[Jeffrey Hagelberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeffrey Hagelberg updated ATLAS-693:

Attachment: rb47466.patch

> Add titan 0.5.4 implemenation of graph database abstraction layer
> -
>
> Key: ATLAS-693
> URL: https://issues.apache.org/jira/browse/ATLAS-693
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 0.6-incubating
>Reporter: Jeffrey Hagelberg
>Assignee: Jeffrey Hagelberg
> Fix For: trunk
>
> Attachments: rb47466.patch
>
>
> In this task, we put in place the titan 0.5.4 implementation of the graph 
> database abstraction layer.  As part of this effort, the existing "titan" 
> maven project will be merged into this new project, since it is specific to 
> titan 0.5.4.  The maven projects in Atlas that depend on the "titan" will be 
> changed to depend on the new project.  Direct dependencies on titan 0.5.4 
> will be removed from all of the atlas projects.  These projects will be 
> changed to depend on this new project instead.  They will transitively obtain 
> the dependencies on titan 0.5.4 that they need.  For this task, very little 
> of the existing Atlas java / scala code will be changed.  The classes from in 
> the titan project will be moved verbatim into the new project.  The only 
> other changes will be the addition of the new classes that implement the 
> abstraction layer interfaces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-823) Atlas should use external HBase and SOLR

[Tom Beerbower (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tom Beerbower updated ATLAS-823:

Description: 
Use of embedded HBase and SOLR is not preferred due to multiple concerns, 
including:
# lack of support by Ambari makes it difficult to handle kerberization, 
upgrade/migration, start/stop
# not suitable for production use - no HA, local-storage

Atlas should remove support for embedded HBase and SOLR. Instead Atlas should 
be updated to use external HBase and SOLR. 

  was:
Use of embedded HBase and SOLR is not preferred due to multiple concerns, 
including:
# lack of support by Ambari makes it difficult to handle kerberization, 
upgrade/migration, start/stop
# not suitable for production use - no HA, local-storage
Atlas should remove support for embedded HBase and SOLR. Instead Atlas should 
be updated to use external HBase and SOLR. 


> Atlas should use external HBase and SOLR
> 
>
> Key: ATLAS-823
> URL: https://issues.apache.org/jira/browse/ATLAS-823
> Project: Atlas
>  Issue Type: Bug
>Reporter: Tom Beerbower
>Assignee: Tom Beerbower
>
> Use of embedded HBase and SOLR is not preferred due to multiple concerns, 
> including:
> # lack of support by Ambari makes it difficult to handle kerberization, 
> upgrade/migration, start/stop
> # not suitable for production use - no HA, local-storage
> Atlas should remove support for embedded HBase and SOLR. Instead Atlas should 
> be updated to use external HBase and SOLR. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-823) Atlas should use external HBase and SOLR

[Tom Beerbower (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tom Beerbower updated ATLAS-823:

Description: 
Use of embedded HBase and SOLR is not preferred due to multiple concerns, 
including:
# lack of support by Ambari makes it difficult to handle kerberization, 
upgrade/migration, start/stop
# not suitable for production use - no HA, local-storage
Atlas should remove support for embedded HBase and SOLR. Instead Atlas should 
be updated to use external HBase and SOLR. 

  was:
Use of embedded HBase and SOLR is not preferred due to multiple concerns, 
including:
# lack of support by Ambari makes it difficult to handle kerberization, 
upgrade/migration, start/stop
# not suitable for production use - no HA, local-storage
Atlas should remove support for embedded HBase and SOLR. Instead Atlas should 
be updated to use only external HBase and SOLR. 


> Atlas should use external HBase and SOLR
> 
>
> Key: ATLAS-823
> URL: https://issues.apache.org/jira/browse/ATLAS-823
> Project: Atlas
>  Issue Type: Bug
>Reporter: Tom Beerbower
>Assignee: Tom Beerbower
>
> Use of embedded HBase and SOLR is not preferred due to multiple concerns, 
> including:
> # lack of support by Ambari makes it difficult to handle kerberization, 
> upgrade/migration, start/stop
> # not suitable for production use - no HA, local-storage
> Atlas should remove support for embedded HBase and SOLR. Instead Atlas should 
> be updated to use external HBase and SOLR. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-823) Atlas should use external HBase and SOLR

[Tom Beerbower (JIRA)]

Tom Beerbower created ATLAS-823:
---

 Summary: Atlas should use external HBase and SOLR
 Key: ATLAS-823
 URL: https://issues.apache.org/jira/browse/ATLAS-823
 Project: Atlas
  Issue Type: Bug
Reporter: Tom Beerbower
Assignee: Tom Beerbower


Use of embedded HBase and SOLR is not preferred due to multiple concerns, 
including:
# lack of support by Ambari makes it difficult to handle kerberization, 
upgrade/migration, start/stop
# not suitable for production use - no HA, local-storage
Atlas should remove support for embedded HBase and SOLR. Instead Atlas should 
be updated to use only external HBase and SOLR. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 47466: ATLAS-693: Add titan 0.5.4 implementation of graph database abstraction layer

[Jeff Hagelberg]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47466/
---

(Updated May 24, 2016, 4:44 p.m.)


Review request for atlas and David Kantor.


Changes
---

The previous diff accidentally undid the changes that were put in to address 
the code review comments.  I've added those back in.


Bugs: ATLAS-693
https://issues.apache.org/jira/browse/ATLAS-693


Repository: atlas


Description
---

Adding titan 0.5.4 implementation of abstraction layer under graphdb/titan0.  
The original TitanGraphProvider class has been morphed into Titan0Database in 
this new project.  This class has the logic that configures Titan and loads the 
graph.  A new TitanGraphProvider has been introduced in atlas-repository that 
delegates everything to Titan0Database.  Now, the TitanGraph is being 
initialized using the code in the new abstraction layer implementation.  In 
addition, the atlas-titan project has been merged into graphdb/titan0.  This 
consolidates all of the titan 0.5.4 configuration logic into the new project.  
All dependencies on atlas-titan have been replaced with dependencies on 
atlas-graphdb-titan0.  The Atlas war now bundles atlas-graphdb-api and 
atlas-graphdb-titan0.  At this point, though, Atlas is still not using the 
abstraction layer classes (AtlasVertex, AtlasEdge, etc) at runtime.  It is 
still hard-coded to use the Titan 0.5.4 classes.  The only logic in the 
abstraction layer
  implementation that is actually being used is the logic to configure Titan 
and create a TitanGraph.

Also added "solr" as a supported key for the solr index.  This is the key that 
is used for the index in Titan 1.0.0 (which has solr support built-in).  solr5 
is still allowed for backward compatibility, but this allows more consistency 
between the implementations and simplifies the test setup logic in the pom.xmls 
(we will need to update fewer properties when testing titan 1.0.0).


Diffs (updated)
-

  graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasEdge.java 
71b577b5c2e8aee8fec816b9224672ebc2cad5b3 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasEdgeDirection.java
 e7da1c9a5075004973ba68b8a36a2c95756e0e62 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasElement.java 
3c41693e531eb523f04a5c5781f1f123ab38ef07 
  graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java 
f312117af9cb6ae8de21d565bfcf3b10d9bdfcc6 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java
 2776b2242c4469dcae344616ccfe100633a79432 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphQuery.java
 f0dca9a28eebd8996d55d71dcc3254cce9cd4098 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasIndexQuery.java
 7ee6b2831a7610e7d5b1d36ae69282a9315dc06b 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasSchemaViolationException.java
 fda83b807f3640c04a660001941376f41be6d9be 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java 
e027b6962cf27a8c2dfbbf6fcc9683fa19e00ae5 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertexQuery.java
 fd20a6524c4795b4212599c2f4ced69c2bbf14c0 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/GraphDatabase.java
 a608eb27921d4be44e639f773ace818115f39ff2 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/GremlinVersion.java
 e817cba200c6d3c4c593b242cb2317e141bab570 
  graphdb/pom.xml 666fe5a01e55d7b761d391e3ef6172f769488254 
  graphdb/titan0/pom.xml PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/GraphDbObjectFactory.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Database.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0DatabaseManager.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Edge.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Element.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Graph.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0GraphQuery.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0IndexQuery.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Vertex.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0VertexQuery.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/TitanObjectFactory.java
 PRE-CREATION 
  

[jira] [Commented] (ATLAS-693) Add titan 0.5.4 implemenation of graph database abstraction layer

[ATLAS QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298388#comment-15298388
 ] 

ATLAS QA commented on ATLAS-693:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12805906/rb47466.patch
  against master revision 74d9b3f.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified test files.

  {color:red}-1 javac{color}.  The applied patch generated 140 javac 
compiler warnings (more than the master's current 136 warnings).

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

+1 checkstyle.  The patch generated 0 code style errors.

{color:red}-1 findbugs{color}.  The patch appears to introduce 380 new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The test build failed in  

Test results: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningswebapp.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningsauthorization.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningscommon.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningstitan0.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningssqoop-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningshdfs-model.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningsstorm-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningsfalcon-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningshive-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningsrepository.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningstypesystem.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningscatalog.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningsclient.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/newPatchFindbugsWarningsnotification.html
Javac warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/248//artifact/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-ATLAS-Build/248//console

This message is automatically generated.

> Add titan 0.5.4 implemenation of graph database abstraction layer
> -
>
> Key: ATLAS-693
> URL: https://issues.apache.org/jira/browse/ATLAS-693
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 0.6-incubating
>Reporter: Jeffrey Hagelberg
>Assignee: Jeffrey Hagelberg
> Fix For: trunk
>
>
> In this task, we put in place the titan 0.5.4 implementation of the graph 
> database abstraction layer.  As part of this effort, the existing "titan" 
> maven project will be merged into this new project, since it is specific to 
> titan 0.5.4.  The maven projects in Atlas that depend on the "titan" will be 
> changed to depend on the new project.  Direct dependencies on titan 0.5.4 
> will be removed from all of the atlas projects.  These projects will be 
> changed to depend on this new project instead.  They will transitively obtain 
> the dependencies on titan 0.5.4 that they need.  For this task, very little 
> of the existing Atlas java / scala code will be changed.  The classes from in 
> the titan project will be moved verbatim into the new project.  The only 
> other changes will be the addition of the new classes that implement the 
> abstraction layer interfaces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-693) Add titan 0.5.4 implemenation of graph database abstraction layer

[Jeffrey Hagelberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeffrey Hagelberg updated ATLAS-693:

Attachment: (was: rb47466.patch)

> Add titan 0.5.4 implemenation of graph database abstraction layer
> -
>
> Key: ATLAS-693
> URL: https://issues.apache.org/jira/browse/ATLAS-693
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 0.6-incubating
>Reporter: Jeffrey Hagelberg
>Assignee: Jeffrey Hagelberg
> Fix For: trunk
>
>
> In this task, we put in place the titan 0.5.4 implementation of the graph 
> database abstraction layer.  As part of this effort, the existing "titan" 
> maven project will be merged into this new project, since it is specific to 
> titan 0.5.4.  The maven projects in Atlas that depend on the "titan" will be 
> changed to depend on the new project.  Direct dependencies on titan 0.5.4 
> will be removed from all of the atlas projects.  These projects will be 
> changed to depend on this new project instead.  They will transitively obtain 
> the dependencies on titan 0.5.4 that they need.  For this task, very little 
> of the existing Atlas java / scala code will be changed.  The classes from in 
> the titan project will be moved verbatim into the new project.  The only 
> other changes will be the addition of the new classes that implement the 
> abstraction layer interfaces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-812) Atlas UI - Associate Terms with Assets

[ATLAS QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298331#comment-15298331
 ] 

ATLAS QA commented on ATLAS-812:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12805880/ATLAS-812.patch
  against master revision 74d9b3f.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

+1 checkstyle.  The patch generated 0 code style errors.

{color:red}-1 findbugs{color}.  The patch appears to introduce 379 new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in :
   
org.apache.atlas.repository.typestore.GraphBackedTypeStoreTest

Test results: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningswebapp.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningsauthorization.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningscommon.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningssqoop-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningshdfs-model.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningsstorm-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningsfalcon-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningshive-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningsrepository.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningstypesystem.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningscatalog.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningsclient.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningsnotification.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/247//artifact/patchprocess/newPatchFindbugsWarningstitan.html
Console output: https://builds.apache.org/job/PreCommit-ATLAS-Build/247//console

This message is automatically generated.

> Atlas UI - Associate Terms with Assets
> --
>
> Key: ATLAS-812
> URL: https://issues.apache.org/jira/browse/ATLAS-812
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-812.patch
>
>
> As part of Business Catalog, it is important to have a feature to associate 
> assets with terms. That feature is not yet been implemented. API is available 
> for the same in ATLAS-491.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-693) Add titan 0.5.4 implemenation of graph database abstraction layer

[Jeffrey Hagelberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeffrey Hagelberg updated ATLAS-693:

Attachment: (was: rb47466.patch)

> Add titan 0.5.4 implemenation of graph database abstraction layer
> -
>
> Key: ATLAS-693
> URL: https://issues.apache.org/jira/browse/ATLAS-693
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 0.6-incubating
>Reporter: Jeffrey Hagelberg
>Assignee: Jeffrey Hagelberg
> Fix For: trunk
>
>
> In this task, we put in place the titan 0.5.4 implementation of the graph 
> database abstraction layer.  As part of this effort, the existing "titan" 
> maven project will be merged into this new project, since it is specific to 
> titan 0.5.4.  The maven projects in Atlas that depend on the "titan" will be 
> changed to depend on the new project.  Direct dependencies on titan 0.5.4 
> will be removed from all of the atlas projects.  These projects will be 
> changed to depend on this new project instead.  They will transitively obtain 
> the dependencies on titan 0.5.4 that they need.  For this task, very little 
> of the existing Atlas java / scala code will be changed.  The classes from in 
> the titan project will be moved verbatim into the new project.  The only 
> other changes will be the addition of the new classes that implement the 
> abstraction layer interfaces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 47466: ATLAS-693: Add titan 0.5.4 implementation of graph database abstraction layer

[Jeff Hagelberg]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47466/
---

(Updated May 24, 2016, 3:01 p.m.)


Review request for atlas and David Kantor.


Changes
---

rebased with latest Atlas changes


Bugs: ATLAS-693
https://issues.apache.org/jira/browse/ATLAS-693


Repository: atlas


Description
---

Adding titan 0.5.4 implementation of abstraction layer under graphdb/titan0.  
The original TitanGraphProvider class has been morphed into Titan0Database in 
this new project.  This class has the logic that configures Titan and loads the 
graph.  A new TitanGraphProvider has been introduced in atlas-repository that 
delegates everything to Titan0Database.  Now, the TitanGraph is being 
initialized using the code in the new abstraction layer implementation.  In 
addition, the atlas-titan project has been merged into graphdb/titan0.  This 
consolidates all of the titan 0.5.4 configuration logic into the new project.  
All dependencies on atlas-titan have been replaced with dependencies on 
atlas-graphdb-titan0.  The Atlas war now bundles atlas-graphdb-api and 
atlas-graphdb-titan0.  At this point, though, Atlas is still not using the 
abstraction layer classes (AtlasVertex, AtlasEdge, etc) at runtime.  It is 
still hard-coded to use the Titan 0.5.4 classes.  The only logic in the 
abstraction layer
  implementation that is actually being used is the logic to configure Titan 
and create a TitanGraph.

Also added "solr" as a supported key for the solr index.  This is the key that 
is used for the index in Titan 1.0.0 (which has solr support built-in).  solr5 
is still allowed for backward compatibility, but this allows more consistency 
between the implementations and simplifies the test setup logic in the pom.xmls 
(we will need to update fewer properties when testing titan 1.0.0).


Diffs (updated)
-

  graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasEdge.java 
71b577b5c2e8aee8fec816b9224672ebc2cad5b3 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasEdgeDirection.java
 e7da1c9a5075004973ba68b8a36a2c95756e0e62 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasElement.java 
3c41693e531eb523f04a5c5781f1f123ab38ef07 
  graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java 
f312117af9cb6ae8de21d565bfcf3b10d9bdfcc6 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java
 2776b2242c4469dcae344616ccfe100633a79432 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphQuery.java
 f0dca9a28eebd8996d55d71dcc3254cce9cd4098 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasIndexQuery.java
 7ee6b2831a7610e7d5b1d36ae69282a9315dc06b 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasSchemaViolationException.java
 fda83b807f3640c04a660001941376f41be6d9be 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java 
e027b6962cf27a8c2dfbbf6fcc9683fa19e00ae5 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertexQuery.java
 fd20a6524c4795b4212599c2f4ced69c2bbf14c0 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/GraphDatabase.java
 a608eb27921d4be44e639f773ace818115f39ff2 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/GremlinVersion.java
 e817cba200c6d3c4c593b242cb2317e141bab570 
  graphdb/pom.xml 666fe5a01e55d7b761d391e3ef6172f769488254 
  graphdb/titan0/pom.xml PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/GraphDbObjectFactory.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Database.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0DatabaseManager.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Edge.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Element.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Graph.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0GraphQuery.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0IndexQuery.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Vertex.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0VertexQuery.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/TitanObjectFactory.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/utils/EdgeToAtlasEdgeFunction.java
 PRE-CREATION 
  

[jira] [Commented] (ATLAS-503) Not all Hive tables are not imported into Atlas when interrupted with search queries while importing.

[Hemanth Yamijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298263#comment-15298263
 ] 

Hemanth Yamijala commented on ATLAS-503:


I am able to replicate the issue with a test case that tries to create multiple 
tags at the same time. The exception stack trace shows a 
{{PermanentLockingException}}. This is being thrown from the shaded 
{{atlas-titan}} module and specifically the 
{{HBaseKeyColumnValueStore.acquireLock}} method. 

Turning on some debug statements, it looks like many threads are trying to 
acquire a transactional lock from these objects, and for some reason, they are 
all trying to acquire a lock on the same key, column, even though the tags 
being created are different. Further, we don't have any retries in place to 
handle a failure to acquire a lock, with the result that we are immediately 
failing in such situations.

So, two questions to consider:

* What are these keys / columns on which everyone is trying to acquire a lock?
* If this is a valid scenario and there is this much contention, should we 
possibly add some retries until the lock succeeds?

Doing further debugging on these lines.

> Not all Hive tables are not imported into Atlas when interrupted with search 
> queries while importing.  
> ---
>
> Key: ATLAS-503
> URL: https://issues.apache.org/jira/browse/ATLAS-503
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sharmadha Sainath
>Assignee: Hemanth Yamijala
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: hiv2atlaslogs.rtf
>
>
> On running a file containing 100 table creation commands using beeline -f , 
> all hive tables are created. But only 81 of them are imported into Atlas 
> (HiveHook enabled) when queries like "hive_table" is searched frequently 
> while the import process for the table is going on.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-819) All user defined types should have a set of common attributes

[David Kantor (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298224#comment-15298224
 ] 

David Kantor commented on ATLAS-819:


The user ID and timestamp attributes seem like system attributes and therefore 
should be read-only i.e. they should not be modifiable by end users.  They 
would be set by the internal Atlas repository code, as the timestamps are today.

> All user defined types should have a set of common attributes
> -
>
> Key: ATLAS-819
> URL: https://issues.apache.org/jira/browse/ATLAS-819
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>
> It would be very convenient if all user defined types have a conventional set 
> of common attributes including:
> * name
> * description
> * owner
> * created at
> * modified at



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-819) All user defined types should have a set of common attributes

[David Kantor (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298222#comment-15298222
 ] 

David Kantor commented on ATLAS-819:


* The name and description attributes seem like they are at a different/higher 
meta-level and many user models are likely to already have these attributes.
* Rather than a single owner attribute, it would be good to have separate 
createdByUser and modifiedByUser attributes, which combined with the existing 
timestamp attributes would provide basic auditing of entity changes.

> All user defined types should have a set of common attributes
> -
>
> Key: ATLAS-819
> URL: https://issues.apache.org/jira/browse/ATLAS-819
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>
> It would be very convenient if all user defined types have a conventional set 
> of common attributes including:
> * name
> * description
> * owner
> * created at
> * modified at



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 47775: ATLAS-812 : Atlas UI - Associate Terms with Assets

[keval bhatt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47775/
---

(Updated May 24, 2016, 2:03 p.m.)


Review request for atlas, Madhan Neethiraj, Shwetha GS, Selvamohan Neethiraj, 
and Hemanth Yamijala.


Bugs: ATLAS-812
https://issues.apache.org/jira/browse/ATLAS-812


Repository: atlas


Description
---

**Attached Patch contains:**

**1) Association of Terms with Assets.**
a) Now, allowing users to associate and dis-associate Terms with Assets from 
DSL search results page.

b) Also allowing users to dis-associate Terms with Assets from Asset details 
page.

**2) Showing associated Terms in asset details view in a tab same as that of 
Tags.**

**Note::** Currently we have implemented a workaround to show Terms separately 
from Entity object's list of Traits (i.e. showing all traits which has " . " in 
it as Terms). 
As and when we have list of terms in Entities or entity object, we will have to 
mke UI changes accordingly.


Diffs
-

  
dashboardv2/public/js/templates/business_catalog/AddTermToEntityLayoutView_tmpl.html
 PRE-CREATION 
  dashboardv2/public/js/templates/business_catalog/AddTermView_tmpl.html 
ddb7eac 
  dashboardv2/public/js/templates/detail_page/DetailPageLayoutView_tmpl.html 
633d497 
  dashboardv2/public/js/templates/search/SearchResultLayoutView_tmpl.html 
db827a3 
  dashboardv2/public/js/utils/CommonViewFunction.js ff0f2d2 
  dashboardv2/public/js/utils/Utils.js 4f57055 
  dashboardv2/public/js/views/business_catalog/AddTermToEntityLayoutView.js 
PRE-CREATION 
  
dashboardv2/public/js/views/business_catalog/BusinessCatalogDetailLayoutView.js 
665d5c6 
  dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js 6b4d05b 
  dashboardv2/public/js/views/search/SearchResultLayoutView.js e2981d1 
  dashboardv2/public/js/views/tag/TagDetailTableLayoutView.js 901130a 

Diff: https://reviews.apache.org/r/47775/diff/


Testing (updated)
---

Verified Term association functionality on Atlas UI manually.


Thanks,

keval bhatt

Review Request 47775: ATLAS-812 : Atlas UI - Associate Terms with Assets

[keval bhatt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47775/
---

Review request for atlas, Madhan Neethiraj, Shwetha GS, Selvamohan Neethiraj, 
and Hemanth Yamijala.


Bugs: ATLAS-812
https://issues.apache.org/jira/browse/ATLAS-812


Repository: atlas


Description
---

**Attached Patch contains:**

**1) Association of Terms with Assets.**
a) Now, allowing users to associate and dis-associate Terms with Assets from 
DSL search results page.

b) Also allowing users to dis-associate Terms with Assets from Asset details 
page.

**2) Showing associated Terms in asset details view in a tab same as that of 
Tags.**

**Note::** Currently we have implemented a workaround to show Terms separately 
from Entity object's list of Traits (i.e. showing all traits which has " . " in 
it as Terms). 
As and when we have list of terms in Entities or entity object, we will have to 
mke UI changes accordingly.


Diffs
-

  
dashboardv2/public/js/templates/business_catalog/AddTermToEntityLayoutView_tmpl.html
 PRE-CREATION 
  dashboardv2/public/js/templates/business_catalog/AddTermView_tmpl.html 
ddb7eac 
  dashboardv2/public/js/templates/detail_page/DetailPageLayoutView_tmpl.html 
633d497 
  dashboardv2/public/js/templates/search/SearchResultLayoutView_tmpl.html 
db827a3 
  dashboardv2/public/js/utils/CommonViewFunction.js ff0f2d2 
  dashboardv2/public/js/utils/Utils.js 4f57055 
  dashboardv2/public/js/views/business_catalog/AddTermToEntityLayoutView.js 
PRE-CREATION 
  
dashboardv2/public/js/views/business_catalog/BusinessCatalogDetailLayoutView.js 
665d5c6 
  dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js 6b4d05b 
  dashboardv2/public/js/views/search/SearchResultLayoutView.js e2981d1 
  dashboardv2/public/js/views/tag/TagDetailTableLayoutView.js 901130a 

Diff: https://reviews.apache.org/r/47775/diff/


Testing
---

Verified Tag association functionality on Atlas UI manually.


Thanks,

keval bhatt

[jira] [Commented] (ATLAS-499) Automatically update inverse references to prevent repository corruption

[ATLAS QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298180#comment-15298180
 ] 

ATLAS QA commented on ATLAS-499:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12805860/rb47658.patch
  against master revision 74d9b3f.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

+1 checkstyle.  The patch generated 0 code style errors.

{color:red}-1 findbugs{color}.  The patch appears to introduce 380 new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in :
   
org.apache.atlas.repository.typestore.GraphBackedTypeStoreTest

Test results: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningswebapp.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningsauthorization.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningscommon.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningssqoop-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningshdfs-model.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningsstorm-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningsfalcon-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningshive-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningsrepository.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningstypesystem.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningscatalog.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningsclient.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningsnotification.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/246//artifact/patchprocess/newPatchFindbugsWarningstitan.html
Console output: https://builds.apache.org/job/PreCommit-ATLAS-Build/246//console

This message is automatically generated.

> Automatically update inverse references to prevent repository corruption
> 
>
> Key: ATLAS-499
> URL: https://issues.apache.org/jira/browse/ATLAS-499
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Kantor
>Assignee: David Kantor
> Fix For: 0.7-incubating
>
> Attachments: rb47658.patch
>
>
> For references that are defined in the type system as bi-directional, Atlas 
> does not enforce that clients update both sides of the reference, and relies 
> on well behaved client applications for repository consistency.   This allows 
> the repository to be easily corrupted with unbalanced references.  Atlas 
> should automatically update the inverse reference to avoid this corruption.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-499) Automatically update inverse references to prevent repository corruption

[David Kantor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Kantor updated ATLAS-499:
---
Attachment: rb47658.patch

Attaching latest review board patch

> Automatically update inverse references to prevent repository corruption
> 
>
> Key: ATLAS-499
> URL: https://issues.apache.org/jira/browse/ATLAS-499
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Kantor
>Assignee: David Kantor
> Fix For: 0.7-incubating
>
> Attachments: rb47658.patch
>
>
> For references that are defined in the type system as bi-directional, Atlas 
> does not enforce that clients update both sides of the reference, and relies 
> on well behaved client applications for repository consistency.   This allows 
> the repository to be easily corrupted with unbalanced references.  Atlas 
> should automatically update the inverse reference to avoid this corruption.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-499) Automatically update inverse references to prevent repository corruption

[David Kantor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Kantor updated ATLAS-499:
---
Attachment: (was: rb47658.patch)

> Automatically update inverse references to prevent repository corruption
> 
>
> Key: ATLAS-499
> URL: https://issues.apache.org/jira/browse/ATLAS-499
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Kantor
>Assignee: David Kantor
> Fix For: 0.7-incubating
>
>
> For references that are defined in the type system as bi-directional, Atlas 
> does not enforce that clients update both sides of the reference, and relies 
> on well behaved client applications for repository consistency.   This allows 
> the repository to be easily corrupted with unbalanced references.  Atlas 
> should automatically update the inverse reference to avoid this corruption.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-804) For hive_db, "locationUri" field is renamed to "location" but the hive data model json is still showing as "locationUri"

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-804:
---
Assignee: Vimal Sharma  (was: Suma Shivaprasad)

> For hive_db, "locationUri" field is renamed to "location" but the hive data 
> model json is still showing as "locationUri"
> 
>
> Key: ATLAS-804
> URL: https://issues.apache.org/jira/browse/ATLAS-804
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Vimal Sharma
> Fix For: 0.7-incubating
>
>
> As part of ATLAS-758, locationuri field is renamed to location. But the hive 
> data model is still pointing to Locationuri field.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-760) Missing table parameters(last_modified_by and last_modified_time)

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-760:
---
Assignee: Vimal Sharma  (was: Suma Shivaprasad)

> Missing table parameters(last_modified_by and last_modified_time)
> -
>
> Key: ATLAS-760
> URL: https://issues.apache.org/jira/browse/ATLAS-760
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Vimal Sharma
>Priority: Critical
> Fix For: 0.7-incubating
>
>
> Missing table parameters(last_modified_by and last_modified_time)
> Hive table metadata showing vboth params..
> {noformat}
> 0: jdbc:hive2://localhost:1/default> describe formatted table3_s1vxgkt0bh;
> OK
> +---+--++--+
> |   col_name|  data_type  
>  |comment |
> +---+--++--+
> | # col_name| data_type   
>  | comment|
> |   | NULL
>  | NULL   |
> | viewtime  | int 
>  ||
> | userid| bigint  
>  ||
> | page_url  | string  
>  ||
> | referrer_url  | string  
>  ||
> | ip| string  
>  ||
> |   | NULL
>  | NULL   |
> | # Partition Information   | NULL
>  | NULL   |
> | # col_name| data_type   
>  | comment|
> |   | NULL
>  | NULL   |
> | dt| string  
>  ||
> | country   | string  
>  | partitioned columns comments.  |
> |   | NULL
>  | NULL   |
> | # Detailed Table Information  | NULL
>  | NULL   |
> | Database: | default 
>  | NULL   |
> | Owner:| apathan 
>  | NULL   |
> | CreateTime:   | Mon May 09 14:42:34 IST 2016
>  | NULL   |
> | LastAccessTime:   | UNKNOWN 
>  | NULL   |
> | Protect Mode: | None
>  | NULL   |
> | Retention:| 0   
>  | NULL   |
> | Location: | 
> hdfs://localhost:9000/user/hive/warehouse/table3_s1vxgkt0bh  | NULL   
> |
> | Table Type:   | MANAGED_TABLE   
>  | NULL   |
> | Table Parameters: | NULL
>  | NULL   |
> |   | last_modified_by
>  | apathan|
> |   | last_modified_time  
>  | 1462785155 |
> |   | transient_lastDdlTime   
>  | 1462785155 |
> 

[jira] [Updated] (ATLAS-642) import-hive should create the lineage for external tables

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-642:
---
Assignee: Vimal Sharma

> import-hive should create the lineage for external tables
> -
>
> Key: ATLAS-642
> URL: https://issues.apache.org/jira/browse/ATLAS-642
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Suma Shivaprasad
>Assignee: Vimal Sharma
> Fix For: 0.7-incubating
>
>
> import-hive can also start showing lineage between HDFS paths and tables. 
> However the create table process should also register the exact query that 
> was  used to create the table. Show create table does that by reconstructing 
> the query. Need to figure out if we can potentially use that to create the 
> query



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-642) import-hive should create the lineage for external tables

[Madhan Neethiraj (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298088#comment-15298088
 ] 

Madhan Neethiraj commented on ATLAS-642:


[~suma.shivaprasad] since most details needed for lineage (HDFS paths, Hive 
table info) are available from Hive metadata, I think it is valuable to get the 
available info into Atlas - even if SQL query used to create the table is not 
available.

> import-hive should create the lineage for external tables
> -
>
> Key: ATLAS-642
> URL: https://issues.apache.org/jira/browse/ATLAS-642
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Suma Shivaprasad
> Fix For: 0.7-incubating
>
>
> import-hive can also start showing lineage between HDFS paths and tables. 
> However the create table process should also register the exact query that 
> was  used to create the table. Show create table does that by reconstructing 
> the query. Need to figure out if we can potentially use that to create the 
> query



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-642) import-hive should create the lineage for external tables

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-642:
---
Summary: import-hive should create the lineage for external tables  (was: 
import hive should create the lineage for external tables)

> import-hive should create the lineage for external tables
> -
>
> Key: ATLAS-642
> URL: https://issues.apache.org/jira/browse/ATLAS-642
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Suma Shivaprasad
> Fix For: 0.7-incubating
>
>
> import-hive can also start showing lineage between HDFS paths and tables. 
> However the create table process should also register the exact query that 
> was  used to create the table. Show create table does that by reconstructing 
> the query. Need to figure out if we can potentially use that to create the 
> query



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything

[Madhan Neethiraj (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298061#comment-15298061
 ] 

Madhan Neethiraj commented on ATLAS-688:


After removing Hive libraries from packaging of import-hive utility, these 
libraries will be referenced using their location in Hive installation 
directory. This would require import-hive utility to be run from the host where 
Hive is installed. Given that Atlas hooks will be installed in Hive's host, it 
might be easier to include import-hive utility in Hive hook (instead of Atlas 
server).

> import-hive should depend on Hive CLASSPATH jars instead of packaging 
> everything
> 
>
> Key: ATLAS-688
> URL: https://issues.apache.org/jira/browse/ATLAS-688
> Project: Atlas
>  Issue Type: Bug
>Reporter: Suma Shivaprasad
>Assignee: Vimal Sharma
>
> Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it 
> should depend on HIVE_CLASSPATH from which it can get the jars and should be 
> paakged along with hook on hive server m/cs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-503) Not all Hive tables are not imported into Atlas when interrupted with search queries while importing.

[Hemanth Yamijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298058#comment-15298058
 ] 

Hemanth Yamijala commented on ATLAS-503:


Started looking at this.

My primary focus for the bug will be to replicate, debug and fix it with a 
HBase backend, as BerkeleyDB is not a recommended production.

I have been trying to replicate the issue by trying what [~ssainath] reported, 
except with HBase as the backend. However, I have had no success so far 
(imported 1000, 5000, 1 tables) in replicating the issue. 

Talking to Sharmadha, I found that we don't see this specific problem with 
HBase, only with BerkeleyDB. However, there are other scenarios where even 
HBase gives a lock exception. Some scenarios are:

* Multiple consumer threads and partitions while importing data into Atlas.
* Multiple threads creating tags

I am assuming the underlying cause will be the same and will try to use one of 
these scenarios to replicate the issue.

> Not all Hive tables are not imported into Atlas when interrupted with search 
> queries while importing.  
> ---
>
> Key: ATLAS-503
> URL: https://issues.apache.org/jira/browse/ATLAS-503
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sharmadha Sainath
>Assignee: Hemanth Yamijala
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: hiv2atlaslogs.rtf
>
>
> On running a file containing 100 table creation commands using beeline -f , 
> all hive tables are created. But only 81 of them are imported into Atlas 
> (HiveHook enabled) when queries like "hive_table" is searched frequently 
> while the import process for the table is going on.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-822) Type updates - add supertypes

[Shwetha G S (JIRA)]

Shwetha G S created ATLAS-822:
-

 Summary: Type updates - add supertypes
 Key: ATLAS-822
 URL: https://issues.apache.org/jira/browse/ATLAS-822
 Project: Atlas
  Issue Type: Bug
Reporter: Shwetha G S
 Fix For: 0.7-incubating


In type updates, we allow adding super types. But set of super types is 
maintained in every entity, and this type update will require updating all the 
entities of that type which is not handled currently. We should either update 
all entities or not allow updating super type information



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: apache-atlas-nightly #286

[Apache Jenkins Server]

See 

Changes:

[sshivalingamurthy] ATLAS-809 JAAS configuration needed for Kafka interaction 
via Atlas

--
[...truncated 5869 lines...]
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @ 
atlas-webapp ---
[WARNING] Zinc server is not available at port 3030 - reverting to normal 
incremental compile
[INFO] Using incremental compilation
[INFO] Compiling 54 Java sources to 

[WARNING] Note: 

 uses or overrides a deprecated API.
[WARNING] Note: Recompile with -Xlint:deprecation for details.
[WARNING] Note: Some input files use unchecked or unsafe operations.
[WARNING] Note: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ atlas-webapp 
---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 54 source files to 

[WARNING] 
:
 

 uses or overrides a deprecated API.
[WARNING] 
:
 Recompile with -Xlint:deprecation for details.
[WARNING] 
:
 Some input files use unchecked or unsafe operations.
[WARNING] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- scala-maven-plugin:3.2.0:testCompile (scala-test-compile-first) @ 
atlas-webapp ---
[WARNING] Zinc server is not available at port 3030 - reverting to normal 
incremental compile
[INFO] Using incremental compilation
[INFO] Compiling 39 Java sources to 

[WARNING] Note: Some input files use unchecked or unsafe operations.
[WARNING] Note: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ 
atlas-webapp ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 39 source files to 

[WARNING] 
:
 Some input files use unchecked or unsafe operations.
[WARNING] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-war-plugin:2.6:war (default-war) @ atlas-webapp ---
[INFO] Packaging webapp
[INFO] Assembling webapp [atlas-webapp] in 
[
[INFO] Processing overlay [ id org.apache.atlas:atlas-dashboardv2]
[INFO] Processing war project
[INFO] Copying webapp resources 
[
[INFO] Building jar: 

[INFO] Webapp assembled in [1877 msecs]
[INFO] Building war: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
atlas-webapp ---
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-webapp ---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use 
forkCount and reuseForks instead.
[INFO] Surefire report directory: 

Re: Review Request 46731: ATLAS-716 Entity update/delete notifications

[Shwetha GS]

> On May 23, 2016, 5:02 a.m., Suma Shivaprasad wrote:
> > repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java,
> >  line 318
> > 
> >
> > why is the check for isComposite removed? Can you pls explain?
> 
> Shwetha GS wrote:
> For example, in case of soft delete, if the column is deleted, we don't 
> update the table's columns attribute. So, table definition returns the list 
> of columns that includes deleted columns as well. 
> 
> For hive_process, when table is deleted, the reference is not removed. To 
> be consistent, for hive_table, when column is deleted, reference is not 
> removed as well
> 
> Suma Shivaprasad wrote:
> Wouldnt this be an issue in case of hard deletes?

the iscomposite condition is used to update the reverse reference
like removing column reference from table columns
currently, it uses just shouldUpdateReverseAttribute check
shouldUpdateReverseAttribute is set to true for hard delete
shouldUpdateReverseAttribute is false for soft delete
after updating the reference, it calls deleteEdge(edge, false);
which takes care of actually deleting the column and its edge


- Shwetha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46731/#review134305
---


On May 24, 2016, 7:34 a.m., Shwetha GS wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46731/
> ---
> 
> (Updated May 24, 2016, 7:34 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-716
> https://issues.apache.org/jira/browse/ATLAS-716
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1. Update/delete entity APIs return list of entities created, updated and 
> deleted
> 2. Listeners are called for all the affected entities
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  254e150 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> 70100f1 
>   client/src/main/java/org/apache/atlas/AtlasClient.java 7e32cc2 
>   client/src/main/java/org/apache/atlas/EntityAuditEvent.java 460f708 
>   client/src/main/java/org/apache/atlas/SerDe.java PRE-CREATION 
>   client/src/test/java/org/apache/atlas/AtlasClientTest.java 0e80573 
>   
> notification/src/main/java/org/apache/atlas/notification/MessageVersion.java 
> 3f16a9a 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java
>  384f383 
>   
> notification/src/main/java/org/apache/atlas/notification/VersionedMessageDeserializer.java
>  290be59 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/NotificationEntityChangeListener.java
>  300cbb5 
>   
> notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
>  e8b55ef 
>   
> repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 
> 540c308 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
>  5b4bdbf 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  8f11322 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> a9e4f39 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  3604277 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> cccafc2 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
>  df28ab3 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/HardDeleteHandler.java
>  f8bbf73 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/SoftDeleteHandler.java
>  aa78582 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  a017536 
>   
> repository/src/main/java/org/apache/atlas/repository/typestore/GraphBackedTypeStore.java
>  3fb128c 
>   
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
>  7cd83f8 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  be407a5 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  1aeedb5 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositoryHardDeleteTest.java
>  d2109d3 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositorySoftDeleteTest.java
>  d9e3ec9 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  1f906ed 
>   

Re: Review Request 46731: ATLAS-716 Entity update/delete notifications

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46731/
---

(Updated May 24, 2016, 7:34 a.m.)


Review request for atlas.


Changes
---

Re-based, addressed review comments


Bugs: ATLAS-716
https://issues.apache.org/jira/browse/ATLAS-716


Repository: atlas


Description
---

1. Update/delete entity APIs return list of entities created, updated and 
deleted
2. Listeners are called for all the affected entities


Diffs (updated)
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 254e150 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
70100f1 
  client/src/main/java/org/apache/atlas/AtlasClient.java 7e32cc2 
  client/src/main/java/org/apache/atlas/EntityAuditEvent.java 460f708 
  client/src/main/java/org/apache/atlas/SerDe.java PRE-CREATION 
  client/src/test/java/org/apache/atlas/AtlasClientTest.java 0e80573 
  notification/src/main/java/org/apache/atlas/notification/MessageVersion.java 
3f16a9a 
  
notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java
 384f383 
  
notification/src/main/java/org/apache/atlas/notification/VersionedMessageDeserializer.java
 290be59 
  
notification/src/main/java/org/apache/atlas/notification/entity/NotificationEntityChangeListener.java
 300cbb5 
  
notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
 e8b55ef 
  repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 
540c308 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
 5b4bdbf 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 8f11322 
  repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
a9e4f39 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
 3604277 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
cccafc2 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
 df28ab3 
  
repository/src/main/java/org/apache/atlas/repository/graph/HardDeleteHandler.java
 f8bbf73 
  
repository/src/main/java/org/apache/atlas/repository/graph/SoftDeleteHandler.java
 aa78582 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 a017536 
  
repository/src/main/java/org/apache/atlas/repository/typestore/GraphBackedTypeStore.java
 3fb128c 
  
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java 
7cd83f8 
  
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
 be407a5 
  
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
 1aeedb5 
  
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositoryHardDeleteTest.java
 d2109d3 
  
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositorySoftDeleteTest.java
 d9e3ec9 
  
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
 1f906ed 
  server-api/src/main/java/org/apache/atlas/RequestContext.java b1d87ea 
  server-api/src/main/java/org/apache/atlas/services/MetadataService.java 
c8c1067 
  typesystem/src/main/java/org/apache/atlas/typesystem/Referenceable.java 
31f157e 
  webapp/src/main/java/org/apache/atlas/LocalAtlasClient.java c6ed85d 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationEntityChangeListener.java
 PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
709fec5 
  webapp/src/main/java/org/apache/atlas/web/service/ServiceModule.java 0f8bcb1 
  webapp/src/test/java/org/apache/atlas/LocalAtlasClientTest.java 7f20652 
  webapp/src/test/java/org/apache/atlas/notification/EntityNotificationIT.java 
6985152 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationEntityChangeListenerTest.java
 PRE-CREATION 
  
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 
aa92bc0 
  webapp/src/test/java/org/apache/atlas/web/service/CuratorFactoryTest.java 
2d510a0 

Diff: https://reviews.apache.org/r/46731/diff/


Testing
---

UTs


Thanks,

Shwetha GS

[jira] [Updated] (ATLAS-812) Associate Terms with Assets

[Keval Bhatt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-812:
--
Summary: Associate Terms with Assets  (was: Associate Terms with 
unclassified Assets)

> Associate Terms with Assets
> ---
>
> Key: ATLAS-812
> URL: https://issues.apache.org/jira/browse/ATLAS-812
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
> Fix For: 0.7-incubating
>
>
> As part of Business Catalog, it is important to have a feature to associate 
> assets with terms. That feature is not yet been implemented. API is available 
> for the same in ATLAS-491.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 46731: ATLAS-716 Entity update/delete notifications

[Suma Shivaprasad]

> On May 23, 2016, 5:02 a.m., Suma Shivaprasad wrote:
> > repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java,
> >  line 318
> > 
> >
> > why is the check for isComposite removed? Can you pls explain?
> 
> Shwetha GS wrote:
> For example, in case of soft delete, if the column is deleted, we don't 
> update the table's columns attribute. So, table definition returns the list 
> of columns that includes deleted columns as well. 
> 
> For hive_process, when table is deleted, the reference is not removed. To 
> be consistent, for hive_table, when column is deleted, reference is not 
> removed as well

Wouldnt this be an issue in case of hard deletes?


- Suma


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46731/#review134305
---


On May 19, 2016, 5:41 a.m., Shwetha GS wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46731/
> ---
> 
> (Updated May 19, 2016, 5:41 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-716
> https://issues.apache.org/jira/browse/ATLAS-716
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1. Update/delete entity APIs return list of entities created, updated and 
> deleted
> 2. Listeners are called for all the affected entities
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  254e150 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> 70100f1 
>   client/src/main/java/org/apache/atlas/AtlasClient.java b3ec95c 
>   client/src/main/java/org/apache/atlas/EntityAuditEvent.java 460f708 
>   client/src/main/java/org/apache/atlas/SerDe.java PRE-CREATION 
>   client/src/test/java/org/apache/atlas/AtlasClientTest.java 0e80573 
>   
> notification/src/main/java/org/apache/atlas/notification/MessageVersion.java 
> 3f16a9a 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java
>  384f383 
>   
> notification/src/main/java/org/apache/atlas/notification/VersionedMessageDeserializer.java
>  290be59 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/NotificationEntityChangeListener.java
>  300cbb5 
>   
> notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
>  e8b55ef 
>   
> repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 
> 540c308 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
>  5b4bdbf 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  8f11322 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> a9e4f39 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  3604277 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> cccafc2 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
>  df28ab3 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/HardDeleteHandler.java
>  f8bbf73 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/SoftDeleteHandler.java
>  aa78582 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  a017536 
>   
> repository/src/main/java/org/apache/atlas/repository/typestore/GraphBackedTypeStore.java
>  3fb128c 
>   
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
>  5195cbe 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  be407a5 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  1aeedb5 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositoryHardDeleteTest.java
>  d2109d3 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositorySoftDeleteTest.java
>  d9e3ec9 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  1f906ed 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java b1d87ea 
>   server-api/src/main/java/org/apache/atlas/services/MetadataService.java 
> 13d20d8 
>   typesystem/src/main/java/org/apache/atlas/typesystem/Referenceable.java 
> 31f157e 
>   webapp/src/main/java/org/apache/atlas/LocalAtlasClient.java c6ed85d 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationEntityChangeListener.java
>  PRE-CREATION 
>