Build failed in Jenkins: apache-atlas-nightly #259

[Apache Jenkins Server]

See 

Changes:

[hyamijala] ATLAS-742 Avoid downloading hbase multiple times (shwethags via

--
[...truncated 7814 lines...]
127.0.0.1 - - [06/May/2016:05:29:08 +] "GET 
/api/atlas/entities/1fd7ece4-3bfc-408a-b295-225a866772c6 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:08 +] "OPTIONS 
/api/atlas/entities?doAs=jenkins=jenkins HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:08 +] "PUT 
/api/atlas/entities?doAs=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:09 +] "OPTIONS 
/api/atlas/entities?doAs=jenkins=jenkins HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:09 +] "PUT 
/api/atlas/entities?doAs=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "OPTIONS 
/api/atlas/entities?type=hive_table=name=default.tablesfirbw3yxn@test=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "GET 
/api/atlas/entities?type=hive_table=name=default.tablesfirbw3yxn@test
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "OPTIONS 
/api/atlas/entities?type=hive_table=name=default.tablesfirbw3yxn@test=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "GET 
/api/atlas/entities?type=hive_table=name=default.tablesfirbw3yxn@test
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "OPTIONS 
/api/atlas/entities?doAs=jenkins=jenkins HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "OPTIONS 
/api/atlas/entities?type=hive_table=name=dbnxcgy0jlyn.table9dq7exjjwx@test=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "GET 
/api/atlas/entities?type=hive_table=name=dbnxcgy0jlyn.table9dq7exjjwx@test
 HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "PUT 
/api/atlas/entities?doAs=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "OPTIONS 
/api/atlas/entities?doAs=jenkins=jenkins HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "OPTIONS 
/api/atlas/entities?type=hive_table=name=dbnxcgy0jlyn.table9dq7exjjwx@test=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "GET 
/api/atlas/entities?type=hive_table=name=dbnxcgy0jlyn.table9dq7exjjwx@test
 HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:10 +] "PUT 
/api/atlas/entities?doAs=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:11 +] "OPTIONS 
/api/atlas/entities?doAs=jenkins=jenkins HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:11 +] "OPTIONS 
/api/atlas/entities?type=hive_table=name=dbnxcgy0jlyn.table9dq7exjjwx@test=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:11 +] "GET 
/api/atlas/entities?type=hive_table=name=dbnxcgy0jlyn.table9dq7exjjwx@test
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:11 +] "OPTIONS 
/api/atlas/entities?type=hive_table=name=dbnxcgy0jlyn.table9dq7exjjwx@test=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:11 +] "POST 
/api/atlas/entities?doAs=jenkins HTTP/1.1" 201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:11 +] "GET 
/api/atlas/entities?type=hive_table=name=dbnxcgy0jlyn.table9dq7exjjwx@test
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:11 +] "OPTIONS 
/api/atlas/lineage/hive/table/dbnxcgy0jlyn.table9dq7exjjwx@test/inputs/graph?user.name=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:11 +] "GET 
/api/atlas/lineage/hive/table/dbnxcgy0jlyn.table9dq7exjjwx@test/inputs/graph 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:13 +] "OPTIONS 
/api/atlas/lineage/hive/table/default.tablesfirbw3yxn@test/outputs/graph?user.name=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:13 +] "GET 
/api/atlas/lineage/hive/table/default.tablesfirbw3yxn@test/outputs/graph 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:16 +] "OPTIONS 
/api/atlas/entities?type=hive_table=name=default.table7nkdk2pv2o@test=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:16 +] "GET 
/api/atlas/entities?type=hive_table=name=default.table7nkdk2pv2o@test
 HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:16 +] "OPTIONS 
/api/atlas/entities?doAs=jenkins=jenkins HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:16 +] "OPTIONS 
/api/atlas/entities?type=hive_table=name=default.table7nkdk2pv2o@test=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [06/May/2016:05:29:16 +] "GET 

[jira] [Issue Comment Deleted] (ATLAS-497) Simple Authorization

[Hemanth Yamijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hemanth Yamijala updated ATLAS-497:
---
Comment: was deleted

(was: Only did a cursory glance at the patch, assuming [~sneethiraj] has looked 
at it closely. My review comments seem to be incorporated. We need ATLAS-611 
committed for this to proceed.)

> Simple Authorization
> 
>
> Key: ATLAS-497
> URL: https://issues.apache.org/jira/browse/ATLAS-497
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.7-incubating
>Reporter: Erik Bergenholtz
>Assignee: Saqeeb Shaikh
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-497.1.patch, ATLAS-497.2.patch, ATLAS-497.6.patch, 
> ATLAS-497.patch
>
>
> Atlas needs to support a simple (out of box) authorization mechanism.
> Defined Roles:
> - Data Scientist: provides a read only view (GET)
> - Data Steward: provides a read/edit view (PUT, POST, DELETE)
> - Admin (can do anything)
> All can comment on entity
> Requirements
> - Atlas will implement a simple file based store for providing user to role 
> mapping
> - The out of box experience will be this file based mechanism for 
> authorization



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-497) Simple Authorization

[Hemanth Yamijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15273641#comment-15273641
 ] 

Hemanth Yamijala commented on ATLAS-497:


Only did a cursory glance at the patch, assuming Selvamohan Neethiraj has 
looked at it closely. My review comments seem to be incorporated. We need 
ATLAS-661 committed for this to proceed.

> Simple Authorization
> 
>
> Key: ATLAS-497
> URL: https://issues.apache.org/jira/browse/ATLAS-497
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.7-incubating
>Reporter: Erik Bergenholtz
>Assignee: Saqeeb Shaikh
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-497.1.patch, ATLAS-497.2.patch, ATLAS-497.6.patch, 
> ATLAS-497.patch
>
>
> Atlas needs to support a simple (out of box) authorization mechanism.
> Defined Roles:
> - Data Scientist: provides a read only view (GET)
> - Data Steward: provides a read/edit view (PUT, POST, DELETE)
> - Admin (can do anything)
> All can comment on entity
> Requirements
> - Atlas will implement a simple file based store for providing user to role 
> mapping
> - The out of box experience will be this file based mechanism for 
> authorization



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-497) Simple Authorization

[Hemanth Yamijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15273633#comment-15273633
 ] 

Hemanth Yamijala commented on ATLAS-497:


Only did a cursory glance at the patch, assuming [~sneethiraj] has looked at it 
closely. My review comments seem to be incorporated. We need ATLAS-611 
committed for this to proceed.

> Simple Authorization
> 
>
> Key: ATLAS-497
> URL: https://issues.apache.org/jira/browse/ATLAS-497
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.7-incubating
>Reporter: Erik Bergenholtz
>Assignee: Saqeeb Shaikh
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-497.1.patch, ATLAS-497.2.patch, ATLAS-497.6.patch, 
> ATLAS-497.patch
>
>
> Atlas needs to support a simple (out of box) authorization mechanism.
> Defined Roles:
> - Data Scientist: provides a read only view (GET)
> - Data Steward: provides a read/edit view (PUT, POST, DELETE)
> - Admin (can do anything)
> All can comment on entity
> Requirements
> - Atlas will implement a simple file based store for providing user to role 
> mapping
> - The out of box experience will be this file based mechanism for 
> authorization



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-491) Business Catalog / Taxonomy

[John Speidel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Speidel updated ATLAS-491:
---
Attachment: business-catalog-WIP.patch

Attached a WIP patch of the business catalog functionality. 
This code is still a work in progress and needs cleanup/refactoring and 
hardening and is therefore not ready to be considered for merging.
Taxonomy and Term resources currently can't be updated or deleted.
Unit tests are not yet included.

> Business Catalog / Taxonomy
> ---
>
> Key: ATLAS-491
> URL: https://issues.apache.org/jira/browse/ATLAS-491
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.7-incubating
>Reporter: Erik Bergenholtz
>Assignee: John Speidel
> Fix For: 0.7-incubating
>
> Attachments: business-catalog-WIP.patch
>
>
> s a Atlas UX user, I need to:
> Browse business taxonomy hieracharlically through tree
> Search by Tags
> Search by combination of keyword, tag, free text in search field
> All search from view context of taxonomy. If 3 levels deep, then the search 
> should be narrow to that scope.
> expose search ordering capabilities available through SOLR.
> As a Data Steward, I need to:
> Manage the taxonomy of objects including
> Objects Management:
> Creating new objects
> Modify the object details - annotations, external links, tags, acceptable use 
> policy, etc
> Delete objects (in UI - changing status in backend,and versioning changes)
> Relationship management:
> creating relationship of objects as a parent or child
> assigning tagging to taxonomy
> verify tag inheritance
> Verify tag impact on tag based policy - ranger API call
> Tag Management:
> List
> Search
> Edit
> Delete (change state to archive)
> All user should be able to comment on any business catalog entity.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 46731: ATLAS-716 Entity update/delete notifications

[Suma Shivaprasad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46731/#review131957
---




client/src/main/java/org/apache/atlas/notification/EntityNotificationImpl.java 
(line 95)


traits null check is missing . Could lead to NPE ? Also would be better to 
split to make it more readable


- Suma Shivaprasad


On April 27, 2016, 1:05 p.m., Shwetha GS wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46731/
> ---
> 
> (Updated April 27, 2016, 1:05 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-716
> https://issues.apache.org/jira/browse/ATLAS-716
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1. Update/delete entity APIs return list of entities created, updated and 
> deleted
> 2. Listeners are called for all the affected entities
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  104c0c5 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
> 813177f 
>   
> addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java 
> ab7e6ee 
>   client/src/main/java/org/apache/atlas/AtlasClient.java be34802 
>   client/src/main/java/org/apache/atlas/EntityAuditEvent.java 460f708 
>   client/src/main/java/org/apache/atlas/SerDe.java PRE-CREATION 
>   client/src/main/java/org/apache/atlas/notification/EntityNotification.java 
> PRE-CREATION 
>   
> client/src/main/java/org/apache/atlas/notification/EntityNotificationImpl.java
>  PRE-CREATION 
>   client/src/main/java/org/apache/atlas/notification/HookNotification.java 
> PRE-CREATION 
>   client/src/test/java/org/apache/atlas/AtlasClientTest.java 8911bf5 
>   
> client/src/test/java/org/apache/atlas/notification/EntityNotificationImplTest.java
>  PRE-CREATION 
>   
> client/src/test/java/org/apache/atlas/notification/HookNotificationTest.java 
> PRE-CREATION 
>   notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 68a868f 
>   
> notification/src/main/java/org/apache/atlas/notification/AbstractNotification.java
>  596f988 
>   
> notification/src/main/java/org/apache/atlas/notification/AbstractNotificationConsumer.java
>  1cadb99 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  1f2df3e 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java
>  ac285aa 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/EntityNotification.java
>  82a1100 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/EntityNotificationImpl.java
>  fda588e 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/NotificationEntityChangeListener.java
>  300cbb5 
>   
> notification/src/main/java/org/apache/atlas/notification/hook/HookNotification.java
>  80c96fa 
>   
> notification/src/test/java/org/apache/atlas/notification/NotificationEntityChangeListenerTest.java
>  PRE-CREATION 
>   
> notification/src/test/java/org/apache/atlas/notification/entity/EntityNotificationImplTest.java
>  385c41f 
>   
> notification/src/test/java/org/apache/atlas/notification/hook/HookNotificationTest.java
>  57b0eea 
>   pom.xml 729b178 
>   repository/pom.xml 533e48b 
>   
> repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 
> 540c308 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
>  5b4bdbf 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  8f11322 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 369a5d5 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  de29e86 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  a3dc7e5 
>   
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
>  9f69940 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  be407a5 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/HBaseTestUtils.java
>  84d7bab 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  ae215f9 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  1f906ed 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java fa94763 
>   server-api/src/main/java/org/apache/atlas/services/MetadataService.java 
> 13d20d8 
>   typesystem/src/main/java/org/apache/atlas/typesystem/Referenceable.java 
> 31f157e 
>   

Re: Review Request 46731: ATLAS-716 Entity update/delete notifications

[Suma Shivaprasad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46731/#review131955
---




client/src/main/java/org/apache/atlas/AtlasClient.java (line 597)


This will cause some users an UI to break rt?. would be better to send a 
mail out to atlas-dev


- Suma Shivaprasad


On April 27, 2016, 1:05 p.m., Shwetha GS wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46731/
> ---
> 
> (Updated April 27, 2016, 1:05 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-716
> https://issues.apache.org/jira/browse/ATLAS-716
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1. Update/delete entity APIs return list of entities created, updated and 
> deleted
> 2. Listeners are called for all the affected entities
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  104c0c5 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
> 813177f 
>   
> addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java 
> ab7e6ee 
>   client/src/main/java/org/apache/atlas/AtlasClient.java be34802 
>   client/src/main/java/org/apache/atlas/EntityAuditEvent.java 460f708 
>   client/src/main/java/org/apache/atlas/SerDe.java PRE-CREATION 
>   client/src/main/java/org/apache/atlas/notification/EntityNotification.java 
> PRE-CREATION 
>   
> client/src/main/java/org/apache/atlas/notification/EntityNotificationImpl.java
>  PRE-CREATION 
>   client/src/main/java/org/apache/atlas/notification/HookNotification.java 
> PRE-CREATION 
>   client/src/test/java/org/apache/atlas/AtlasClientTest.java 8911bf5 
>   
> client/src/test/java/org/apache/atlas/notification/EntityNotificationImplTest.java
>  PRE-CREATION 
>   
> client/src/test/java/org/apache/atlas/notification/HookNotificationTest.java 
> PRE-CREATION 
>   notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 68a868f 
>   
> notification/src/main/java/org/apache/atlas/notification/AbstractNotification.java
>  596f988 
>   
> notification/src/main/java/org/apache/atlas/notification/AbstractNotificationConsumer.java
>  1cadb99 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  1f2df3e 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java
>  ac285aa 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/EntityNotification.java
>  82a1100 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/EntityNotificationImpl.java
>  fda588e 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/NotificationEntityChangeListener.java
>  300cbb5 
>   
> notification/src/main/java/org/apache/atlas/notification/hook/HookNotification.java
>  80c96fa 
>   
> notification/src/test/java/org/apache/atlas/notification/NotificationEntityChangeListenerTest.java
>  PRE-CREATION 
>   
> notification/src/test/java/org/apache/atlas/notification/entity/EntityNotificationImplTest.java
>  385c41f 
>   
> notification/src/test/java/org/apache/atlas/notification/hook/HookNotificationTest.java
>  57b0eea 
>   pom.xml 729b178 
>   repository/pom.xml 533e48b 
>   
> repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 
> 540c308 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
>  5b4bdbf 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  8f11322 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 369a5d5 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  de29e86 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  a3dc7e5 
>   
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
>  9f69940 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  be407a5 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/HBaseTestUtils.java
>  84d7bab 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  ae215f9 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  1f906ed 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java fa94763 
>   server-api/src/main/java/org/apache/atlas/services/MetadataService.java 
> 13d20d8 
>   typesystem/src/main/java/org/apache/atlas/typesystem/Referenceable.java 
> 31f157e 
>   

Re: Review Request 46731: ATLAS-716 Entity update/delete notifications

[Suma Shivaprasad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46731/#review131952
---




client/src/main/java/org/apache/atlas/AtlasClient.java (line 577)


would be better to keep the result consistent for all CRUD operations and 
standardize as EntityResult


- Suma Shivaprasad


On April 27, 2016, 1:05 p.m., Shwetha GS wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46731/
> ---
> 
> (Updated April 27, 2016, 1:05 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-716
> https://issues.apache.org/jira/browse/ATLAS-716
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1. Update/delete entity APIs return list of entities created, updated and 
> deleted
> 2. Listeners are called for all the affected entities
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  104c0c5 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
> 813177f 
>   
> addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java 
> ab7e6ee 
>   client/src/main/java/org/apache/atlas/AtlasClient.java be34802 
>   client/src/main/java/org/apache/atlas/EntityAuditEvent.java 460f708 
>   client/src/main/java/org/apache/atlas/SerDe.java PRE-CREATION 
>   client/src/main/java/org/apache/atlas/notification/EntityNotification.java 
> PRE-CREATION 
>   
> client/src/main/java/org/apache/atlas/notification/EntityNotificationImpl.java
>  PRE-CREATION 
>   client/src/main/java/org/apache/atlas/notification/HookNotification.java 
> PRE-CREATION 
>   client/src/test/java/org/apache/atlas/AtlasClientTest.java 8911bf5 
>   
> client/src/test/java/org/apache/atlas/notification/EntityNotificationImplTest.java
>  PRE-CREATION 
>   
> client/src/test/java/org/apache/atlas/notification/HookNotificationTest.java 
> PRE-CREATION 
>   notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 68a868f 
>   
> notification/src/main/java/org/apache/atlas/notification/AbstractNotification.java
>  596f988 
>   
> notification/src/main/java/org/apache/atlas/notification/AbstractNotificationConsumer.java
>  1cadb99 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  1f2df3e 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java
>  ac285aa 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/EntityNotification.java
>  82a1100 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/EntityNotificationImpl.java
>  fda588e 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/NotificationEntityChangeListener.java
>  300cbb5 
>   
> notification/src/main/java/org/apache/atlas/notification/hook/HookNotification.java
>  80c96fa 
>   
> notification/src/test/java/org/apache/atlas/notification/NotificationEntityChangeListenerTest.java
>  PRE-CREATION 
>   
> notification/src/test/java/org/apache/atlas/notification/entity/EntityNotificationImplTest.java
>  385c41f 
>   
> notification/src/test/java/org/apache/atlas/notification/hook/HookNotificationTest.java
>  57b0eea 
>   pom.xml 729b178 
>   repository/pom.xml 533e48b 
>   
> repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 
> 540c308 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
>  5b4bdbf 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  8f11322 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 369a5d5 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  de29e86 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  a3dc7e5 
>   
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
>  9f69940 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  be407a5 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/HBaseTestUtils.java
>  84d7bab 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  ae215f9 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  1f906ed 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java fa94763 
>   server-api/src/main/java/org/apache/atlas/services/MetadataService.java 
> 13d20d8 
>   typesystem/src/main/java/org/apache/atlas/typesystem/Referenceable.java 
> 31f157e 
>   

[jira] [Commented] (ATLAS-692) Create abstraction layer for graph databases

[Jeffrey Hagelberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272927#comment-15272927
 ] 

Jeffrey Hagelberg commented on ATLAS-692:
-

This task is still waiting for review.  Please commit the patch if there are no 
further comments.

> Create abstraction layer for graph databases
> 
>
> Key: ATLAS-692
> URL: https://issues.apache.org/jira/browse/ATLAS-692
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Jeffrey Hagelberg
>Assignee: Jeffrey Hagelberg
> Fix For: trunk
>
> Attachments: rb46646.patch
>
>
> This task is to put together an abstraction layer that will be used to 
> insulate atlas from the underlying graph database.  Implementations of this 
> interface will be introduced in the other sub tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-742) Avoid downloading hbase multiple times

[Tom Beerbower (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272895#comment-15272895
 ] 

Tom Beerbower commented on ATLAS-742:
-

+1 patch looks good

> Avoid downloading hbase multiple times
> --
>
> Key: ATLAS-742
> URL: https://issues.apache.org/jira/browse/ATLAS-742
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
>Priority: Minor
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-742.patch
>
>
> In distro module, hbase package which is ~100mb is downloaded to target. mvn 
> clean is required during development for clean build and target gets cleaned 
> up and hbase is downloaded everytime. Instead, download hbase to a location 
> outside target and skip download if exists



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-747) Hive CTAS entity registration fails because userName is null

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272594#comment-15272594
 ] 

Suma Shivaprasad commented on ATLAS-747:


Can you pls add an assert in one of the tests if possible?

> Hive CTAS entity registration fails because userName is null
> 
>
> Key: ATLAS-747
> URL: https://issues.apache.org/jira/browse/ATLAS-747
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-747.patch
>
>
> With apache hive 1.2.x and hive CLI, hookContext.getUserName() is null and 
> hive_process entity registration fails as userName is required parameter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (ATLAS-747) Hive CTAS entity registration fails because userName is null

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272593#comment-15272593
 ] 

Suma Shivaprasad edited comment on ATLAS-747 at 5/5/16 4:40 PM:


Should we remove or change this in AtlasHook? Looks like its used only in 
SqoopHook

 /**
 * Returns the logged in user.
 * @return
 */
public static String getUser() {
return getUser(null, null);
}


was (Author: suma.shivaprasad):
Should we remove or change this? Looks like its used only in SqoopHook

 /**
 * Returns the logged in user.
 * @return
 */
public static String getUser() {
return getUser(null, null);
}

> Hive CTAS entity registration fails because userName is null
> 
>
> Key: ATLAS-747
> URL: https://issues.apache.org/jira/browse/ATLAS-747
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-747.patch
>
>
> With apache hive 1.2.x and hive CLI, hookContext.getUserName() is null and 
> hive_process entity registration fails as userName is required parameter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-747) Hive CTAS entity registration fails because userName is null

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272593#comment-15272593
 ] 

Suma Shivaprasad commented on ATLAS-747:


Should we remove or change this? Looks like its used only in SqoopHook

 /**
 * Returns the logged in user.
 * @return
 */
public static String getUser() {
return getUser(null, null);
}

> Hive CTAS entity registration fails because userName is null
> 
>
> Key: ATLAS-747
> URL: https://issues.apache.org/jira/browse/ATLAS-747
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-747.patch
>
>
> With apache hive 1.2.x and hive CLI, hookContext.getUserName() is null and 
> hive_process entity registration fails as userName is required parameter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (ATLAS-750) docker_operator - Improve the integration with swarm

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S resolved ATLAS-750.
---
Resolution: Invalid

Not related to Atlas project

> docker_operator - Improve the integration with swarm
> 
>
> Key: ATLAS-750
> URL: https://issues.apache.org/jira/browse/ATLAS-750
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Amikam Snir
>Priority: Minor
>
> Swarm is not well supported by docker_operator, due to this issue: 
> https://github.com/docker/swarm/issues/475
> In order to fix it, we will use cpu_shares instead of cpus.
> p.s. The default value is None.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-752) Column renames should retain traits/tags

[Suma Shivaprasad (JIRA)]

Suma Shivaprasad created ATLAS-752:
--

 Summary: Column renames should retain traits/tags
 Key: ATLAS-752
 URL: https://issues.apache.org/jira/browse/ATLAS-752
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.7-incubating
Reporter: Suma Shivaprasad
 Fix For: 0.7-incubating


Currently ALTER TABLE REPLACE COLUMNS which renames a column, recreates the 
columns since the name has changed. Instead it should just update column name 
and qualifiedName



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-597) Cluster Name should be changed to cluster Id

[Suma Shivaprasad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated ATLAS-597:
---
Affects Version/s: 0.7-incubating

> Cluster Name should be changed to cluster Id
> 
>
> Key: ATLAS-597
> URL: https://issues.apache.org/jira/browse/ATLAS-597
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Fix For: 0.7-incubating
>
>
> Ambari Cluster Name or User specified cluster name could easily be changed by 
> users and cluster  should be represented by an id which is constant and tied 
> to clusterName which could change



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-751) Make qualifiedName an expression instead of persisting it

[Suma Shivaprasad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated ATLAS-751:
---
Affects Version/s: 0.7-incubating
Fix Version/s: 0.7-incubating

> Make qualifiedName an expression instead of persisting it
> -
>
> Key: ATLAS-751
> URL: https://issues.apache.org/jira/browse/ATLAS-751
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Fix For: 0.7-incubating
>
>
> The qualified Name for an entity should be an expression to solve multiple 
> usecases
> 1. In case of soft deletes,consider the following scenario
> a. Table A -> insert overwrite -> Table B
> If the table A  and B are dropped and recreated again and the insert 
> overwrite query is rerun, then it should create another lineage process since 
> the tables are different here. Similarly for CTAS etc. However the same 
> process will get updated due to the current way of storing qualifiedName 
> 2. Storing qualified names inherently leads to a lot of updates during 
> operations renames for eg: if a table is renames, all its 
> columns.qualifiedName, sd.qualifiedName etc get updated. If this is done with 
> partitions, then the updates will definitely take a lot of time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-751) Make qualifiedName an expression instead of persisting it

[Suma Shivaprasad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated ATLAS-751:
---
Description: 
The qualified Name for an entity should be an expression to solve multiple 
usecases

1. In case of soft deletes,consider the following scenario

a. Table A -> insert overwrite -> Table B

If the table A  and B are dropped and recreated again and the insert overwrite 
query is rerun, then it should create another lineage process since the tables 
are different here. Similarly for CTAS etc. However the same process will get 
updated due to the current way of storing qualifiedName 

2. Storing qualified names inherently leads to a lot of updates during 
operations renames for eg: if a table is renames, all its 
columns.qualifiedName, sd.qualifiedName etc get updated. If this is done with 
partitions, then the updates will definitely take a lot of time.




> Make qualifiedName an expression instead of persisting it
> -
>
> Key: ATLAS-751
> URL: https://issues.apache.org/jira/browse/ATLAS-751
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Suma Shivaprasad
>
> The qualified Name for an entity should be an expression to solve multiple 
> usecases
> 1. In case of soft deletes,consider the following scenario
> a. Table A -> insert overwrite -> Table B
> If the table A  and B are dropped and recreated again and the insert 
> overwrite query is rerun, then it should create another lineage process since 
> the tables are different here. Similarly for CTAS etc. However the same 
> process will get updated due to the current way of storing qualifiedName 
> 2. Storing qualified names inherently leads to a lot of updates during 
> operations renames for eg: if a table is renames, all its 
> columns.qualifiedName, sd.qualifiedName etc get updated. If this is done with 
> partitions, then the updates will definitely take a lot of time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-542) Make qualifiedName and name consistent across all Datasets

[Suma Shivaprasad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated ATLAS-542:
---
Affects Version/s: 0.7-incubating

> Make qualifiedName and name consistent across all Datasets
> --
>
> Key: ATLAS-542
> URL: https://issues.apache.org/jira/browse/ATLAS-542
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Fix For: 0.7-incubating
>
>
> Currently hive_db, hive_column has qualifiedName holding the fully qualified 
> Name and name having the shorter name. 
> However hive_table has the fully qualified Name in "name" and shorter Name in 
> tableName. We should make this consistent for better search experience by 
> removing tableName and keep it consistent with db, column



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-597) Cluster Name should be changed to cluster Id

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272560#comment-15272560
 ] 

Suma Shivaprasad commented on ATLAS-597:


This requires Cluster to be modelled as an entity that is updated periodically 
through Ambari/user defined file.

> Cluster Name should be changed to cluster Id
> 
>
> Key: ATLAS-597
> URL: https://issues.apache.org/jira/browse/ATLAS-597
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Fix For: 0.7-incubating
>
>
> Ambari Cluster Name or User specified cluster name could easily be changed by 
> users which is used in qualified names to dedup entities. For eg: hive_table 
> as  . Hence all entities will be recreated if 
> they are created again unless a mass update on repository is done with new 
> cluster name.
> Hence cluster  should be represented by an id which is constant and tied to 
> clusterName which could change



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-597) Cluster Name should be changed to cluster Id

[Suma Shivaprasad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated ATLAS-597:
---
Description: 
Ambari Cluster Name or User specified cluster name could easily be changed by 
users which is used in qualified names to dedup entities. For eg: hive_table as 
 . Hence all entities will be recreated if they 
are created again unless a mass update on repository is done with new cluster 
name.

Hence cluster  should be represented by an id which is constant and tied to 
clusterName which could change

  was:Ambari Cluster Name or User specified cluster name could easily be 
changed by users and cluster  should be represented by an id which is constant 
and tied to clusterName which could change


> Cluster Name should be changed to cluster Id
> 
>
> Key: ATLAS-597
> URL: https://issues.apache.org/jira/browse/ATLAS-597
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Fix For: 0.7-incubating
>
>
> Ambari Cluster Name or User specified cluster name could easily be changed by 
> users which is used in qualified names to dedup entities. For eg: hive_table 
> as  . Hence all entities will be recreated if 
> they are created again unless a mass update on repository is done with new 
> cluster name.
> Hence cluster  should be represented by an id which is constant and tied to 
> clusterName which could change



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-661) REST API Authentication

[ATLAS QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272555#comment-15272555
 ] 

ATLAS QA commented on ATLAS-661:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12802434/ATLAS-661.2.patch
  against master revision dab9cb7.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 5 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

+1 checkstyle.  The patch generated 0 code style errors.

{color:red}-1 findbugs{color}.  The patch appears to introduce 363 new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in :
 
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.TypesJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.AdminJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.MetadataDiscoveryJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.RexsterGraphJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.EntityJerseyResourceIT
  org.apache.atlas.web.security.NegativeSSLAndKerberosTest
./webapp/target/surefire-reports/junitreports/TEST-org.apache.atlas.web.security.NegativeSSLAndKerberosTest

Test results: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//artifact/patchprocess/newPatchFindbugsWarningshive-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//artifact/patchprocess/newPatchFindbugsWarningssqoop-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//artifact/patchprocess/newPatchFindbugsWarningsstorm-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//artifact/patchprocess/newPatchFindbugsWarningshdfs-model.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//artifact/patchprocess/newPatchFindbugsWarningsclient.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//artifact/patchprocess/newPatchFindbugsWarningsnotification.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//artifact/patchprocess/newPatchFindbugsWarningstypesystem.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//artifact/patchprocess/newPatchFindbugsWarningscommon.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//artifact/patchprocess/newPatchFindbugsWarningstitan.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//artifact/patchprocess/newPatchFindbugsWarningsrepository.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/207//artifact/patchprocess/newPatchFindbugsWarningswebapp.html
Console output: https://builds.apache.org/job/PreCommit-ATLAS-Build/207//console

This message is automatically generated.

> REST API Authentication
> ---
>
> Key: ATLAS-661
> URL: https://issues.apache.org/jira/browse/ATLAS-661
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 0.7-incubating
>Reporter: Erik Bergenholtz
>Assignee: Nixon Rodrigues
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-661.2.patch, ATLAS-661.patch
>
>
> Atlas needs to authorize the REST API invocations; as necessary requests will 
> be forced to authenticate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-635) Process showing old entity name where as actual entity is renamed.

[Suma Shivaprasad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated ATLAS-635:
---
Affects Version/s: (was: trunk)
   0.7-incubating
Fix Version/s: (was: trunk)
   0.7-incubating

> Process showing old entity name where as actual entity is renamed.
> --
>
> Key: ATLAS-635
> URL: https://issues.apache.org/jira/browse/ATLAS-635
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Suma Shivaprasad
> Fix For: 0.7-incubating
>
> Attachments: screenshot-1.png
>
>
> Process showing old entity name where as actual entity is renamed.
> Example:
> 1. Create table as select.
> 2. Rename the target table to new name.
> 3. Now check the lineage: The process is still showing the old table name 
> where as the target table is changed. This is confusing from customer 
> perspective.
> One possible solution is:
> Source >   Process(with old name) > Target(old entity name + marked 
> as delete) ---> Process(rename from old to new) ---> Target(with new entity 
> name).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-619) Partition queries should be normalized when tracking lineage at table

[Suma Shivaprasad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated ATLAS-619:
---
Fix Version/s: 0.7-incubating

> Partition queries should be normalized when  tracking lineage at table
> --
>
> Key: ATLAS-619
> URL: https://issues.apache.org/jira/browse/ATLAS-619
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
> Fix For: 0.7-incubating
>
>
> Partition type and entities are no loner being created after ATLAS-599. Even 
> prior to this change , the linage for partition based queries are being 
> tracked at the table level and not at the partition level which would be 
> ideal. To track at the table level, the queries should be canonicialized for 
> the partition values or only the latest few partition queries should be 
> tracked since otherwise there could be potentially thousands of such queries 
> while tracking lineage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-751) Make qualifiedName an expression instead of persisting it

[Suma Shivaprasad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated ATLAS-751:
---
Issue Type: Improvement  (was: Bug)

> Make qualifiedName an expression instead of persisting it
> -
>
> Key: ATLAS-751
> URL: https://issues.apache.org/jira/browse/ATLAS-751
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Suma Shivaprasad
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-751) Make qualifiedName an expression instead of persisting it

[Suma Shivaprasad (JIRA)]

Suma Shivaprasad created ATLAS-751:
--

 Summary: Make qualifiedName an expression instead of persisting it
 Key: ATLAS-751
 URL: https://issues.apache.org/jira/browse/ATLAS-751
 Project: Atlas
  Issue Type: Bug
Reporter: Suma Shivaprasad






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 46700: ATLAS-497 : Simple Authorization

[Selvamohan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46700/#review131838
---


Ship it!




Ship It!

- Selvamohan Neethiraj


On May 5, 2016, 10:48 a.m., Saqeeb Shaikh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46700/
> ---
> 
> (Updated May 5, 2016, 10:48 a.m.)
> 
> 
> Review request for atlas, Erik Bergenholtz, Shwetha GS, Selvamohan Neethiraj, 
> and Hemanth Yamijala.
> 
> 
> Bugs: ATLAS-497
> https://issues.apache.org/jira/browse/ATLAS-497
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Patch Contains** Simple file based authorization for Atlas REST APIs.
> 
> Implementation contains addition of policy based authorization, where 
> policy-store.txt contains policies for each group / user which will login to 
> Atlas. As initial implementation introducing permissions for 3 Groups of 
> Users : 
> 
> Data Scientist : Users in this Group will have Read only access to the Atlas 
> resources
> Data Steward : Users in this Group will have RWU (read, write, update) 
> accesses to the Atlas resources
> Admin : Users in this Group will have RWUD  (read, write, update, delete) 
> accesses to the Atlas resources
> 
> **Assuming following resources for various APIs**
>  
> TYPE :  for accessing traits,classes 
> ENTITY : for accessing entity, discovery, lineage APIs
> OPERATION : For APIs related to Admin (version, stack, rextergraph related 
> APIs)
> 
> 
> Diffs
> -
> 
>   distro/src/conf/atlas-application.properties 290105f 
>   distro/src/conf/policy-store.txt PRE-CREATION 
>   pom.xml 729b178 
>   webapp/src/main/java/org/apache/atlas/authorize/AtlasAccessRequest.java 
> PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/authorize/AtlasAccessorTypes.java 
> PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/authorize/AtlasActionTypes.java 
> PRE-CREATION 
>   
> webapp/src/main/java/org/apache/atlas/authorize/AtlasAuthorizationException.java
>  PRE-CREATION 
>   
> webapp/src/main/java/org/apache/atlas/authorize/AtlasAuthorizationUtils.java 
> PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/authorize/AtlasAuthorizer.java 
> PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/authorize/AtlasResourceTypes.java 
> PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/authorize/PolicyDef.java PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/authorize/PolicyParser.java 
> PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/authorize/PolicyUtil.java 
> PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/authorize/SimpleAtlasAuthorizer.java 
> PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/util/FileReaderUtil.java PRE-CREATION 
>   
> webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthorizationFilter.java
>  PRE-CREATION 
>   webapp/src/test/java/org/apache/atlas/authorize/PolicyParserTest.java 
> PRE-CREATION 
>   webapp/src/test/java/org/apache/atlas/authorize/PolicyUtilTest.java 
> PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/authorize/SimpleAtlasAuthorizerTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46700/diff/
> 
> 
> Testing
> ---
> 
> Verified authorization by logging in to Atlas as user of different groups. 
> Verified that Data Scientist should not be allowed to write / update or 
> delete  any of the resources. 
> Added test cases to cater to parsing of policies from the policy store. 
> 
> Verifed : mvn clean compile package -Pdist
> 
> 
> Thanks,
> 
> Saqeeb Shaikh
> 
>

[jira] [Commented] (ATLAS-497) Simple Authorization

[Saqeeb Shaikh (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272467#comment-15272467
 ] 

Saqeeb Shaikh commented on ATLAS-497:
-

Thanks for the review comment, [~yhemanth]. Please find the updated patch on 
the review board : https://reviews.apache.org/r/46700/

> Simple Authorization
> 
>
> Key: ATLAS-497
> URL: https://issues.apache.org/jira/browse/ATLAS-497
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.7-incubating
>Reporter: Erik Bergenholtz
>Assignee: Saqeeb Shaikh
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-497.1.patch, ATLAS-497.2.patch, ATLAS-497.patch
>
>
> Atlas needs to support a simple (out of box) authorization mechanism.
> Defined Roles:
> - Data Scientist: provides a read only view (GET)
> - Data Steward: provides a read/edit view (PUT, POST, DELETE)
> - Admin (can do anything)
> All can comment on entity
> Requirements
> - Atlas will implement a simple file based store for providing user to role 
> mapping
> - The out of box experience will be this file based mechanism for 
> authorization



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 46700: ATLAS-497 : Simple Authorization

[Saqeeb Shaikh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46700/
---

(Updated May 5, 2016, 2:48 p.m.)


Review request for atlas, Erik Bergenholtz, Shwetha GS, Selvamohan Neethiraj, 
and Hemanth Yamijala.


Bugs: ATLAS-497
https://issues.apache.org/jira/browse/ATLAS-497


Repository: atlas


Description
---

**Patch Contains** Simple file based authorization for Atlas REST APIs.

Implementation contains addition of policy based authorization, where 
policy-store.txt contains policies for each group / user which will login to 
Atlas. As initial implementation introducing permissions for 3 Groups of Users 
: 

Data Scientist : Users in this Group will have Read only access to the Atlas 
resources
Data Steward : Users in this Group will have RWU (read, write, update) accesses 
to the Atlas resources
Admin : Users in this Group will have RWUD  (read, write, update, delete) 
accesses to the Atlas resources

**Assuming following resources for various APIs**
 
TYPE :  for accessing traits,classes 
ENTITY : for accessing entity, discovery, lineage APIs
OPERATION : For APIs related to Admin (version, stack, rextergraph related APIs)


Diffs (updated)
-

  distro/src/conf/atlas-application.properties 290105f 
  distro/src/conf/policy-store.txt PRE-CREATION 
  pom.xml 729b178 
  webapp/src/main/java/org/apache/atlas/authorize/AtlasAccessRequest.java 
PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/authorize/AtlasAccessorTypes.java 
PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/authorize/AtlasActionTypes.java 
PRE-CREATION 
  
webapp/src/main/java/org/apache/atlas/authorize/AtlasAuthorizationException.java
 PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/authorize/AtlasAuthorizationUtils.java 
PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/authorize/AtlasAuthorizer.java 
PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/authorize/AtlasResourceTypes.java 
PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/authorize/PolicyDef.java PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/authorize/PolicyParser.java 
PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/authorize/PolicyUtil.java PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/authorize/SimpleAtlasAuthorizer.java 
PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/util/FileReaderUtil.java PRE-CREATION 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthorizationFilter.java 
PRE-CREATION 
  webapp/src/test/java/org/apache/atlas/authorize/PolicyParserTest.java 
PRE-CREATION 
  webapp/src/test/java/org/apache/atlas/authorize/PolicyUtilTest.java 
PRE-CREATION 
  
webapp/src/test/java/org/apache/atlas/authorize/SimpleAtlasAuthorizerTest.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/46700/diff/


Testing
---

Verified authorization by logging in to Atlas as user of different groups. 
Verified that Data Scientist should not be allowed to write / update or delete  
any of the resources. 
Added test cases to cater to parsing of policies from the policy store. 

Verifed : mvn clean compile package -Pdist


Thanks,

Saqeeb Shaikh

Re: Review Request 46699: ATLAS-661 : Rest API Authentication.

[Nixon Rodrigues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46699/
---

(Updated May 5, 2016, 2:52 p.m.)


Review request for atlas, Erik Bergenholtz, Shwetha GS, Selvamohan Neethiraj, 
Suma Shivaprasad, and Hemanth Yamijala.


Changes
---

Handled Shwetha's comments -- updated patch contains :: 

1) Removal of default grantedAuthorities() method for LDAP & AD providor (in 
the LDAP and AD User / Group sync section).
2) Prompting username & password for client utilities such as quick_start.py, 
import-hive.sh & atlas_admin.py for basic api authentication.
 AtlasClient changes to support basic authentication in non kerboros mode.
3) Added Test cases for user role mapping & validation for Filebased 
authentication.

**Testing Done**
Manually verifed Authentication feature 
Ran mvn commands 
Added more test cases to cover role mapping.

*NOTE:  AD role check setting is verified & for LDAP role check is pending*


Bugs: ATLAS-661
https://issues.apache.org/jira/browse/ATLAS-661


Repository: atlas


Description
---

Introduced Rest API Authentication.
Introduced role and encoded password in user-credential file.
Authentication to fall back to file type if LDAP fails.
Changes in quick_start.py, atlas_admin.py & import_hive.sh to take password 
argument for basic authentication


Diffs (updated)
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 104c0c5 
  client/src/main/java/org/apache/atlas/AtlasAdminClient.java 473f72a 
  client/src/main/java/org/apache/atlas/AtlasClient.java be34802 
  common/src/main/java/org/apache/atlas/utils/AuthenticationUtil.java 
PRE-CREATION 
  dashboardv2/public/js/utils/Utils.js 7b2db79 
  distro/src/conf/users-credentials.properties 94e7cf4 
  webapp/src/main/java/org/apache/atlas/examples/QuickStart.java 79b8124 
  webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 76784c4 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationEntryPoint.java
 b77bd28 
  
webapp/src/main/java/org/apache/atlas/web/security/AtlasADAuthenticationProvider.java
 96dca45 
  
webapp/src/main/java/org/apache/atlas/web/security/AtlasAbstractAuthenticationProvider.java
 b2b11da 
  
webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationProvider.java
 7f87bd5 
  
webapp/src/main/java/org/apache/atlas/web/security/AtlasFileAuthenticationProvider.java
 b3e3354 
  
webapp/src/main/java/org/apache/atlas/web/security/AtlasLdapAuthenticationProvider.java
 d911c1b 
  webapp/src/main/resources/atlas-admin-site.xml ab8ba3d 
  webapp/src/main/resources/spring-security.xml 8533473 
  webapp/src/main/webapp/WEB-INF/web.xml deb97d9 
  webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 
54d8d92 
  
webapp/src/test/java/org/apache/atlas/web/security/FileAuthenticationTest.java 
7e4c2c8 
  
webapp/src/test/java/org/apache/atlas/web/security/NegativeSSLAndKerberosTest.java
 9aac32a 
  webapp/src/test/java/org/apache/atlas/web/security/SSLAndKerberosTest.java 
2be52e7 
  webapp/src/test/java/org/apache/atlas/web/security/UserDaoTest.java 5b5c70e 

Diff: https://reviews.apache.org/r/46699/diff/


Testing
---

Manual Testing done.
performed mvn clean install
performed mvn clean package -Pdist

Found Failed tests: 
  
HiveLineageJerseyResourceIT.setUp:52->setupInstances:169->table:234->BaseResourceIT.createInstance:129
 » AtlasService
  
Tests run: 10, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 22.476 sec <<< 
FAILURE! - in 
org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest
testDisconnectBidirectionalReferences(org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest)
  Time elapsed: 0.566 sec  <<< FAILURE!


Thanks,

Nixon Rodrigues

[jira] [Updated] (ATLAS-497) Simple Authorization

[Saqeeb Shaikh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Saqeeb Shaikh updated ATLAS-497:

Attachment: ATLAS-497.6.patch

> Simple Authorization
> 
>
> Key: ATLAS-497
> URL: https://issues.apache.org/jira/browse/ATLAS-497
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.7-incubating
>Reporter: Erik Bergenholtz
>Assignee: Saqeeb Shaikh
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-497.1.patch, ATLAS-497.2.patch, ATLAS-497.6.patch, 
> ATLAS-497.patch
>
>
> Atlas needs to support a simple (out of box) authorization mechanism.
> Defined Roles:
> - Data Scientist: provides a read only view (GET)
> - Data Steward: provides a read/edit view (PUT, POST, DELETE)
> - Admin (can do anything)
> All can comment on entity
> Requirements
> - Atlas will implement a simple file based store for providing user to role 
> mapping
> - The out of box experience will be this file based mechanism for 
> authorization



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-661) REST API Authentication

[Nixon Rodrigues (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues updated ATLAS-661:
--
Attachment: ATLAS-661.2.patch

> REST API Authentication
> ---
>
> Key: ATLAS-661
> URL: https://issues.apache.org/jira/browse/ATLAS-661
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 0.7-incubating
>Reporter: Erik Bergenholtz
>Assignee: Nixon Rodrigues
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-661.2.patch, ATLAS-661.patch
>
>
> Atlas needs to authorize the REST API invocations; as necessary requests will 
> be forced to authenticate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-747) Hive CTAS entity registration fails because userName is null

[ATLAS QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272285#comment-15272285
 ] 

ATLAS QA commented on ATLAS-747:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12802389/ATLAS-747.patch
  against master revision dab9cb7.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

+1 checkstyle.  The patch generated 0 code style errors.

{color:red}-1 findbugs{color}.  The patch appears to introduce 361 new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in :
 
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.RexsterGraphJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.AdminJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.EntityJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.MetadataDiscoveryJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.TypesJerseyResourceIT

Test results: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//artifact/patchprocess/newPatchFindbugsWarningswebapp.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//artifact/patchprocess/newPatchFindbugsWarningscommon.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//artifact/patchprocess/newPatchFindbugsWarningssqoop-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//artifact/patchprocess/newPatchFindbugsWarningshdfs-model.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//artifact/patchprocess/newPatchFindbugsWarningsstorm-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//artifact/patchprocess/newPatchFindbugsWarningshive-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//artifact/patchprocess/newPatchFindbugsWarningsrepository.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//artifact/patchprocess/newPatchFindbugsWarningstypesystem.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//artifact/patchprocess/newPatchFindbugsWarningsclient.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//artifact/patchprocess/newPatchFindbugsWarningsnotification.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/205//artifact/patchprocess/newPatchFindbugsWarningstitan.html
Console output: https://builds.apache.org/job/PreCommit-ATLAS-Build/205//console

This message is automatically generated.

> Hive CTAS entity registration fails because userName is null
> 
>
> Key: ATLAS-747
> URL: https://issues.apache.org/jira/browse/ATLAS-747
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-747.patch
>
>
> With apache hive 1.2.x and hive CLI, hookContext.getUserName() is null and 
> hive_process entity registration fails as userName is required parameter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-629) Kafka messages in ATLAS_HOOK might be lost in HA mode at the instant of failover.

[Hemanth Yamijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272262#comment-15272262
 ] 

Hemanth Yamijala commented on ATLAS-629:


Started looking at the approach to fix this problem. With Kafka's (old) high 
level consumer, we only have *atmost-once* delivery because the offsets read 
from the partitions are auto committed by default. So if a message is read and 
offset auto committed, but before the metadata ingest is completed, the server 
reboots, then this message could be lost for processing.

To fix this issue, I am looking at *atleast-once* delivery semantics with 
Kafka, under the assumption that *message processing can be idempotent on the 
server*. Given we use transactions in Titan and also have create-or-update 
semantics, this may be mostly true - but not really sure. Will need to test.

To move to atleast-once processing, the predominant approach people follow 
seems to be to:
* disable auto commit
* Create one ConsumerConnector per partition of a topic.

The latter is because the old high level consumer does not provide for commit 
per partition. It can only commit all offsets read by all partitions it is 
connected to [(Reference 
1)|http://grokbase.com/t/kafka/users/144b80h269/consumerconnector-commitoffsets].
 The above suggestion of one consumer connector per partition has been proposed 
by Kafka experts in many threads [(Reference 
2)|http://mail-archives.apache.org/mod_mbox/kafka-users/201409.mbox/%3CCAHBV8WeYj8ce6G5J0k3a1hGgdNskGv3bsaP8JXSM=kwbnuj...@mail.gmail.com%3E].

The other option could be to move to the newer consumer API in Kafka (with 
0.9+) that (I think) provides better options for handling a per partition 
commit. However, the new consumer is still marked beta, so not really sure. Can 
check with some Kafka committers internally.

For now, I will try out the first approach and see. In the meantime, happy to 
hear feedback from others.

> Kafka messages in ATLAS_HOOK might be lost in HA mode at the instant of 
> failover.
> -
>
> Key: ATLAS-629
> URL: https://issues.apache.org/jira/browse/ATLAS-629
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Hemanth Yamijala
>Assignee: Hemanth Yamijala
>Priority: Critical
> Fix For: 0.7-incubating
>
>
> Write data to Kafka continuously from Hive hook - can do this by writing a 
> script that constantly creates tables. Bring down the Active instance with 
> kill -9. Ensure writes continue after passive becomes active. The expectation 
> is the number of tables created and the number of tables in Atlas match.
> In one test, wrote 180 tables and switched over 6 times from one instance to 
> another. Found that 1 table was lost of the lot. i.e. 179 tables were 
> created, and 1 did not get in.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-682) Set HBase root dir to be relative to test target directory for HBaseBasedAuditRepositoryTest

[ATLAS QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272241#comment-15272241
 ] 

ATLAS QA commented on ATLAS-682:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12802383/ATLAS-682.patch
  against master revision feff0cf.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

+1 checkstyle.  The patch generated 0 code style errors.

{color:red}-1 findbugs{color}.  The patch appears to introduce 362 new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in :
 
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.RexsterGraphJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.AdminJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.EntityJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.MetadataDiscoveryJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.TypesJerseyResourceIT

Test results: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//artifact/patchprocess/newPatchFindbugsWarningswebapp.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//artifact/patchprocess/newPatchFindbugsWarningscommon.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//artifact/patchprocess/newPatchFindbugsWarningssqoop-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//artifact/patchprocess/newPatchFindbugsWarningshdfs-model.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//artifact/patchprocess/newPatchFindbugsWarningsstorm-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//artifact/patchprocess/newPatchFindbugsWarningshive-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//artifact/patchprocess/newPatchFindbugsWarningsrepository.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//artifact/patchprocess/newPatchFindbugsWarningstypesystem.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//artifact/patchprocess/newPatchFindbugsWarningsclient.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//artifact/patchprocess/newPatchFindbugsWarningsnotification.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/204//artifact/patchprocess/newPatchFindbugsWarningstitan.html
Console output: https://builds.apache.org/job/PreCommit-ATLAS-Build/204//console

This message is automatically generated.

> Set HBase root dir to be relative to test target directory for 
> HBaseBasedAuditRepositoryTest
> 
>
> Key: ATLAS-682
> URL: https://issues.apache.org/jira/browse/ATLAS-682
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-682.patch
>
>
> Sometimes when running HBaseBasedAuditRepositoryTest, the test is failing 
> with a timeout. Logs reveal a specific kind of stack trace:
> {code}
> 2016-04-18 15:11:44,437 FATAL - [10.200.4.44:49249.activeMasterManager:] ~ 
> Unhandled exception. Starting shutdown. (HMaster:2098)
> org.apache.hadoop.hbase.util.FileSystemVersionException: HBase file layout 
> needs to be upgraded. You have version null and I want version 8. Consult 
> http://hbase.apache.org/book.html for further information about upgrading 
> HBase. Is your hbase.rootdir valid? If so, you may need to run 'hbase hbck 
> -fixVersionFile'.
>   at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:677)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:455)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:146)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:126)
>   at 
> 

Re: Review Request 46444: ATLAS-659: Fix issues with using embedded HBase on Windows

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46444/#review131809
---


Ship it!




Ship It!

- Shwetha GS


On May 4, 2016, 4:04 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46444/
> ---
> 
> (Updated May 4, 2016, 4:04 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-659
> https://issues.apache.org/jira/browse/ATLAS-659
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-659: Fix issues with using embedded HBase on Windows so that HBase 
> successfully starts and stops as part of Atlas startup/shutdown.
> - Use correct HBase start and stop scripts for Windows
> - Start HBase server after checking existing Atlas server
> - Use Windows-specific template for hbase-site.xml - file URL for 
> hbase.rootdir needs three slashes to avoid error: "Illegal character in 
> authority at index 5"
> - In TestMetadata.py, check mock for HBase Windows startup script
> 
> 
> Diffs
> -
> 
>   distro/src/bin/atlas_config.py 2b6bc825858cc5b36d55cba5b85b2087e8b4ecfe 
>   distro/src/bin/atlas_start.py 4199e372c1357e490171fb1c9328c7529dbe0492 
>   distro/src/bin/atlas_stop.py 5653244cff141b9e9fd68cf6877c4d6ec952ee0f 
>   distro/src/conf/hbase/hbase-site.xml.template 
> 4ace2fdf1a79b92c3c93d46e7c6f218ccd877151 
>   distro/src/test/python/scripts/TestMetadata.py 
> bb74f2004501473fc726fe58e8f9ad1ead3e18f1 
> 
> Diff: https://reviews.apache.org/r/46444/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

[jira] [Commented] (ATLAS-585) NotificationHookConsumer creates new AtlasClient for every message

[ATLAS QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272209#comment-15272209
 ] 

ATLAS QA commented on ATLAS-585:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12802381/ATLAS-585-v2.patch
  against master revision feff0cf.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 12 new 
or modified test files.

  {color:red}-1 javac{color}.  The applied patch generated 119 javac 
compiler warnings (more than the master's current 113 warnings).

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

+1 checkstyle.  The patch generated 0 code style errors.

{color:red}-1 findbugs{color}.  The patch appears to introduce 364 new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in :
 
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.RexsterGraphJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.AdminJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.EntityJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.MetadataDiscoveryJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.TypesJerseyResourceIT

Test results: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/newPatchFindbugsWarningswebapp.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/newPatchFindbugsWarningscommon.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/newPatchFindbugsWarningssqoop-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/newPatchFindbugsWarningshdfs-model.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/newPatchFindbugsWarningsstorm-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/newPatchFindbugsWarningshive-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/newPatchFindbugsWarningsrepository.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/newPatchFindbugsWarningstypesystem.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/newPatchFindbugsWarningsclient.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/newPatchFindbugsWarningsnotification.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/newPatchFindbugsWarningstitan.html
Javac warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/203//artifact/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-ATLAS-Build/203//console

This message is automatically generated.

> NotificationHookConsumer creates new AtlasClient for every message
> --
>
> Key: ATLAS-585
> URL: https://issues.apache.org/jira/browse/ATLAS-585
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-585-v2.patch, ATLAS-585-v3.patch, ATLAS-585.patch
>
>
> Every AtlasClient instance creates com.sun.jersey.api.client.Client which is 
> expensive. As part of ATLAS-577, NotificationHookConsumer creates new atlas 
> client for every message. This is done to pass the user information(the user 
> who created the message/entities) through the APIs. This wouldn't work in 
> kerberos environment where the credentials are not available on the atlas 
> server. We should probably make the API call as atlas user and pass in the 
> actual user information in another query param(like doas user)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-749) Make EntityAuditRepository implementation configurable

[Shwetha G S (JIRA)]

Shwetha G S created ATLAS-749:
-

 Summary: Make EntityAuditRepository implementation configurable
 Key: ATLAS-749
 URL: https://issues.apache.org/jira/browse/ATLAS-749
 Project: Atlas
  Issue Type: Improvement
Reporter: Shwetha G S
 Fix For: 0.7-incubating


EntityAuditRepository is bound to HBaseBasedAuditRepository in 
RepositoryMetadataModule, which essentially forces hbase setup. Make it 
configurable using atlas-application.properties on the lines of DeleteHandler 
configuration



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-742) Avoid downloading hbase multiple times

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272173#comment-15272173
 ] 

Shwetha G S commented on ATLAS-742:
---

Have modified .gitignore in the patch already:)

> Avoid downloading hbase multiple times
> --
>
> Key: ATLAS-742
> URL: https://issues.apache.org/jira/browse/ATLAS-742
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
>Priority: Minor
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-742.patch
>
>
> In distro module, hbase package which is ~100mb is downloaded to target. mvn 
> clean is required during development for clean build and target gets cleaned 
> up and hbase is downloaded everytime. Instead, download hbase to a location 
> outside target and skip download if exists



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-585) NotificationHookConsumer creates new AtlasClient for every message

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated ATLAS-585:
--
Attachment: ATLAS-585-v3.patch

> NotificationHookConsumer creates new AtlasClient for every message
> --
>
> Key: ATLAS-585
> URL: https://issues.apache.org/jira/browse/ATLAS-585
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-585-v2.patch, ATLAS-585-v3.patch, ATLAS-585.patch
>
>
> Every AtlasClient instance creates com.sun.jersey.api.client.Client which is 
> expensive. As part of ATLAS-577, NotificationHookConsumer creates new atlas 
> client for every message. This is done to pass the user information(the user 
> who created the message/entities) through the APIs. This wouldn't work in 
> kerberos environment where the credentials are not available on the atlas 
> server. We should probably make the API call as atlas user and pass in the 
> actual user information in another query param(like doas user)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 46650: ATLAS-585 NotificationHookConsumer creates new AtlasClient for every message

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46650/
---

(Updated May 5, 2016, 10:32 a.m.)


Review request for atlas.


Changes
---

Added test in NotificationHookConsumerIT for user from hook message should be 
same as the one used in audit


Bugs: ATLAS-585
https://issues.apache.org/jira/browse/ATLAS-585


Repository: atlas


Description
---

Added local atlas client that calls the resources directly. So, it bypasses the 
filters


Diffs (updated)
-

  .gitignore ff7c32e 
  addons/falcon-bridge/pom.xml 9b07c9f 
  addons/hive-bridge/pom.xml e125f18 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 104c0c5 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
b0d4c5c 
  addons/sqoop-bridge/pom.xml 4b5dbb1 
  addons/storm-bridge/pom.xml d8d98f5 
  client/src/main/java/org/apache/atlas/AtlasAdminClient.java 473f72a 
  client/src/main/java/org/apache/atlas/AtlasClient.java be34802 
  client/src/main/java/org/apache/atlas/AtlasServiceException.java 6f68a71 
  client/src/main/java/org/apache/atlas/security/SecureClientUtils.java d3b474a 
  client/src/test/java/org/apache/atlas/AtlasClientTest.java 8911bf5 
  distro/src/conf/atlas-log4j.xml 1ac4082 
  
notification/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 1f2df3e 
  
notification/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
 177de6d 
  pom.xml c14e0b3 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
 d83c08c 
  webapp/pom.xml 393863c 
  webapp/src/main/java/org/apache/atlas/LocalAtlasClient.java PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/LocalServletRequest.java PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/examples/QuickStart.java 79b8124 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 PRE-CREATION 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
 01b1cd3 
  webapp/src/main/java/org/apache/atlas/web/filters/AuditFilter.java 9d60e1a 
  webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java 
1eca174 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
6068007 
  webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
b14aa80 
  webapp/src/main/java/org/apache/atlas/web/util/Servlets.java 480a232 
  webapp/src/test/java/org/apache/atlas/LocalAtlasClientTest.java PRE-CREATION 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerIT.java
 03a0d3f 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
 PRE-CREATION 
  
webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationKerberosFilterIT.java
 9e1e08f 
  
webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationKerberosFilterTest.java
 PRE-CREATION 
  
webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationSimpleFilterIT.java
 ca53096 
  
webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationSimpleFilterTest.java
 PRE-CREATION 
  webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 
54d8d92 
  
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 
720ce79 
  
webapp/src/test/java/org/apache/atlas/web/security/BaseSSLAndKerberosTest.java 
d497230 
  webapp/src/test/java/org/apache/atlas/web/security/BaseSecurityTest.java 
270a20d 
  webapp/src/test/java/org/apache/atlas/web/security/SSLTest.java 521c037 

Diff: https://reviews.apache.org/r/46650/diff/


Testing
---

UTs and end to end test


Thanks,

Shwetha GS

[jira] [Commented] (ATLAS-742) Avoid downloading hbase multiple times

[venkata madugundu (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272163#comment-15272163
 ] 

venkata madugundu commented on ATLAS-742:
-

That is a good idea. May be /distro should be added to gitignore 
to avoid being recognized as a change.
May be distro can be named as .distro just so to indicate it is a transient 
directory.

Otherwise, if there is a version attached with the hbase tar, then it is nearly 
equivalent to a Maven artifact (only that it is not available as a maven 
artifact).

> Avoid downloading hbase multiple times
> --
>
> Key: ATLAS-742
> URL: https://issues.apache.org/jira/browse/ATLAS-742
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
>Priority: Minor
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-742.patch
>
>
> In distro module, hbase package which is ~100mb is downloaded to target. mvn 
> clean is required during development for clean build and target gets cleaned 
> up and hbase is downloaded everytime. Instead, download hbase to a location 
> outside target and skip download if exists



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-747) Hive CTAS entity registration fails because userName is null

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated ATLAS-747:
--
Attachment: ATLAS-747.patch

> Hive CTAS entity registration fails because userName is null
> 
>
> Key: ATLAS-747
> URL: https://issues.apache.org/jira/browse/ATLAS-747
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-747.patch
>
>
> With apache hive 1.2.x and hive CLI, hookContext.getUserName() is null and 
> hive_process entity registration fails as userName is required parameter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-742) Avoid downloading hbase multiple times

[ATLAS QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272154#comment-15272154
 ] 

ATLAS QA commented on ATLAS-742:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12802382/ATLAS-742.patch
  against master revision feff0cf.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

+1 checkstyle.  The patch generated 0 code style errors.

{color:red}-1 findbugs{color}.  The patch appears to introduce 362 new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in :
 
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.RexsterGraphJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.AdminJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.EntityJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.MetadataDiscoveryJerseyResourceIT
./webapp/test-output/junitreports/TEST-org.apache.atlas.web.resources.TypesJerseyResourceIT

Test results: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//artifact/patchprocess/newPatchFindbugsWarningswebapp.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//artifact/patchprocess/newPatchFindbugsWarningscommon.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//artifact/patchprocess/newPatchFindbugsWarningssqoop-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//artifact/patchprocess/newPatchFindbugsWarningshdfs-model.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//artifact/patchprocess/newPatchFindbugsWarningsstorm-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//artifact/patchprocess/newPatchFindbugsWarningshive-bridge.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//artifact/patchprocess/newPatchFindbugsWarningsrepository.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//artifact/patchprocess/newPatchFindbugsWarningstypesystem.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//artifact/patchprocess/newPatchFindbugsWarningsclient.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//artifact/patchprocess/newPatchFindbugsWarningsnotification.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ATLAS-Build/202//artifact/patchprocess/newPatchFindbugsWarningstitan.html
Console output: https://builds.apache.org/job/PreCommit-ATLAS-Build/202//console

This message is automatically generated.

> Avoid downloading hbase multiple times
> --
>
> Key: ATLAS-742
> URL: https://issues.apache.org/jira/browse/ATLAS-742
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
>Priority: Minor
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-742.patch
>
>
> In distro module, hbase package which is ~100mb is downloaded to target. mvn 
> clean is required during development for clean build and target gets cleaned 
> up and hbase is downloaded everytime. Instead, download hbase to a location 
> outside target and skip download if exists



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-748) UI: UI should use entity id based lineage and schema

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated ATLAS-748:
--
Summary: UI: UI should use entity id based lineage and schema  (was: UI: UI 
should use entity id based lineage and schema0)

> UI: UI should use entity id based lineage and schema
> 
>
> Key: ATLAS-748
> URL: https://issues.apache.org/jira/browse/ATLAS-748
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Shwetha G S
> Fix For: 0.7-incubating
>
>
> Currently, lineage and schema APIs use table qualified name as argument. With 
> soft delete, there can be multiple entities for a given table qualified 
> name(at most one active entity and >=0 deleted entities). In UI, the lineage 
> and schema is shown in the context of a single entity. So, even lineage and 
> schema should use the lineage and schema corresponding to that entity
> This requires new REST APIs that will be added as part of ATLAS-713



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-748) UI: UI should use entity id based lineage and schema0

[Shwetha G S (JIRA)]

Shwetha G S created ATLAS-748:
-

 Summary: UI: UI should use entity id based lineage and schema0
 Key: ATLAS-748
 URL: https://issues.apache.org/jira/browse/ATLAS-748
 Project: Atlas
  Issue Type: Improvement
Reporter: Shwetha G S
 Fix For: 0.7-incubating


Currently, lineage and schema APIs use table qualified name as argument. With 
soft delete, there can be multiple entities for a given table qualified name(at 
most one active entity and >=0 deleted entities). In UI, the lineage and schema 
is shown in the context of a single entity. So, even lineage and schema should 
use the lineage and schema corresponding to that entity

This requires new REST APIs that will be added as part of ATLAS-713



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (ATLAS-682) Set HBase root dir to be relative to test target directory for HBaseBasedAuditRepositoryTest

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272113#comment-15272113
 ] 

Shwetha G S edited comment on ATLAS-682 at 5/5/16 9:43 AM:
---

HBaseTestingUtility.createLocalHTU() sets hbase.rootdir to target/test-data


was (Author: shwethags):
HBaseTestingUtility.createLocalHTU() sets hbase.rootdir

> Set HBase root dir to be relative to test target directory for 
> HBaseBasedAuditRepositoryTest
> 
>
> Key: ATLAS-682
> URL: https://issues.apache.org/jira/browse/ATLAS-682
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-682.patch
>
>
> Sometimes when running HBaseBasedAuditRepositoryTest, the test is failing 
> with a timeout. Logs reveal a specific kind of stack trace:
> {code}
> 2016-04-18 15:11:44,437 FATAL - [10.200.4.44:49249.activeMasterManager:] ~ 
> Unhandled exception. Starting shutdown. (HMaster:2098)
> org.apache.hadoop.hbase.util.FileSystemVersionException: HBase file layout 
> needs to be upgraded. You have version null and I want version 8. Consult 
> http://hbase.apache.org/book.html for further information about upgrading 
> HBase. Is your hbase.rootdir valid? If so, you may need to run 'hbase hbck 
> -fixVersionFile'.
>   at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:677)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:455)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:146)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:126)
>   at 
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:649)
>   at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
>   at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
>   at java.lang.Thread.run(Thread.java:745)
> {code}
> It appears that the HBase root directory is being set to 
> /tmp/hbase-{username} and if this has left over data it is failing. Under Mac 
> OSX - this is resolving to a very hard to find directory by itself. Filing 
> this to see if we can fix the temporary directories to be under the target 
> folder of the module so that mvn clean actually will fix problems.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-747) Hive CTAS entity registration fails because userName is null

[Shwetha G S (JIRA)]

Shwetha G S created ATLAS-747:
-

 Summary: Hive CTAS entity registration fails because userName is 
null
 Key: ATLAS-747
 URL: https://issues.apache.org/jira/browse/ATLAS-747
 Project: Atlas
  Issue Type: Bug
Reporter: Shwetha G S
Assignee: Shwetha G S
 Fix For: 0.7-incubating


With apache hive 1.2.x and hive CLI, hookContext.getUserName() is null and 
hive_process entity registration fails as userName is required parameter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-682) Set HBase root dir to be relative to test target directory for HBaseBasedAuditRepositoryTest

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272113#comment-15272113
 ] 

Shwetha G S commented on ATLAS-682:
---

HBaseTestingUtility.createLocalHTU() sets hbase.rootdir

> Set HBase root dir to be relative to test target directory for 
> HBaseBasedAuditRepositoryTest
> 
>
> Key: ATLAS-682
> URL: https://issues.apache.org/jira/browse/ATLAS-682
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-682.patch
>
>
> Sometimes when running HBaseBasedAuditRepositoryTest, the test is failing 
> with a timeout. Logs reveal a specific kind of stack trace:
> {code}
> 2016-04-18 15:11:44,437 FATAL - [10.200.4.44:49249.activeMasterManager:] ~ 
> Unhandled exception. Starting shutdown. (HMaster:2098)
> org.apache.hadoop.hbase.util.FileSystemVersionException: HBase file layout 
> needs to be upgraded. You have version null and I want version 8. Consult 
> http://hbase.apache.org/book.html for further information about upgrading 
> HBase. Is your hbase.rootdir valid? If so, you may need to run 'hbase hbck 
> -fixVersionFile'.
>   at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:677)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:455)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:146)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:126)
>   at 
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:649)
>   at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
>   at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
>   at java.lang.Thread.run(Thread.java:745)
> {code}
> It appears that the HBase root directory is being set to 
> /tmp/hbase-{username} and if this has left over data it is failing. Under Mac 
> OSX - this is resolving to a very hard to find directory by itself. Filing 
> this to see if we can fix the temporary directories to be under the target 
> folder of the module so that mvn clean actually will fix problems.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-682) Set HBase root dir to be relative to test target directory for HBaseBasedAuditRepositoryTest

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated ATLAS-682:
--
Attachment: ATLAS-682.patch

> Set HBase root dir to be relative to test target directory for 
> HBaseBasedAuditRepositoryTest
> 
>
> Key: ATLAS-682
> URL: https://issues.apache.org/jira/browse/ATLAS-682
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Shwetha G S
> Attachments: ATLAS-682.patch
>
>
> Sometimes when running HBaseBasedAuditRepositoryTest, the test is failing 
> with a timeout. Logs reveal a specific kind of stack trace:
> {code}
> 2016-04-18 15:11:44,437 FATAL - [10.200.4.44:49249.activeMasterManager:] ~ 
> Unhandled exception. Starting shutdown. (HMaster:2098)
> org.apache.hadoop.hbase.util.FileSystemVersionException: HBase file layout 
> needs to be upgraded. You have version null and I want version 8. Consult 
> http://hbase.apache.org/book.html for further information about upgrading 
> HBase. Is your hbase.rootdir valid? If so, you may need to run 'hbase hbck 
> -fixVersionFile'.
>   at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:677)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:455)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:146)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:126)
>   at 
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:649)
>   at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
>   at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
>   at java.lang.Thread.run(Thread.java:745)
> {code}
> It appears that the HBase root directory is being set to 
> /tmp/hbase-{username} and if this has left over data it is failing. Under Mac 
> OSX - this is resolving to a very hard to find directory by itself. Filing 
> this to see if we can fix the temporary directories to be under the target 
> folder of the module so that mvn clean actually will fix problems.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (ATLAS-682) Set HBase root dir to be relative to test target directory for HBaseBasedAuditRepositoryTest

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S reassigned ATLAS-682:
-

Assignee: Shwetha G S

> Set HBase root dir to be relative to test target directory for 
> HBaseBasedAuditRepositoryTest
> 
>
> Key: ATLAS-682
> URL: https://issues.apache.org/jira/browse/ATLAS-682
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Shwetha G S
>
> Sometimes when running HBaseBasedAuditRepositoryTest, the test is failing 
> with a timeout. Logs reveal a specific kind of stack trace:
> {code}
> 2016-04-18 15:11:44,437 FATAL - [10.200.4.44:49249.activeMasterManager:] ~ 
> Unhandled exception. Starting shutdown. (HMaster:2098)
> org.apache.hadoop.hbase.util.FileSystemVersionException: HBase file layout 
> needs to be upgraded. You have version null and I want version 8. Consult 
> http://hbase.apache.org/book.html for further information about upgrading 
> HBase. Is your hbase.rootdir valid? If so, you may need to run 'hbase hbck 
> -fixVersionFile'.
>   at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:677)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:455)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:146)
>   at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:126)
>   at 
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:649)
>   at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
>   at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
>   at java.lang.Thread.run(Thread.java:745)
> {code}
> It appears that the HBase root directory is being set to 
> /tmp/hbase-{username} and if this has left over data it is failing. Under Mac 
> OSX - this is resolving to a very hard to find directory by itself. Filing 
> this to see if we can fix the temporary directories to be under the target 
> folder of the module so that mvn clean actually will fix problems.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-658) Improve Lineage with Backbone porting

[Keval Bhatt (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272107#comment-15272107
 ] 

Keval Bhatt commented on ATLAS-658:
---

[~yhemanth] Thanks for pointing out, I will check the older implementation and 
get back to you.

> Improve Lineage with Backbone porting 
> --
>
> Key: ATLAS-658
> URL: https://issues.apache.org/jira/browse/ATLAS-658
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-658.patch, atlas_658_hive_lineage_issues.png
>
>
> Improved display of vertices and edges in lineage graph that gets generated. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-742) Avoid downloading hbase multiple times

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated ATLAS-742:
--
Attachment: ATLAS-742.patch

> Avoid downloading hbase multiple times
> --
>
> Key: ATLAS-742
> URL: https://issues.apache.org/jira/browse/ATLAS-742
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
>Priority: Minor
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-742.patch
>
>
> In distro module, hbase package which is ~100mb is downloaded to target. mvn 
> clean is required during development for clean build and target gets cleaned 
> up and hbase is downloaded everytime. Instead, download hbase to a location 
> outside target and skip download if exists



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-585) NotificationHookConsumer creates new AtlasClient for every message

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated ATLAS-585:
--
Attachment: ATLAS-585-v2.patch

> NotificationHookConsumer creates new AtlasClient for every message
> --
>
> Key: ATLAS-585
> URL: https://issues.apache.org/jira/browse/ATLAS-585
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-585-v2.patch, ATLAS-585.patch
>
>
> Every AtlasClient instance creates com.sun.jersey.api.client.Client which is 
> expensive. As part of ATLAS-577, NotificationHookConsumer creates new atlas 
> client for every message. This is done to pass the user information(the user 
> who created the message/entities) through the APIs. This wouldn't work in 
> kerberos environment where the credentials are not available on the atlas 
> server. We should probably make the API call as atlas user and pass in the 
> actual user information in another query param(like doas user)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 46650: ATLAS-585 NotificationHookConsumer creates new AtlasClient for every message

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46650/
---

(Updated May 5, 2016, 9:07 a.m.)


Review request for atlas.


Changes
---

Addressed review comments


Bugs: ATLAS-585
https://issues.apache.org/jira/browse/ATLAS-585


Repository: atlas


Description
---

Added local atlas client that calls the resources directly. So, it bypasses the 
filters


Diffs (updated)
-

  .gitignore ff7c32e 
  addons/falcon-bridge/pom.xml 9b07c9f 
  addons/hive-bridge/pom.xml e125f18 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 104c0c5 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
b0d4c5c 
  addons/sqoop-bridge/pom.xml 4b5dbb1 
  addons/storm-bridge/pom.xml d8d98f5 
  client/src/main/java/org/apache/atlas/AtlasAdminClient.java 473f72a 
  client/src/main/java/org/apache/atlas/AtlasClient.java be34802 
  client/src/main/java/org/apache/atlas/AtlasServiceException.java 6f68a71 
  client/src/main/java/org/apache/atlas/security/SecureClientUtils.java d3b474a 
  client/src/test/java/org/apache/atlas/AtlasClientTest.java 8911bf5 
  distro/src/conf/atlas-log4j.xml 1ac4082 
  
notification/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 1f2df3e 
  
notification/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
 177de6d 
  pom.xml c14e0b3 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
 d83c08c 
  webapp/pom.xml 393863c 
  webapp/src/main/java/org/apache/atlas/LocalAtlasClient.java PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/LocalServletRequest.java PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/examples/QuickStart.java 79b8124 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 PRE-CREATION 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
 01b1cd3 
  webapp/src/main/java/org/apache/atlas/web/filters/AuditFilter.java 9d60e1a 
  webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java 
1eca174 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
6068007 
  webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
b14aa80 
  webapp/src/main/java/org/apache/atlas/web/util/Servlets.java 480a232 
  webapp/src/test/java/org/apache/atlas/LocalAtlasClientTest.java PRE-CREATION 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerIT.java
 03a0d3f 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
 PRE-CREATION 
  
webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationKerberosFilterIT.java
 9e1e08f 
  
webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationKerberosFilterTest.java
 PRE-CREATION 
  
webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationSimpleFilterIT.java
 ca53096 
  
webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationSimpleFilterTest.java
 PRE-CREATION 
  webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 
54d8d92 
  
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 
720ce79 
  
webapp/src/test/java/org/apache/atlas/web/security/BaseSSLAndKerberosTest.java 
d497230 
  webapp/src/test/java/org/apache/atlas/web/security/BaseSecurityTest.java 
270a20d 
  webapp/src/test/java/org/apache/atlas/web/security/SSLTest.java 521c037 

Diff: https://reviews.apache.org/r/46650/diff/


Testing
---

UTs and end to end test


Thanks,

Shwetha GS

Re: Review Request 46650: ATLAS-585 NotificationHookConsumer creates new AtlasClient for every message

[Shwetha GS]

> On May 4, 2016, 7:34 a.m., Hemanth Yamijala wrote:
> > client/src/main/java/org/apache/atlas/AtlasAdminClient.java, line 63
> > 
> >
> > Not related to this patch, but since we are touching it, should we just 
> > get currentUGI and shortUserName here, as we are doing in quick start?

Will add in AtlasClient itself


> On May 4, 2016, 7:34 a.m., Hemanth Yamijala wrote:
> > webapp/src/main/java/org/apache/atlas/LocalAtlasClient.java, line 122
> > 
> >
> > EntityResource is marked as a Singleton resource. While this doesn't 
> > prevent creation of multiple instances, it might just be confusing (for 
> > e.g. if someone makes changes to EntityResource on the basis of this). I 
> > don't feel strongly about it, but if we can do something (either remove the 
> > annotation, or get a single instance here) it might be nice to do.

I wanted to override HttpServletRequest, but something didn't work at that 
time. I will try to clean it up


> On May 4, 2016, 7:34 a.m., Hemanth Yamijala wrote:
> > webapp/src/main/java/org/apache/atlas/web/util/Servlets.java, line 86
> > 
> >
> > For my understanding, will any of our current clients be sending this 
> > query parameter?

AtlasClient uses DelegationTokenAuthenticatedURL which always sets doAs param. 
If there is proxy, there will be both doAs and user.name params. Without proxy, 
there will be just doAs param. So, everyone who uses AtlasClient will set doAs 
param


- Shwetha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46650/#review131623
---


On April 25, 2016, 6:15 p.m., Shwetha GS wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46650/
> ---
> 
> (Updated April 25, 2016, 6:15 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-585
> https://issues.apache.org/jira/browse/ATLAS-585
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Added local atlas client that calls the resources directly. So, it bypasses 
> the filters
> 
> 
> Diffs
> -
> 
>   addons/falcon-bridge/pom.xml 9b07c9f 
>   addons/hive-bridge/pom.xml e125f18 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  104c0c5 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> b0d4c5c 
>   addons/sqoop-bridge/pom.xml 4b5dbb1 
>   addons/storm-bridge/pom.xml d8d98f5 
>   client/src/main/java/org/apache/atlas/AtlasAdminClient.java 473f72a 
>   client/src/main/java/org/apache/atlas/AtlasClient.java be34802 
>   client/src/main/java/org/apache/atlas/AtlasServiceException.java 6f68a71 
>   client/src/main/java/org/apache/atlas/security/SecureClientUtils.java 
> d3b474a 
>   client/src/test/java/org/apache/atlas/AtlasClientTest.java 8911bf5 
>   distro/src/conf/atlas-log4j.xml 1ac4082 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  1f2df3e 
>   
> notification/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
>  177de6d 
>   pom.xml 729b178 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  d83c08c 
>   webapp/pom.xml 1df4112 
>   webapp/src/main/java/org/apache/atlas/LocalAtlasClient.java PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/examples/QuickStart.java 79b8124 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  PRE-CREATION 
>   
> webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
>  01b1cd3 
>   webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java 
> 1eca174 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 6068007 
>   webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
> b14aa80 
>   webapp/src/main/java/org/apache/atlas/web/resources/TypesResource.java 
> 74d89ce 
>   webapp/src/main/java/org/apache/atlas/web/util/Servlets.java 480a232 
>   webapp/src/test/java/org/apache/atlas/LocalAtlasClientTest.java 
> PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerIT.java
>  03a0d3f 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
>  PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationKerberosFilterIT.java
>  9e1e08f 
>   
> webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationKerberosFilterTest.java
>  PRE-CREATION 
>   
> 

[jira] [Commented] (ATLAS-742) Avoid downloading hbase multiple times

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272084#comment-15272084
 ] 

Shwetha G S commented on ATLAS-742:
---

Instead of modifying global state, outside of the project(which can have 
concurrency issues), I was planning to write to /distro/hbase(outside of target)

> Avoid downloading hbase multiple times
> --
>
> Key: ATLAS-742
> URL: https://issues.apache.org/jira/browse/ATLAS-742
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
>Priority: Minor
>
> In distro module, hbase package which is ~100mb is downloaded to target. mvn 
> clean is required during development for clean build and target gets cleaned 
> up and hbase is downloaded everytime. Instead, download hbase to a location 
> outside target and skip download if exists



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-443) Add test for hive create table without columns

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated ATLAS-443:
--
Assignee: (was: Shwetha G S)

> Add test for hive create table without columns
> --
>
> Key: ATLAS-443
> URL: https://issues.apache.org/jira/browse/ATLAS-443
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
> Fix For: trunk
>
>
> Add test for issue fixed in ATLAS-409



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-443) Add test for hive create table without columns

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated ATLAS-443:
--
Attachment: (was: ATLAS-443.patch)

> Add test for hive create table without columns
> --
>
> Key: ATLAS-443
> URL: https://issues.apache.org/jira/browse/ATLAS-443
> Project: Atlas
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: trunk
>
>
> Add test for issue fixed in ATLAS-409



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (ATLAS-670) Entity update/delete operations shouldn't be supported on deleted entities

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S resolved ATLAS-670.
---
Resolution: Not A Problem

Updates to entities like add some comments(if there is any comments attribute), 
add/delete tag should still be supported on deleted entities as they are 
useful. 

> Entity update/delete operations shouldn't be supported on deleted entities
> --
>
> Key: ATLAS-670
> URL: https://issues.apache.org/jira/browse/ATLAS-670
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Shwetha G S
>Assignee: Shwetha G S
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: apache-atlas-nightly #257

[Apache Jenkins Server]

See 

--
[...truncated 16020 lines...]
127.0.0.1 - - [05/May/2016:08:24:21 +] "OPTIONS 
/api/atlas/entities?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:21 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:21 +] "OPTIONS 
/api/atlas/entities/69703579-e04c-4e1d-ac63-31c63f1b1105/audit?count=100=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:21 +] "GET 
/api/atlas/entities/69703579-e04c-4e1d-ac63-31c63f1b1105/audit?count=100 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "OPTIONS 
/api/atlas/types?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "OPTIONS 
/api/atlas/entities?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "OPTIONS 
/api/atlas/types?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "PUT /api/atlas/types HTTP/1.1" 200 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "OPTIONS 
/api/atlas/entities/5300507d-cac8-46b9-a5f2-b3691870a21f?user.name=jenkins 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "GET 
/api/atlas/entities/5300507d-cac8-46b9-a5f2-b3691870a21f HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "OPTIONS 
/api/atlas/entities?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "POST /api/atlas/entities HTTP/1.1" 
400 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "OPTIONS 
/api/atlas/entities?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "POST /api/atlas/entities HTTP/1.1" 
400 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "OPTIONS 
/api/atlas/entities?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "OPTIONS 
/api/atlas/entities?type=hive_db=name=XJEiGfmlYk=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "GET 
/api/atlas/entities?type=hive_db=name=XJEiGfmlYk HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "GET /api/atlas/entities?type=blah 
HTTP/1.1" 400 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:22 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:23 +] "GET 
/api/atlas/entities?type=testxLxHnumUBT HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:23 +] "GET /api/atlas/entities/blah 
HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:23 +] "OPTIONS 
/api/atlas/entities?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:23 +] "POST /api/atlas/entities HTTP/1.1" 
400 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:23 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:23 +] "OPTIONS 
/api/atlas/types/%EE%92%AA%E1%8A%A7%E5%BA%B2%E8%BF%9D%E3%84%A5%EC%B9%99%EE%BD%BF%E2%87%8E%E6%96%AE%EC%8B%92?user.name=jenkins
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:23 +] "GET 
/api/atlas/types/%EE%92%AA%E1%8A%A7%E5%BA%B2%E8%BF%9D%E3%84%A5%EC%B9%99%EE%BD%BF%E2%87%8E%E6%96%AE%EC%8B%92
 HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:23 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:23 +] "OPTIONS 
/api/atlas/entities?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:23 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:23 +] "GET 
/api/atlas/entities/4818f9fa-cfaf-42ab-98ba-50fcaea4dd8f HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
Tests run: 31, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.624 sec <<< 
FAILURE! - in org.apache.atlas.web.resources.EntityJerseyResourceIT
testEntityDeduping(org.apache.atlas.web.resources.EntityJerseyResourceIT)  Time 
elapsed: 1.103 sec  <<< FAILURE!
java.lang.Exception: Waiting timed out after 1000 msec
at 
org.apache.atlas.web.resources.EntityJerseyResourceIT.testEntityDeduping(EntityJerseyResourceIT.java:155)

Running org.apache.atlas.web.resources.TypesJerseyResourceIT
127.0.0.1 - - [05/May/2016:08:24:27 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [05/May/2016:08:24:27 +]