Re: Review Request 71651: RANGER-2630: Ensure that entity deletes are handled even when Atlas sets deleted entity's state as not ACTIVE

2019-10-23 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71651/#review218378
---


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 23, 2019, 11:41 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71651/
> ---
> 
> (Updated Oct. 23, 2019, 11:41 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj, Sarath Subramanian, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2630
> https://issues.apache.org/jira/browse/RANGER-2630
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently, status of Atlas Entity in the delete notification received by 
> TagSync is ACTIVE. Because of this, current implementation of tagsync works 
> as expected. Tagsync should be able to handle delete operaton even if Atlas 
> decides to set the deleted entity's state to something other than ACTIVE.
> 
> 
> Diffs
> -
> 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
>  a4cab28e8 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/EntityNotificationWrapper.java
>  9781aa646 
> 
> 
> Diff: https://reviews.apache.org/r/71651/diff/6/
> 
> 
> Testing
> ---
> 
> Passes all unit tests
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 71651: RANGER-2630: Ensure that entity deletes are handled even when Atlas sets deleted entity's state as not ACTIVE

2019-10-23 Thread Abhay Kulkarni 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71651/
---

(Updated Oct. 23, 2019, 11:41 p.m.)


Review request for ranger, Madhan Neethiraj, Sarath Subramanian, and Velmurugan 
Periasamy.


Changes
---

Addressed review comments


Bugs: RANGER-2630
https://issues.apache.org/jira/browse/RANGER-2630


Repository: ranger


Description
---

Currently, status of Atlas Entity in the delete notification received by 
TagSync is ACTIVE. Because of this, current implementation of tagsync works as 
expected. Tagsync should be able to handle delete operaton even if Atlas 
decides to set the deleted entity's state to something other than ACTIVE.


Diffs (updated)
-

  
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
 a4cab28e8 
  
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/EntityNotificationWrapper.java
 9781aa646 


Diff: https://reviews.apache.org/r/71651/diff/6/

Changes: https://reviews.apache.org/r/71651/diff/5-6/


Testing
---

Passes all unit tests


Thanks,

Abhay Kulkarni

Re: Review Request 71651: RANGER-2630: Ensure that entity deletes are handled even when Atlas sets deleted entity's state as not ACTIVE

2019-10-23 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71651/#review218376
---




tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
Line 120 (original), 121 (patched)


Please add "entityNotification.getIsEntityActive()" condition here as well 
- similar to #142:

  ret = entityNotification.getIsEntityActive() && ! 
entityNotification.getIsEmptyClassifications();
  
It is possible for Atlas to send entity-create/entity-update notifications 
for deleted entities as well - during import operation.



tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
Line 128 (original), 129 (patched)


Comment for #121 applies for line #129 as well.


- Madhan Neethiraj


On Oct. 23, 2019, 9:53 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71651/
> ---
> 
> (Updated Oct. 23, 2019, 9:53 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj, Sarath Subramanian, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2630
> https://issues.apache.org/jira/browse/RANGER-2630
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently, status of Atlas Entity in the delete notification received by 
> TagSync is ACTIVE. Because of this, current implementation of tagsync works 
> as expected. Tagsync should be able to handle delete operaton even if Atlas 
> decides to set the deleted entity's state to something other than ACTIVE.
> 
> 
> Diffs
> -
> 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
>  a4cab28e8 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/EntityNotificationWrapper.java
>  9781aa646 
> 
> 
> Diff: https://reviews.apache.org/r/71651/diff/5/
> 
> 
> Testing
> ---
> 
> Passes all unit tests
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 71651: RANGER-2630: Ensure that entity deletes are handled even when Atlas sets deleted entity's state as not ACTIVE

2019-10-23 Thread Abhay Kulkarni 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71651/
---

(Updated Oct. 23, 2019, 9:53 p.m.)


Review request for ranger, Madhan Neethiraj, Sarath Subramanian, and Velmurugan 
Periasamy.


Changes
---

Addressed edge case due to disassociation of propagated tag from a deleted 
entity


Bugs: RANGER-2630
https://issues.apache.org/jira/browse/RANGER-2630


Repository: ranger


Description
---

Currently, status of Atlas Entity in the delete notification received by 
TagSync is ACTIVE. Because of this, current implementation of tagsync works as 
expected. Tagsync should be able to handle delete operaton even if Atlas 
decides to set the deleted entity's state to something other than ACTIVE.


Diffs (updated)
-

  
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
 a4cab28e8 
  
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/EntityNotificationWrapper.java
 9781aa646 


Diff: https://reviews.apache.org/r/71651/diff/5/

Changes: https://reviews.apache.org/r/71651/diff/4-5/


Testing
---

Passes all unit tests


Thanks,

Abhay Kulkarni

Re: Review Request 71651: RANGER-2630: Ensure that entity deletes are handled even when Atlas sets deleted entity's state as not ACTIVE

2019-10-23 Thread Sarath Subramanian 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71651/#review218363
---


Ship it!




Ship It!

- Sarath Subramanian


On Oct. 23, 2019, 10:38 a.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71651/
> ---
> 
> (Updated Oct. 23, 2019, 10:38 a.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj, Sarath Subramanian, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2630
> https://issues.apache.org/jira/browse/RANGER-2630
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently, status of Atlas Entity in the delete notification received by 
> TagSync is ACTIVE. Because of this, current implementation of tagsync works 
> as expected. Tagsync should be able to handle delete operaton even if Atlas 
> decides to set the deleted entity's state to something other than ACTIVE.
> 
> 
> Diffs
> -
> 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
>  a4cab28e8 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/EntityNotificationWrapper.java
>  9781aa646 
> 
> 
> Diff: https://reviews.apache.org/r/71651/diff/4/
> 
> 
> Testing
> ---
> 
> Passes all unit tests
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 71651: RANGER-2630: Ensure that entity deletes are handled even when Atlas sets deleted entity's state as not ACTIVE

2019-10-23 Thread Abhay Kulkarni 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71651/
---

(Updated Oct. 23, 2019, 5:38 p.m.)


Review request for ranger, Madhan Neethiraj, Sarath Subramanian, and Velmurugan 
Periasamy.


Summary (updated)
-

RANGER-2630: Ensure that entity deletes are handled even when Atlas sets 
deleted entity's state as not ACTIVE


Bugs: RANGER-2630
https://issues.apache.org/jira/browse/RANGER-2630


Repository: ranger


Description
---

Currently, status of Atlas Entity in the delete notification received by 
TagSync is ACTIVE. Because of this, current implementation of tagsync works as 
expected. Tagsync should be able to handle delete operaton even if Atlas 
decides to set the deleted entity's state to something other than ACTIVE.


Diffs (updated)
-

  
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
 a4cab28e8 
  
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/EntityNotificationWrapper.java
 9781aa646 


Diff: https://reviews.apache.org/r/71651/diff/4/

Changes: https://reviews.apache.org/r/71651/diff/3-4/


Testing (updated)
---

Passes all unit tests


Thanks,

Abhay Kulkarni

[jira] [Created] (RANGER-2630) Ensure that entity deletes are handled even when Atlas sets deleted entity's state as not ACTIVE

2019-10-23 Thread Abhay Kulkarni (Jira) 
Abhay Kulkarni created RANGER-2630:
--

 Summary: Ensure that entity deletes are handled even when Atlas 
sets deleted entity's state as not ACTIVE
 Key: RANGER-2630
 URL: https://issues.apache.org/jira/browse/RANGER-2630
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Affects Versions: master
Reporter: Abhay Kulkarni
Assignee: Abhay Kulkarni
 Fix For: master


Currently, status of Atlas Entity in the delete notification received by 
TagSync is ACTIVE. Because of this, current implementation of tagsync works as 
expected. Tagsync should be able to handle delete operaton even if Atlas 
decides to set the deleted entity's state to something other than ACTIVE.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 71552: RANGER-2594: Improve policy validation performance during delete

2019-10-23 Thread Velmurugan Periasamy 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71552/#review218356
---


Ship it!




Ship It!

- Velmurugan Periasamy


On Oct. 17, 2019, 7:25 a.m., Andrew Luo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71552/
> ---
> 
> (Updated Oct. 17, 2019, 7:25 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-2594
> https://issues.apache.org/jira/browse/RANGER-2594
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Deletion of large policies (those with several thousand users or resources) 
> is slow - one reason for this is that we fetch the entire policy during 
> validation, when we only need to check if the policy exists or not.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerPolicyValidator.java
>  a85410782 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerValidator.java
>  f31483e97 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/ServiceStore.java 
> ba7407fc4 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 85db57719 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyDao.java 
> 06a4063a6 
>   security-admin/src/main/resources/META-INF/jpa_named_queries.xml 9714fa91b 
> 
> 
> Diff: https://reviews.apache.org/r/71552/diff/3/
> 
> 
> Testing
> ---
> 
> Unit tests, and tested end-to-end
> 
> 
> Thanks,
> 
> Andrew Luo
> 
>

Re: Review Request 71554: RANGER-2595: Improve policy delete performance using batch delete

2019-10-23 Thread Velmurugan Periasamy 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71554/#review218353
---




security-admin/src/main/resources/META-INF/jpa_named_queries.xml
Lines 1601 (patched)


Patch fails to apply. can you please check? Thanks.

```
$ git apply --check -v 
~/Downloads/0001-RANGER-2595-Improve-policy-delete-performance-using-.patch
Checking patch 
security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java...
Checking patch 
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefAccessTypeDao.java...
Checking patch 
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefConditionDao.java...
Checking patch 
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefDataMaskTypeDao.java...
Checking patch 
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefGroupDao.java...
Checking patch 
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefResourceDao.java...
Checking patch 
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefRoleDao.java...
error: while searching for:
return ret;
}

}

error: patch failed: 
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefRoleDao.java:96
error: 
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefRoleDao.java: 
patch does not apply
Checking patch 
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefUserDao.java...
Checking patch 
security-admin/src/main/resources/META-INF/jpa_named_queries.xml...
Hunk #1 succeeded at 592 (offset 4 lines).
Hunk #2 succeeded at 605 (offset 4 lines).
Hunk #3 succeeded at 623 (offset 4 lines).
Hunk #4 succeeded at 778 (offset 4 lines).
Hunk #5 succeeded at 791 (offset 4 lines).
Hunk #6 succeeded at 808 (offset 4 lines).
error: while searching for:






error: patch failed: 
security-admin/src/main/resources/META-INF/jpa_named_queries.xml:1574
error: security-admin/src/main/resources/META-INF/jpa_named_queries.xml: 
patch does not apply
```


- Velmurugan Periasamy


On Oct. 17, 2019, 5:02 a.m., Andrew Luo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71554/
> ---
> 
> (Updated Oct. 17, 2019, 5:02 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-2595
> https://issues.apache.org/jira/browse/RANGER-2595
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Policy deletion is slow for policies that have a large number of users or 
> resources.  This patch uses batch SQL deletion to speed up deletion.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
> 055cd382b 
>   
> security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefAccessTypeDao.java
>  1ef01bb28 
>   
> security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefConditionDao.java
>  2c04ab483 
>   
> security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefDataMaskTypeDao.java
>  258e3b0ba 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefGroupDao.java 
> 08829d4f7 
>   
> security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefResourceDao.java 
> e259ee864 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefRoleDao.java 
> b92f8066f 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefUserDao.java 
> f7b6131c6 
>   security-admin/src/main/resources/META-INF/jpa_named_queries.xml 7e2139916 
> 
> 
> Diff: https://reviews.apache.org/r/71554/diff/2/
> 
> 
> Testing
> ---
> 
> Tested manually, ran build/tests
> 
> 
> Thanks,
> 
> Andrew Luo
> 
>

[jira] [Commented] (RANGER-2629) Service disabled should not use the last known version of policies

2019-10-23 Thread Peng Xing (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16957678#comment-16957678
 ] 

Peng Xing commented on RANGER-2629:
---

Review Requet：
https://reviews.apache.org/r/71654/

> Service disabled should not use the last known version of policies
> --
>
> Key: RANGER-2629
> URL: https://issues.apache.org/jira/browse/RANGER-2629
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Affects Versions: 2.0.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
>  Labels: patch
> Fix For: 2.1.0
>
> Attachments: 
> 0001-RANGER-2629-Service-disabled-should-not-use-the-last.patch
>
>
> When service is disabled, the plugins should use an empty policy instead of 
> the last known version of policies.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2629) Service disabled should not use the last known version of policies

2019-10-23 Thread Peng Xing (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peng Xing updated RANGER-2629:
--
Attachment: 0001-RANGER-2629-Service-disabled-should-not-use-the-last.patch

> Service disabled should not use the last known version of policies
> --
>
> Key: RANGER-2629
> URL: https://issues.apache.org/jira/browse/RANGER-2629
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Affects Versions: 2.0.0
>Reporter: Peng Xing
>Assignee: Peng Xing
>Priority: Major
>  Labels: patch
> Fix For: 2.1.0
>
> Attachments: 
> 0001-RANGER-2629-Service-disabled-should-not-use-the-last.patch
>
>
> When service is disabled, the plugins should use an empty policy instead of 
> the last known version of policies.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 71654: RANGER-2629 Service disabled should not use the last known version of policies

2019-10-23 Thread Xing Peng 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71654/
---

Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nitin Galave, 
pengjianhua, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
Polavarapu, sam  rome, Venkat Ranganathan, Velmurugan Periasamy, Qiang Zhang, 
and Zsombor Gegesy.


Bugs: RANGER-2629
https://issues.apache.org/jira/browse/RANGER-2629


Repository: ranger


Description
---

When service is disabled, the plugins should use an empty policy instead of the 
last known version of policies.


Diffs
-

  
agents-common/src/main/java/org/apache/ranger/admin/client/RangerAdminRESTClient.java
 86469fd5a 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/PolicyRefresher.java 
7ec8495bc 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerServiceDisabledException.java
 PRE-CREATION 
  
knox-agent/src/main/java/org/apache/ranger/admin/client/RangerAdminJersey2RESTClient.java
 0fafa6e4b 
  security-admin/src/main/java/org/apache/ranger/common/ServiceUtil.java 
2b1a3fa30 


Diff: https://reviews.apache.org/r/71654/diff/1/


Testing
---

Test Result:

Tested.


Thanks,

Xing Peng

[jira] [Created] (RANGER-2629) Service disabled should not use the last known version of policies

2019-10-23 Thread Peng Xing (Jira) 
Peng Xing created RANGER-2629:
-

 Summary: Service disabled should not use the last known version of 
policies
 Key: RANGER-2629
 URL: https://issues.apache.org/jira/browse/RANGER-2629
 Project: Ranger
  Issue Type: Bug
  Components: admin
Affects Versions: 2.0.0
Reporter: Peng Xing
Assignee: Peng Xing


When service is disabled, the plugins should use an empty policy instead of the 
last known version of policies.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)