Re: Review Request 72539: RANGER-2839: Assorted improvements for debugging and handling of thread terminations, clean-up of unused data, etc.

2020-05-31 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72539/#review220923
---


Ship it!




Ship It!

- Madhan Neethiraj


On May 31, 2020, 9:36 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72539/
> ---
> 
> (Updated May 31, 2020, 9:36 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2839
> https://issues.apache.org/jira/browse/RANGER-2839
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> These are various improvements/fixes needed for issues discovered during 
> testing of plugins with incremental policy/delta enabled.
> 
> Highlights:
> 1. Threads are named for better debugging information.
> 2. super.join() call is made in a loop to ensure that thread is really 
> terminated even in the face of spurious/out-of-sync interruptions.
> 3. preCleanup() call ensures that clean-up happens at least and exactly once, 
> when incremental policy/tag feature is enabled.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java
>  5d6a4036a 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerUserStoreRefresher.java
>  867ac90ad 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java
>  943b3618f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/PolicyEngine.java
>  297d02fbb 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
>  b59440933 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
>  197c30f0d 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
>  236a4ab17 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/PolicyRefresher.java
>  63a98f2d4 
>   
> security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyAdminCache.java
>  df3c10d3f 
>   
> security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyAdminImpl.java 
> 1c63e94c0 
> 
> 
> Diff: https://reviews.apache.org/r/72539/diff/3/
> 
> 
> Testing
> ---
> 
> Tested along with HDFS and Hive plugins. Works as expected and provides good 
> set of debugging help for tracking creation and termination of threads 
> managed by Ranger plugin.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 72539: RANGER-2839: Assorted improvements for debugging and handling of thread terminations, clean-up of unused data, etc.

2020-05-31 Thread Abhay Kulkarni 


> On May 28, 2020, 5:35 p.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java
> > Line 562 (original), 567 (patched)
> > 
> >
> > serviceDefHelper.getServiceDef() => 
> > serviceDefHelper.getServiceDef().getName()
> >   - note that serviceDefHelper.getServiceDef() can return null, so this 
> > condition must be handled

As RangerServiceDefHelper object's initialization ensures that its reference to 
RangerServiceDef object is never null, RangerServiceDefHelper.getServiceDef() 
API is changed to never return null.


- Abhay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72539/#review220906
---


On May 28, 2020, 5:12 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72539/
> ---
> 
> (Updated May 28, 2020, 5:12 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2839
> https://issues.apache.org/jira/browse/RANGER-2839
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> These are various improvements/fixes needed for issues discovered during 
> testing of plugins with incremental policy/delta enabled.
> 
> Highlights:
> 1. Threads are named for better debugging information.
> 2. super.join() call is made in a loop to ensure that thread is really 
> terminated even in the face of spurious/out-of-sync interruptions.
> 3. preCleanup() call ensures that clean-up happens at least and exactly once, 
> when incremental policy/tag feature is enabled.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java
>  5d6a4036a 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java
>  943b3618f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/PolicyEngine.java
>  297d02fbb 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
>  b59440933 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
>  197c30f0d 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
>  236a4ab17 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/PolicyRefresher.java
>  63a98f2d4 
>   
> security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyAdminCache.java
>  df3c10d3f 
>   
> security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyAdminImpl.java 
> 1c63e94c0 
> 
> 
> Diff: https://reviews.apache.org/r/72539/diff/2/
> 
> 
> Testing
> ---
> 
> Tested along with HDFS and Hive plugins. Works as expected and provides good 
> set of debugging help for tracking creation and termination of threads 
> managed by Ranger plugin.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 72539: RANGER-2839: Assorted improvements for debugging and handling of thread terminations, clean-up of unused data, etc.

2020-05-31 Thread Abhay Kulkarni 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72539/
---

(Updated May 31, 2020, 9:36 p.m.)


Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.


Changes
---

Updated to fix issues raised by review comments.


Bugs: RANGER-2839
https://issues.apache.org/jira/browse/RANGER-2839


Repository: ranger


Description
---

These are various improvements/fixes needed for issues discovered during 
testing of plugins with incremental policy/delta enabled.

Highlights:
1. Threads are named for better debugging information.
2. super.join() call is made in a loop to ensure that thread is really 
terminated even in the face of spurious/out-of-sync interruptions.
3. preCleanup() call ensures that clean-up happens at least and exactly once, 
when incremental policy/tag feature is enabled.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java
 5d6a4036a 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerUserStoreRefresher.java
 867ac90ad 
  
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java
 943b3618f 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/PolicyEngine.java
 297d02fbb 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
 b59440933 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
 197c30f0d 
  
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
 236a4ab17 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/PolicyRefresher.java 
63a98f2d4 
  
security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyAdminCache.java 
df3c10d3f 
  security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyAdminImpl.java 
1c63e94c0 


Diff: https://reviews.apache.org/r/72539/diff/3/

Changes: https://reviews.apache.org/r/72539/diff/2-3/


Testing
---

Tested along with HDFS and Hive plugins. Works as expected and provides good 
set of debugging help for tracking creation and termination of threads managed 
by Ranger plugin.


Thanks,

Abhay Kulkarni