Re: Review Request 74778: RANGER-4587: blog: dynamic expressions

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74778/#review226045
---


Ship it!




Ship It!

- Barbara Eckman


On Dec. 11, 2023, 2:16 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74778/
> ---
> 
> (Updated Dec. 11, 2023, 2:16 a.m.)
> 
> 
> Review request for ranger, Anand Nadar, Ankita Sinha, Abhay Kulkarni, Monika 
> Kachhadiya, Ramesh Mani, Sailaja Polavarapu, Subhrat Chaudhary, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4587
> https://issues.apache.org/jira/browse/RANGER-4587
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> added blog with details of using dynamic expressions in Apache Ranger policies
> 
> 
> Diffs
> -
> 
>   docs/src/site/resources/blogs/dynamic_expressions.html PRE-CREATION 
>   docs/src/site/xdoc/blogs.xml 2f81ef7c4 
> 
> 
> Diff: https://reviews.apache.org/r/74778/diff/1/
> 
> 
> Testing
> ---
> 
> - built and ran updated docs with mvn site:run
> - verified that the new blog is seen in the blog list
> - verified that the new blog renders correctly in browser
> 
> 
> File Attachments
> 
> 
> Apache Ranger - dynamic expressions.pdf
>   
> https://reviews.apache.org/media/uploaded/files/2023/12/11/a8d4e7f9-5ec7-4527-9d3c-245c15f49573__Apache_Ranger_-_dynamic_expressions.pdf
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 74778: RANGER-4587: blog: dynamic expressions

[Barbara Eckman via Review Board]

> On Dec. 11, 2023, 6 p.m., Barbara Eckman wrote:
> > This is a great idea! It's very useful to have all dynamic expressions in 
> > one place!
> > 
> > One example is incorrect, though:  "data_source in 
> > (${{USER.allowedSources}})".  The problem is that IN requires an array on 
> > the RHS, but (${{USER.allowedSources}}) evaluates to "(source1, source2)." 
> > Using GET_USER_ATTR_Q just adds quotes around yields "source1, source2", 
> > but it doesn't create an array.  A split is needed, or the notation 
> > [[USER.allowedSources]], a la RANGER-2434.
> 
> Barbara Eckman wrote:
> My preferred method of addressing this would be changing the type of the 
> UserStore from Map> to Map ArrayList> or some such.  Apache Atlas handles array-valued 
> attributes of tags. It would be great if Ranger handled arrays too!
> 
> Madhan Neethiraj wrote:
> @Barbara -  good point! I will add a note in the blog that the expression 
> should evaluate to a valid value for the context. In this example, 
> data_source is a numeric column with USER.allowedSources having list of 
> numbers separated by a comma - like 10,11,18.
> 
> Let's look for an easier way to handle the use case you raised. How about 
> this?:  data_source in (QUOTED_CSV(USER.allowedSources))
> 
> Barbara Eckman wrote:
> I tried QUOTED_CSV() but got this error:
> ERROR [main] (RangerRequestScriptEvaluator.java:264) - 
> RangerRequestScriptEvaluator.evaluateScript(): failed to evaluate script
> javax.script.ScriptException: ReferenceError: "QUOTED_CSV" is not defined 
> in  at line number 1
> 
> Barbara Eckman wrote:
> Maybe i didn't understand your comment, but I tried the following:
> USER.EventType = "10,20,30"
> condition: 10 in (USER.EventType)
> Error message:
> ERROR [main] (RangerRequestScriptEvaluator.java:264) - 
> RangerRequestScriptEvaluator.evaluateScript(): failed to evaluate script
> javax.script.ScriptException: TypeError: Right hand side of "in" cannot 
> be non-Object, found string in  at line number 1
> 
> Am I missing something?
> 
> Madhan Neethiraj wrote:
> Barbara - the example row-filter expression (given below) would be 
> evaluated by Ranger policy engine to "data_source in (10,11,16)", which would 
> then be executed in a SQL engine.
>   data_source in (${{USER.allowedSources}})
> 
> Also, reference to QUOTED_CSV() in my earlier comment was about an 
> enhancement TBD. Ranger community will continue to review and enhance to 
> support more use cases.
> 
> In your use case here, user attribute EventType is a 
> comma-separated-value string. This needs to be parsed into an array/list 
> before checking if a value exists is present. Perhaps we should look into 
> adding PARSE_CSV() method? Then the condition can be written as:
>   PARSE_CSV(USER.EventType).includes(10)

Hi Madhan,

Sorry for the confusion, I should have realized that in a row filter the syntax 
will be of the underlying engine.

You said: 
"In your use case here, user attribute EventType is a comma-separated-value 
string. This needs to be parsed into an array/list before checking if a value 
exists is present. Perhaps we should look into adding PARSE_CSV() method? Then 
the condition can be written as:
  PARSE_CSV(USER.EventType).includes(10)"

This is PRECISELY the use case for RANGER-2434: eliminate the need for 
"split"-ing delimited strings into arrays by using the notation [[ 
,'']].  Personally, I prefer this notation to 
something referring to "comma-separated values" and then taking a different 
delimiter as a param.:-). It does seems to work in row filters as well as other 
policy conditions, but I admit I haven't tried it with hive.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74778/#review226035
---


On Dec. 11, 2023, 2:16 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74778/
> ---
> 
> (Updated Dec. 11, 2023, 2:16 a.m.)
> 
> 
> Review request for ranger, Anand Nadar, Ankita Sinha, Abhay Kulkarni, Monika 
> Kachhadiya, Ramesh Mani, Sailaja Polavarapu, Subhrat Chaudhary, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4587
> https://issues.apache.org/jira/browse/RANGER-4587
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> added blog with details of using dynamic expressions in Apache Ranger policies
> 
> 
> Diffs
> -
> 
>   docs/src/site/resources/blogs/dynamic_expressions.html PRE-CREATION 
>   docs/src/site/xdoc/blogs.xml 2f81ef7c4 
> 
> 
> Diff: https://reviews.apache.org/r/74778/diff/1/
> 
> 
> Testing
> ---
> 
> - built and ran updated docs with mvn site:run
> - verified 

Re: Review Request 74778: RANGER-4587: blog: dynamic expressions

[Barbara Eckman via Review Board]

> On Dec. 11, 2023, 6 p.m., Barbara Eckman wrote:
> > This is a great idea! It's very useful to have all dynamic expressions in 
> > one place!
> > 
> > One example is incorrect, though:  "data_source in 
> > (${{USER.allowedSources}})".  The problem is that IN requires an array on 
> > the RHS, but (${{USER.allowedSources}}) evaluates to "(source1, source2)." 
> > Using GET_USER_ATTR_Q just adds quotes around yields "source1, source2", 
> > but it doesn't create an array.  A split is needed, or the notation 
> > [[USER.allowedSources]], a la RANGER-2434.
> 
> Barbara Eckman wrote:
> My preferred method of addressing this would be changing the type of the 
> UserStore from Map> to Map ArrayList> or some such.  Apache Atlas handles array-valued 
> attributes of tags. It would be great if Ranger handled arrays too!
> 
> Madhan Neethiraj wrote:
> @Barbara -  good point! I will add a note in the blog that the expression 
> should evaluate to a valid value for the context. In this example, 
> data_source is a numeric column with USER.allowedSources having list of 
> numbers separated by a comma - like 10,11,18.
> 
> Let's look for an easier way to handle the use case you raised. How about 
> this?:  data_source in (QUOTED_CSV(USER.allowedSources))
> 
> Barbara Eckman wrote:
> I tried QUOTED_CSV() but got this error:
> ERROR [main] (RangerRequestScriptEvaluator.java:264) - 
> RangerRequestScriptEvaluator.evaluateScript(): failed to evaluate script
> javax.script.ScriptException: ReferenceError: "QUOTED_CSV" is not defined 
> in  at line number 1

Maybe i didn't understand your comment, but I tried the following:
USER.EventType = "10,20,30"
condition: 10 in (USER.EventType)
Error message:
ERROR [main] (RangerRequestScriptEvaluator.java:264) - 
RangerRequestScriptEvaluator.evaluateScript(): failed to evaluate script
javax.script.ScriptException: TypeError: Right hand side of "in" cannot be 
non-Object, found string in  at line number 1

Am I missing something?


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74778/#review226035
---


On Dec. 11, 2023, 2:16 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74778/
> ---
> 
> (Updated Dec. 11, 2023, 2:16 a.m.)
> 
> 
> Review request for ranger, Anand Nadar, Ankita Sinha, Abhay Kulkarni, Monika 
> Kachhadiya, Ramesh Mani, Sailaja Polavarapu, Subhrat Chaudhary, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4587
> https://issues.apache.org/jira/browse/RANGER-4587
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> added blog with details of using dynamic expressions in Apache Ranger policies
> 
> 
> Diffs
> -
> 
>   docs/src/site/resources/blogs/dynamic_expressions.html PRE-CREATION 
>   docs/src/site/xdoc/blogs.xml 2f81ef7c4 
> 
> 
> Diff: https://reviews.apache.org/r/74778/diff/1/
> 
> 
> Testing
> ---
> 
> - built and ran updated docs with mvn site:run
> - verified that the new blog is seen in the blog list
> - verified that the new blog renders correctly in browser
> 
> 
> File Attachments
> 
> 
> Apache Ranger - dynamic expressions.pdf
>   
> https://reviews.apache.org/media/uploaded/files/2023/12/11/a8d4e7f9-5ec7-4527-9d3c-245c15f49573__Apache_Ranger_-_dynamic_expressions.pdf
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 74778: RANGER-4587: blog: dynamic expressions

[Barbara Eckman via Review Board]

> On Dec. 11, 2023, 6 p.m., Barbara Eckman wrote:
> > This is a great idea! It's very useful to have all dynamic expressions in 
> > one place!
> > 
> > One example is incorrect, though:  "data_source in 
> > (${{USER.allowedSources}})".  The problem is that IN requires an array on 
> > the RHS, but (${{USER.allowedSources}}) evaluates to "(source1, source2)." 
> > Using GET_USER_ATTR_Q just adds quotes around yields "source1, source2", 
> > but it doesn't create an array.  A split is needed, or the notation 
> > [[USER.allowedSources]], a la RANGER-2434.
> 
> Barbara Eckman wrote:
> My preferred method of addressing this would be changing the type of the 
> UserStore from Map> to Map ArrayList> or some such.  Apache Atlas handles array-valued 
> attributes of tags. It would be great if Ranger handled arrays too!
> 
> Madhan Neethiraj wrote:
> @Barbara -  good point! I will add a note in the blog that the expression 
> should evaluate to a valid value for the context. In this example, 
> data_source is a numeric column with USER.allowedSources having list of 
> numbers separated by a comma - like 10,11,18.
> 
> Let's look for an easier way to handle the use case you raised. How about 
> this?:  data_source in (QUOTED_CSV(USER.allowedSources))

I tried QUOTED_CSV() but got this error:
ERROR [main] (RangerRequestScriptEvaluator.java:264) - 
RangerRequestScriptEvaluator.evaluateScript(): failed to evaluate script
javax.script.ScriptException: ReferenceError: "QUOTED_CSV" is not defined in 
 at line number 1


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74778/#review226035
---


On Dec. 11, 2023, 2:16 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74778/
> ---
> 
> (Updated Dec. 11, 2023, 2:16 a.m.)
> 
> 
> Review request for ranger, Anand Nadar, Ankita Sinha, Abhay Kulkarni, Monika 
> Kachhadiya, Ramesh Mani, Sailaja Polavarapu, Subhrat Chaudhary, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4587
> https://issues.apache.org/jira/browse/RANGER-4587
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> added blog with details of using dynamic expressions in Apache Ranger policies
> 
> 
> Diffs
> -
> 
>   docs/src/site/resources/blogs/dynamic_expressions.html PRE-CREATION 
>   docs/src/site/xdoc/blogs.xml 2f81ef7c4 
> 
> 
> Diff: https://reviews.apache.org/r/74778/diff/1/
> 
> 
> Testing
> ---
> 
> - built and ran updated docs with mvn site:run
> - verified that the new blog is seen in the blog list
> - verified that the new blog renders correctly in browser
> 
> 
> File Attachments
> 
> 
> Apache Ranger - dynamic expressions.pdf
>   
> https://reviews.apache.org/media/uploaded/files/2023/12/11/a8d4e7f9-5ec7-4527-9d3c-245c15f49573__Apache_Ranger_-_dynamic_expressions.pdf
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 74778: RANGER-4587: blog: dynamic expressions

[Barbara Eckman via Review Board]

> On Dec. 11, 2023, 6 p.m., Barbara Eckman wrote:
> > This is a great idea! It's very useful to have all dynamic expressions in 
> > one place!
> > 
> > One example is incorrect, though:  "data_source in 
> > (${{USER.allowedSources}})".  The problem is that IN requires an array on 
> > the RHS, but (${{USER.allowedSources}}) evaluates to "(source1, source2)." 
> > Using GET_USER_ATTR_Q just adds quotes around yields "source1, source2", 
> > but it doesn't create an array.  A split is needed, or the notation 
> > [[USER.allowedSources]], a la RANGER-2434.

My preferred method of addressing this would be changing the type of the 
UserStore from Map> to Map> or some such.  Apache Atlas handles array-valued attributes 
of tags. It would be great if Ranger handled arrays too!


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74778/#review226035
---


On Dec. 11, 2023, 2:16 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74778/
> ---
> 
> (Updated Dec. 11, 2023, 2:16 a.m.)
> 
> 
> Review request for ranger, Anand Nadar, Ankita Sinha, Abhay Kulkarni, Monika 
> Kachhadiya, Ramesh Mani, Sailaja Polavarapu, Subhrat Chaudhary, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4587
> https://issues.apache.org/jira/browse/RANGER-4587
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> added blog with details of using dynamic expressions in Apache Ranger policies
> 
> 
> Diffs
> -
> 
>   docs/src/site/resources/blogs/dynamic_expressions.html PRE-CREATION 
>   docs/src/site/xdoc/blogs.xml 2f81ef7c4 
> 
> 
> Diff: https://reviews.apache.org/r/74778/diff/1/
> 
> 
> Testing
> ---
> 
> - built and ran updated docs with mvn site:run
> - verified that the new blog is seen in the blog list
> - verified that the new blog renders correctly in browser
> 
> 
> File Attachments
> 
> 
> Apache Ranger - dynamic expressions.pdf
>   
> https://reviews.apache.org/media/uploaded/files/2023/12/11/a8d4e7f9-5ec7-4527-9d3c-245c15f49573__Apache_Ranger_-_dynamic_expressions.pdf
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 74778: RANGER-4587: blog: dynamic expressions

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74778/#review226035
---



This is a great idea! It's very useful to have all dynamic expressions in one 
place!

One example is incorrect, though:  "data_source in (${{USER.allowedSources}})". 
 The problem is that IN requires an array on the RHS, but 
(${{USER.allowedSources}}) evaluates to "(source1, source2)." Using 
GET_USER_ATTR_Q just adds quotes around yields "source1, source2", but it 
doesn't create an array.  A split is needed, or the notation 
[[USER.allowedSources]], a la RANGER-2434.

- Barbara Eckman


On Dec. 11, 2023, 2:16 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74778/
> ---
> 
> (Updated Dec. 11, 2023, 2:16 a.m.)
> 
> 
> Review request for ranger, Anand Nadar, Ankita Sinha, Abhay Kulkarni, Monika 
> Kachhadiya, Ramesh Mani, Sailaja Polavarapu, Subhrat Chaudhary, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4587
> https://issues.apache.org/jira/browse/RANGER-4587
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> added blog with details of using dynamic expressions in Apache Ranger policies
> 
> 
> Diffs
> -
> 
>   docs/src/site/resources/blogs/dynamic_expressions.html PRE-CREATION 
>   docs/src/site/xdoc/blogs.xml 2f81ef7c4 
> 
> 
> Diff: https://reviews.apache.org/r/74778/diff/1/
> 
> 
> Testing
> ---
> 
> - built and ran updated docs with mvn site:run
> - verified that the new blog is seen in the blog list
> - verified that the new blog renders correctly in browser
> 
> 
> File Attachments
> 
> 
> Apache Ranger - dynamic expressions.pdf
>   
> https://reviews.apache.org/media/uploaded/files/2023/12/11/a8d4e7f9-5ec7-4527-9d3c-245c15f49573__Apache_Ranger_-_dynamic_expressions.pdf
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/
---

(Updated Sept. 25, 2023, 7:34 p.m.)


Review request for ranger and madhan.


Changes
---

added JsvaScriptEdits import to RangerRequestScriptEvaluator


Repository: ranger


Description
---

RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
policy conditions


Diffs (updated)
-

  agents-common/dev-support/spotbugsIncludeFile.xml PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
 7ac20764f 


Diff: https://reviews.apache.org/r/74609/diff/7/

Changes: https://reviews.apache.org/r/74609/diff/6-7/


Testing
---


File Attachments


0001-RANGER-4234.patch
  
https://reviews.apache.org/media/uploaded/files/2023/09/22/19380bc1-e7ad-4fab-880d-324e38783c05__0001-RANGER-4234.patch
RANGER-4234.patch
  
https://reviews.apache.org/media/uploaded/files/2023/09/22/0f300895-c3a6-4dbd-8bd3-23dc1d6e3b49__RANGER-4234.patch


Thanks,

Barbara Eckman

Re: Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/
---

(Updated Sept. 25, 2023, 7:02 p.m.)


Review request for ranger and madhan.


Changes
---

version 6 includes changes to RangerRequestScriptEvaluator, which were 
inadvertently left out of the most recent patch (for which I apologize 
abjectly!). It was made by cloning the current version of origin/master, making 
that one change, and generating a patch.


Repository: ranger


Description
---

RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
policy conditions


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
 7ac20764f 


Diff: https://reviews.apache.org/r/74609/diff/6/

Changes: https://reviews.apache.org/r/74609/diff/5-6/


Testing
---


File Attachments


0001-RANGER-4234.patch
  
https://reviews.apache.org/media/uploaded/files/2023/09/22/19380bc1-e7ad-4fab-880d-324e38783c05__0001-RANGER-4234.patch
RANGER-4234.patch
  
https://reviews.apache.org/media/uploaded/files/2023/09/22/0f300895-c3a6-4dbd-8bd3-23dc1d6e3b49__RANGER-4234.patch


Thanks,

Barbara Eckman

Re: Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/
---

(Updated Sept. 22, 2023, 5:46 p.m.)


Review request for ranger and madhan.


Repository: ranger


Description
---

RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
policy conditions


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultRowFilterPolicyItemEvaluator.java
 d2b3e746b 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java 
PRE-CREATION 
  
agents-common/src/test/java/org/apache/ranger/plugin/util/JavaScriptEditsTest.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74609/diff/5/

Changes: https://reviews.apache.org/r/74609/diff/4-5/


Testing
---


File Attachments (updated)


0001-RANGER-4234.patch
  
https://reviews.apache.org/media/uploaded/files/2023/09/22/19380bc1-e7ad-4fab-880d-324e38783c05__0001-RANGER-4234.patch
RANGER-4234.patch
  
https://reviews.apache.org/media/uploaded/files/2023/09/22/0f300895-c3a6-4dbd-8bd3-23dc1d6e3b49__RANGER-4234.patch


Thanks,

Barbara Eckman

Re: Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

> On Sept. 21, 2023, 4:06 p.m., Madhan Neethiraj wrote:
> > Barbara - the recent update to the review doesn't include the changes. Can 
> > you please update again?
> 
> Barbara Eckman wrote:
> Yes, I apologize. I'm having trouble generating a new patch:
> % git format-patch origin/master
> 0001-doubleBracketsReplace-stuff.patch
> 0002-added-replaceDoubleBrackets-to-RangerDefaultRowFilte.patch
> Trying to upload 0002, I get this error: "The uploaded diff uses short 
> revisions, but Review Board requires full revisions.  Please generate a new 
> diff using the --full-index parameter."
> When I do this the result is the same:
>  % git format-patch --full-index origin/master
> 0001-doubleBracketsReplace-stuff.patch
> 0002-added-replaceDoubleBrackets-to-RangerDefaultRowFilte.patch
> snd I get the same error on trying to upload.
> 
> *** FINALLY, I ADDED THE 0002 FILE UNDER UPDATE DIFF-> ADD FILE. ***  I 
> apologize for the inconvenience.

new DIFF created and posted, to get rid of the confusion.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/#review225776
---


On Sept. 22, 2023, 3:43 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74609/
> ---
> 
> (Updated Sept. 22, 2023, 3:43 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
> policy conditions
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultRowFilterPolicyItemEvaluator.java
>  d2b3e746b 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74609/diff/4/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> 0001-RANGER-4234.patch
>   
> https://reviews.apache.org/media/uploaded/files/2023/09/22/19380bc1-e7ad-4fab-880d-324e38783c05__0001-RANGER-4234.patch
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/
---

(Updated Sept. 22, 2023, 3:43 p.m.)


Review request for ranger and madhan.


Changes
---

new diff produced to avoid confusion.


Repository: ranger


Description
---

RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
policy conditions


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultRowFilterPolicyItemEvaluator.java
 d2b3e746b 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/74609/diff/4/

Changes: https://reviews.apache.org/r/74609/diff/3-4/


Testing
---


File Attachments (updated)


0001-RANGER-4234.patch
  
https://reviews.apache.org/media/uploaded/files/2023/09/22/19380bc1-e7ad-4fab-880d-324e38783c05__0001-RANGER-4234.patch


Thanks,

Barbara Eckman

Re: Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

> On Sept. 21, 2023, 4:06 p.m., Madhan Neethiraj wrote:
> > Barbara - the recent update to the review doesn't include the changes. Can 
> > you please update again?

Yes, I apologize. I'm having trouble generating a new patch:
% git format-patch origin/master
0001-doubleBracketsReplace-stuff.patch
0002-added-replaceDoubleBrackets-to-RangerDefaultRowFilte.patch
Trying to upload 0002, I get this error: "The uploaded diff uses short 
revisions, but Review Board requires full revisions.  Please generate a new 
diff using the --full-index parameter."
When I do this the result is the same:
 % git format-patch --full-index origin/master
0001-doubleBracketsReplace-stuff.patch
0002-added-replaceDoubleBrackets-to-RangerDefaultRowFilte.patch
snd I get the same error on trying to upload.

*** FINALLY, I ADDED THE 0002 FILE UNDER UPDATE DIFF-> ADD FILE. ***  I 
apologize for the inconvenience.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/#review225776
---


On Sept. 18, 2023, 8:36 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74609/
> ---
> 
> (Updated Sept. 18, 2023, 8:36 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
> policy conditions
> 
> 
> Diffs
> -
> 
>   agents-common/dev-support/spotbugsIncludeFile.xml PRE-CREATION 
>   agents-common/pom.xml b753c1368 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
> 9e5a94b1a 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
>  7ac20764f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74609/diff/3/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> 0001-doubleBracketsReplace-stuff.patch
>   
> https://reviews.apache.org/media/uploaded/files/2023/09/18/fe38eef9-22e6-4c91-85a0-46fe337ba87b__0001-doubleBracketsReplace-stuff.patch
> 0001-doubleBracketsReplace-stuff.patch
>   
> https://reviews.apache.org/media/uploaded/files/2023/09/18/062f1050-96d7-4ed0-9008-fd65311ea7b0__0001-doubleBracketsReplace-stuff.patch
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

> On Sept. 18, 2023, 9:55 p.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java
> > Lines 1783 (patched)
> > 
> >
> > Can this validation/rewrite be moved out of RangerPolicy class i.e., 
> > outside of org.apache.ranger.plugin.model package contents. Classes in this 
> > package are used in REST API serialization/deserialization, hence prefer to 
> > only have simple get/set methods.
> > 
> > Also, given "[[...]]" is handled in  
> > RangerRequestScriptEvaluator.evaluateScript(), are these updates in 
> > RangerPolicy necessary?
> 
> Barbara Eckman wrote:
> In my experience, with row filters, using 
> RangerRequestScriptEvaluator.evaluateScript() alone strips out part of the 
> policy condition.  I'm happy to move it out of RangerPolicy class,though.  
> Perhaps we can have a call to discuss.

Please ignore the request for a call.  More depth of explanation: 
RangerRequestScriptEvaluator.evaluateScript() understands the script as 
"USER.partner", and returns the condition as 
"[["comcast,xglobal,cox"]].includes(jsonAttr.partner)".  I can't call 
replaceDoubleBrackets() in my RecordFilterJavaScript class because by this time 
the delimiter is part of what needs to be matched, and comma isn't allowed by 
the regex.
I've moved the hasDoubleBrackets() handling for row filters to 
RangerDefaultRowFilterPolicyItemEvaluator().  It works great now, and this 
should be a more reasonable place than in the model.

Policy Condition: "RangerRequestScriptEvaluator.evaluateScript()


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/#review225769
---


On Sept. 18, 2023, 8:36 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74609/
> ---
> 
> (Updated Sept. 18, 2023, 8:36 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
> policy conditions
> 
> 
> Diffs
> -
> 
>   agents-common/dev-support/spotbugsIncludeFile.xml PRE-CREATION 
>   agents-common/pom.xml b753c1368 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
> 9e5a94b1a 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
>  7ac20764f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74609/diff/3/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> 0001-doubleBracketsReplace-stuff.patch
>   
> https://reviews.apache.org/media/uploaded/files/2023/09/18/fe38eef9-22e6-4c91-85a0-46fe337ba87b__0001-doubleBracketsReplace-stuff.patch
> 0001-doubleBracketsReplace-stuff.patch
>   
> https://reviews.apache.org/media/uploaded/files/2023/09/18/062f1050-96d7-4ed0-9008-fd65311ea7b0__0001-doubleBracketsReplace-stuff.patch
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

> On Sept. 18, 2023, 9:55 p.m., Madhan Neethiraj wrote:
> > agents-common/pom.xml
> > Line 43 (original), 43 (patched)
> > 
> >
> > Is this update necessary? pom.xml for other Ranger modulues have 
> > relativePath as "..". If this update is not needed, I suggedt to revert.

no, I'll revert it. Sorry.


> On Sept. 18, 2023, 9:55 p.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java
> > Lines 1783 (patched)
> > 
> >
> > Can this validation/rewrite be moved out of RangerPolicy class i.e., 
> > outside of org.apache.ranger.plugin.model package contents. Classes in this 
> > package are used in REST API serialization/deserialization, hence prefer to 
> > only have simple get/set methods.
> > 
> > Also, given "[[...]]" is handled in  
> > RangerRequestScriptEvaluator.evaluateScript(), are these updates in 
> > RangerPolicy necessary?

In my experience, with row filters, using 
RangerRequestScriptEvaluator.evaluateScript() alone strips out part of the 
policy condition.  I'm happy to move it out of RangerPolicy class,though.  
Perhaps we can have a call to discuss.


> On Sept. 18, 2023, 9:55 p.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
> > Lines 32 (patched)
> > 
> >
> > It will to include few examples of input/output.

Will do.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/#review225769
---


On Sept. 18, 2023, 8:36 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74609/
> ---
> 
> (Updated Sept. 18, 2023, 8:36 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
> policy conditions
> 
> 
> Diffs
> -
> 
>   agents-common/dev-support/spotbugsIncludeFile.xml PRE-CREATION 
>   agents-common/pom.xml b753c1368 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
> 9e5a94b1a 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
>  7ac20764f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74609/diff/3/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> 0001-doubleBracketsReplace-stuff.patch
>   
> https://reviews.apache.org/media/uploaded/files/2023/09/18/fe38eef9-22e6-4c91-85a0-46fe337ba87b__0001-doubleBracketsReplace-stuff.patch
> 0001-doubleBracketsReplace-stuff.patch
>   
> https://reviews.apache.org/media/uploaded/files/2023/09/18/062f1050-96d7-4ed0-9008-fd65311ea7b0__0001-doubleBracketsReplace-stuff.patch
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

> On Sept. 18, 2023, 6:18 p.m., Don Bosco Durai wrote:
> >

Sorry, I uploaded diff 2 twice. there's no difference between diff 2 and 3.


> On Sept. 18, 2023, 6:18 p.m., Don Bosco Durai wrote:
> > agents-common/pom.xml
> > Line 20 (original), 20 (patched)
> > 
> >
> > Should we have the prefix "-3.0.0-20230918"?

No, sorry, that was from my internal version. I'll change the pom file.


> On Sept. 18, 2023, 6:18 p.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
> > Lines 51 (patched)
> > 
> >
> > I hope this won't throw an out of bound exception, but only returns null

It only returns null.  Here's what happens for the input 'var result = 
[[USER.dataCollection_Sensitivity]]. filter(function(n){ return [[n,"_"]][0] == 
TAG.value}); result.length !=0', for the 2 iterations of the while loop:
INFO [main] (JavaScriptEdits.java:56) - ==> 
s0=[[USER.dataCollection_Sensitivity]] s1=USER.dataCollection_Sensitivity 
s2=null delim=,
 INFO [main] (JavaScriptEdits.java:56) - ==> s0=[[n,"_"]] s1=n s2=,"_" delim=_


> On Sept. 18, 2023, 6:18 p.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
> > Lines 56 (patched)
> > 
> >
> > Would there be an case where s0 to s2 will be null or not present

if m.find() is true, then s0 and s1 won't be null, because they represent what 
was found.  s2 can be null, as we saw above, with no problems.


> On Sept. 18, 2023, 6:18 p.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
> > Lines 60 (patched)
> > 
> >
> > Should we store the original string and log both here to see what the 
> > original string was transformed?

Done


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/#review225767
---


On Sept. 18, 2023, 8:36 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74609/
> ---
> 
> (Updated Sept. 18, 2023, 8:36 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
> policy conditions
> 
> 
> Diffs
> -
> 
>   agents-common/dev-support/spotbugsIncludeFile.xml PRE-CREATION 
>   agents-common/pom.xml b753c1368 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
> 9e5a94b1a 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
>  7ac20764f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74609/diff/3/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> 0001-doubleBracketsReplace-stuff.patch
>   
> https://reviews.apache.org/media/uploaded/files/2023/09/18/fe38eef9-22e6-4c91-85a0-46fe337ba87b__0001-doubleBracketsReplace-stuff.patch
> 0001-doubleBracketsReplace-stuff.patch
>   
> https://reviews.apache.org/media/uploaded/files/2023/09/18/062f1050-96d7-4ed0-9008-fd65311ea7b0__0001-doubleBracketsReplace-stuff.patch
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/
---

(Updated Sept. 18, 2023, 8:36 p.m.)


Review request for ranger and madhan.


Repository: ranger


Description
---

RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
policy conditions


Diffs (updated)
-

  agents-common/dev-support/spotbugsIncludeFile.xml PRE-CREATION 
  agents-common/pom.xml b753c1368 
  agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
9e5a94b1a 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
 7ac20764f 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/74609/diff/3/

Changes: https://reviews.apache.org/r/74609/diff/2-3/


Testing
---


File Attachments (updated)


0001-doubleBracketsReplace-stuff.patch
  
https://reviews.apache.org/media/uploaded/files/2023/09/18/fe38eef9-22e6-4c91-85a0-46fe337ba87b__0001-doubleBracketsReplace-stuff.patch
0001-doubleBracketsReplace-stuff.patch
  
https://reviews.apache.org/media/uploaded/files/2023/09/18/062f1050-96d7-4ed0-9008-fd65311ea7b0__0001-doubleBracketsReplace-stuff.patch


Thanks,

Barbara Eckman

Re: Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/
---

(Updated Sept. 18, 2023, 8:31 p.m.)


Review request for ranger and madhan.


Repository: ranger


Description
---

RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
policy conditions


Diffs (updated)
-

  agents-common/dev-support/spotbugsIncludeFile.xml PRE-CREATION 
  agents-common/pom.xml b753c1368 
  agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
9e5a94b1a 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
 7ac20764f 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/74609/diff/2/

Changes: https://reviews.apache.org/r/74609/diff/1-2/


Testing
---


File Attachments (updated)


0001-doubleBracketsReplace-stuff.patch
  
https://reviews.apache.org/media/uploaded/files/2023/09/18/fe38eef9-22e6-4c91-85a0-46fe337ba87b__0001-doubleBracketsReplace-stuff.patch


Thanks,

Barbara Eckman

Review Request 74609: RANGER-4234

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/
---

Review request for ranger and madhan.


Repository: ranger


Description
---

RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
policy conditions


Diffs
-

  agents-common/dev-support/spotbugsIncludeFile.xml PRE-CREATION 
  agents-common/pom.xml b753c1368 
  agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
9e5a94b1a 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
 7ac20764f 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/74609/diff/1/


Testing
---


Thanks,

Barbara Eckman

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/
---

(Updated Nov. 29, 2022, 10:52 p.m.)


Review request for ranger and madhan.


Bugs: Ranger-3855
https://issues.apache.org/jira/browse/Ranger-3855


Repository: ranger


Description
---

RangerExternalUserStoreRetriever class Ranger-3855

Ranger version 3.0.0 provides a means, via a context enricher, to add or 
retrieve attributes to the database of users for whom Ranger controls access. 
This permits syntax like "Dumbo" in $USER.aliases any Ranger policy condition, 
including row and tag filters.   This greatly enhances the ability to provide 
custom Attribute-based Access Control based on the specific business needs of 
one's organization.

I believe that the original assumption was that such attributes would be added 
to AD/LDAP and enter Ranger via regular user sync's. However, this process does 
not currently work with Azure AD, which many organizations use. Neither does it 
provide timely support for organizations for whom adding each new attribute to 
AD would be subject to prolonged scrutiny by overworked security teams.  

In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
written a RangerExternalUserStoreRetriever class which adds arbitrary 
attributes to Ranger users via external API calls, thus freeing additions to 
the UserStore from dependency on AD/LDAP.   We have also written a 
RangerRoleUserStoreRetriever class, which transforms role membership into user 
attributes, for ease of use in complex policy conditions.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/LICENSE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/NOTICE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/RangerMultiSourceUserStoreRetriever.java
 PRE-CREATION 
  dev-support/spotbugsIncludeFile.xml 3621e8c08 


Diff: https://reviews.apache.org/r/74142/diff/8/

Changes: https://reviews.apache.org/r/74142/diff/7-8/


Testing
---


Thanks,

Barbara Eckman

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Nov. 18, 2022, 6:02 p.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
> > Lines 59 (patched)
> > 
> >
> > allUserAttrMap is initialized only in init(), which is called only 
> > once. Shouldn't this be updated in retrieveUserStoreInfo() as well?
> 
> Barbara Eckman wrote:
> I made it an instance member, and moved its initialization outside of 
> init() to line 42.  Does that not work?

Done.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224887
---


On Nov. 28, 2022, 9:45 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Nov. 28, 2022, 9:45 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/LICENSE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/NOTICE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74142/diff/7/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Nov. 29, 2022, 5 a.m., Madhan Neethiraj wrote:
> >

Thank you for your comments, as always!


> On Nov. 29, 2022, 5 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
> > Lines 39 (patched)
> > 
> >
> > Consider renaming AllRangerUserStoreRetrievers to 
> > RangerMultiSourceUserStoreRetriever.

Cool.


> On Nov. 29, 2022, 5 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
> > Lines 42 (patched)
> > 
> >
> > To avoid creation of RangerRoleUtils for every call to 
> > retrieveUserStoreInfo(), in #246, consider replacing "RangerRoles 
> > rangerRoles" here with "RangerRoleUtil roleUtil". This should be refreshed 
> > only when role updates are downloaded in #62.

Yep.


> On Nov. 29, 2022, 5 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
> > Lines 43 (patched)
> > 
> >
> > allUserAttrsMap is used only within retrieveUserStoreInfo(). Consider 
> > moving #43 to within this method i.e. avoid instance member.

Yep, I thought of that last night after I uploaded the Diff.


> On Nov. 29, 2022, 5 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
> > Lines 44 (patched)
> > 
> >
> > enricherOptionsMap => retrieverOptions

Yep.  That name was from a much earlier version and never updated.


> On Nov. 29, 2022, 5 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
> > Lines 62 (patched)
> > 
> >
> > Consider sending existing roles version to getRolesIfUpdated() so that 
> > roles will be downloaded from Ranger only when there are updates.

Good.


> On Nov. 29, 2022, 5 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
> > Lines 55 (patched)
> > 
> >
> > To ensure httpClient is closed, consider moving #55 to within try() at 
> > #62.

good


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224914
---


On Nov. 28, 2022, 9:45 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Nov. 28, 2022, 9:45 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/
---

(Updated Nov. 28, 2022, 9:45 p.m.)


Review request for ranger and madhan.


Changes
---

refactor AllRetrievers to move population of userStore from init() to 
retrieveUserStoreInfo


Bugs: Ranger-3855
https://issues.apache.org/jira/browse/Ranger-3855


Repository: ranger


Description
---

RangerExternalUserStoreRetriever class Ranger-3855

Ranger version 3.0.0 provides a means, via a context enricher, to add or 
retrieve attributes to the database of users for whom Ranger controls access. 
This permits syntax like "Dumbo" in $USER.aliases any Ranger policy condition, 
including row and tag filters.   This greatly enhances the ability to provide 
custom Attribute-based Access Control based on the specific business needs of 
one's organization.

I believe that the original assumption was that such attributes would be added 
to AD/LDAP and enter Ranger via regular user sync's. However, this process does 
not currently work with Azure AD, which many organizations use. Neither does it 
provide timely support for organizations for whom adding each new attribute to 
AD would be subject to prolonged scrutiny by overworked security teams.  

In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
written a RangerExternalUserStoreRetriever class which adds arbitrary 
attributes to Ranger users via external API calls, thus freeing additions to 
the UserStore from dependency on AD/LDAP.   We have also written a 
RangerRoleUserStoreRetriever class, which transforms role membership into user 
attributes, for ease of use in complex policy conditions.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/LICENSE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/NOTICE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74142/diff/7/

Changes: https://reviews.apache.org/r/74142/diff/6-7/


Testing
---


Thanks,

Barbara Eckman

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Nov. 18, 2022, 6:02 p.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
> > Lines 59 (patched)
> > 
> >
> > allUserAttrMap is initialized only in init(), which is called only 
> > once. Shouldn't this be updated in retrieveUserStoreInfo() as well?

I made it an instance member, and moved its initialization outside of init() to 
line 42.  Does that not work?


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224887
---


On Nov. 18, 2022, 5:45 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Nov. 18, 2022, 5:45 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/LICENSE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/NOTICE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74142/diff/6/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/
---

(Updated Nov. 18, 2022, 5:45 p.m.)


Review request for ranger and madhan.


Bugs: Ranger-3855
https://issues.apache.org/jira/browse/Ranger-3855


Repository: ranger


Description
---

RangerExternalUserStoreRetriever class Ranger-3855

Ranger version 3.0.0 provides a means, via a context enricher, to add or 
retrieve attributes to the database of users for whom Ranger controls access. 
This permits syntax like "Dumbo" in $USER.aliases any Ranger policy condition, 
including row and tag filters.   This greatly enhances the ability to provide 
custom Attribute-based Access Control based on the specific business needs of 
one's organization.

I believe that the original assumption was that such attributes would be added 
to AD/LDAP and enter Ranger via regular user sync's. However, this process does 
not currently work with Azure AD, which many organizations use. Neither does it 
provide timely support for organizations for whom adding each new attribute to 
AD would be subject to prolonged scrutiny by overworked security teams.  

In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
written a RangerExternalUserStoreRetriever class which adds arbitrary 
attributes to Ranger users via external API calls, thus freeing additions to 
the UserStore from dependency on AD/LDAP.   We have also written a 
RangerRoleUserStoreRetriever class, which transforms role membership into user 
attributes, for ease of use in complex policy conditions.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/LICENSE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/NOTICE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74142/diff/6/

Changes: https://reviews.apache.org/r/74142/diff/5-6/


Testing
---


Thanks,

Barbara Eckman

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Oct. 25, 2022, 7:43 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/RangerExternalUserStoreRetriever.java
> > Lines 56 (patched)
> > 
> >
> > Since the user-store returned by a given instance of 
> > RangerExternalUserStoreRetriever always contains the same userAttrMap, it 
> > might be useful to instantiate RangerUserStore in init() method itself.
> 
> Barbara Eckman wrote:
> userStore is used in retrieveUserStoreInfo method as well as init, so it 
> needs to be initiated outside.

Now that the AllRangerUserStoreRetrievers class covers all retrievers, any 
given instance of it could contain one of 4 userAttrMaps, depending on the 
retriever used.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224827
---


On Nov. 17, 2022, 9:15 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Nov. 17, 2022, 9:15 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/LICENSE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/NOTICE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74142/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Nov. 18, 2022, 12:29 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
> > Lines 66 (patched)
> > 
> >
> > retrieveUserStoreInfo() returns userStore initialized in init() method. 
> > How are changes to user-attributes handled? For example:
> > 1. changes to user-roles assignment
> > 2. changes to attributes in remote server (retrieved via GetFromURL()).
> > 3. changes to attributes loaded from data file (retrieved via 
> > GetFromDataFile()

You are right. I moved the assignment of userAttrsMap to userStore to the 
retrieveUserStoreInfo method. Hopefully that will solve this problem.


> On Nov. 18, 2022, 12:29 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
> > Lines 198 (patched)
> > 
> >
> > Please review and update the package name: 
> > com.comcast.dx.ranger.contextenricher.externalretrievers

Sorry, yes, done.


> On Nov. 18, 2022, 12:29 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
> > Lines 222 (patched)
> > 
> >
> > rangerRoles is an instance member, hence it is not necessary to send as 
> > method parameter. Please review and update.

Done


> On Nov. 18, 2022, 12:29 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
> > Lines 226 (patched)
> > 
> >
> > gson is unused. Please review and remove.

done.


> On Nov. 18, 2022, 12:29 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
> > Lines 99 (patched)
> > 
> >
> > roleName here actually refers to the prefix of the roleName in Ranger. 
> > Also, roleName here is the name of the user-attribute. Hence, consider 
> > renaming this to attrName.
> > 
> >   "retriever2_role": "attrName=salesRegion",
> >   "retriever3_role": "attrName=sensitivityLevel"

Good idea. I need the "." for pattern matching against role names, in case one 
attrName is a prefix for another one, eg these roles: region.northeast and 
regionSales.northeast.  But (duh) I can add that to the attrName before 
matching against roles, in the code rather requiring it in the servicedef.  So 
I will change this.  Thanks!


> On Nov. 18, 2022, 12:29 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
> > Lines 132 (patched)
> > 
> >
> > Given each retriever handles a single attribute, 'name' option many not 
> > be useful/necessary. Consider the following simplified option string:
> >  "retriever0_api":  
> > "attrName=partner,userStoreURL=http://localhost:8000/security/getPartnersByUser;,
> >  "retriever1_api":  
> > "attrName=ownedResources,dataFile=/var/ranger/data/userOwnerResource.txt",
> >  "retriever2_role": "attrName=salesRegion",
> >  "retriever3_role": "attrName=sensitivityLevel"

What you say makes sense.  Originally I had thought the name was useful for 
documentation, at least. But now each retriever does the same thing: maps a 
user to an attribute.  So if attrName is given, no additional documentation is 
needed.
So I made the change.


> On Nov. 18, 2022, 12:29 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
> > Lines 134 (patched)
> > 
> >
> > serviceType and serviceName are unused. Please review and remove from 
> > #134 and #135.

Yes, sorry, I thought I'd fixed that.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224885
---


On Nov. 17, 2022, 9:15 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Nov. 17, 2022, 9:15 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> 

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/
---

(Updated Nov. 17, 2022, 9:15 p.m.)


Review request for ranger and madhan.


Bugs: Ranger-3855
https://issues.apache.org/jira/browse/Ranger-3855


Repository: ranger


Description
---

RangerExternalUserStoreRetriever class Ranger-3855

Ranger version 3.0.0 provides a means, via a context enricher, to add or 
retrieve attributes to the database of users for whom Ranger controls access. 
This permits syntax like "Dumbo" in $USER.aliases any Ranger policy condition, 
including row and tag filters.   This greatly enhances the ability to provide 
custom Attribute-based Access Control based on the specific business needs of 
one's organization.

I believe that the original assumption was that such attributes would be added 
to AD/LDAP and enter Ranger via regular user sync's. However, this process does 
not currently work with Azure AD, which many organizations use. Neither does it 
provide timely support for organizations for whom adding each new attribute to 
AD would be subject to prolonged scrutiny by overworked security teams.  

In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
written a RangerExternalUserStoreRetriever class which adds arbitrary 
attributes to Ranger users via external API calls, thus freeing additions to 
the UserStore from dependency on AD/LDAP.   We have also written a 
RangerRoleUserStoreRetriever class, which transforms role membership into user 
attributes, for ease of use in complex policy conditions.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/LICENSE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/NOTICE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74142/diff/5/

Changes: https://reviews.apache.org/r/74142/diff/4-5/


Testing
---


Thanks,

Barbara Eckman

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Oct. 15, 2022, 4:17 p.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 44 (patched)
> > 
> >
> > Yes, it was a question. I was not sure whether we are printing would 
> > have sensitive information. If it doesn't the suggestion is not to print 
> > them. The reason being, it is common for applications to be configured to 
> > DEBUG level during troubleshooting sessions and also in some cases, these 
> > logs are sent to external systems like DataDog (in the cloud) or other log 
> > aggregation tools and it would be difficult to enforce any policies in 
> > those tools.

I removed debug logging for sensitive info everywhere. It will make debugging 
harder but I agree it is necessary.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224797
---


On Nov. 3, 2022, 5:36 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Nov. 3, 2022, 5:36 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/LICENSE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/NOTICE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
>  PRE-CREATION 
>   dev-support/spotbugsIncludeFile.xml 3621e8c08 
>   plugin-nestedstructure/README.md ea878f6a2 
> 
> 
> Diff: https://reviews.apache.org/r/74142/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/
---

(Updated Nov. 3, 2022, 5:36 p.m.)


Review request for ranger and madhan.


Bugs: Ranger-3855
https://issues.apache.org/jira/browse/Ranger-3855


Repository: ranger


Description
---

RangerExternalUserStoreRetriever class Ranger-3855

Ranger version 3.0.0 provides a means, via a context enricher, to add or 
retrieve attributes to the database of users for whom Ranger controls access. 
This permits syntax like "Dumbo" in $USER.aliases any Ranger policy condition, 
including row and tag filters.   This greatly enhances the ability to provide 
custom Attribute-based Access Control based on the specific business needs of 
one's organization.

I believe that the original assumption was that such attributes would be added 
to AD/LDAP and enter Ranger via regular user sync's. However, this process does 
not currently work with Azure AD, which many organizations use. Neither does it 
provide timely support for organizations for whom adding each new attribute to 
AD would be subject to prolonged scrutiny by overworked security teams.  

In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
written a RangerExternalUserStoreRetriever class which adds arbitrary 
attributes to Ranger users via external API calls, thus freeing additions to 
the UserStore from dependency on AD/LDAP.   We have also written a 
RangerRoleUserStoreRetriever class, which transforms role membership into user 
attributes, for ease of use in complex policy conditions.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/LICENSE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/NOTICE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
 PRE-CREATION 
  dev-support/spotbugsIncludeFile.xml 3621e8c08 
  plugin-nestedstructure/README.md ea878f6a2 


Diff: https://reviews.apache.org/r/74142/diff/4/

Changes: https://reviews.apache.org/r/74142/diff/3-4/


Testing
---


Thanks,

Barbara Eckman

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/
---

(Updated Nov. 2, 2022, 8:18 p.m.)


Review request for ranger and madhan.


Changes
---

Here's what I hope will be close to the final revision:  I refactored into a 
single userStoreRetriever class that calls methods to accommodate multiple 
userStoreRetriever enrichers of the two currently supported source types: "api" 
(formerly "external") and "role" (roles-based).


Bugs: Ranger-3855
https://issues.apache.org/jira/browse/Ranger-3855


Repository: ranger


Description
---

RangerExternalUserStoreRetriever class Ranger-3855

Ranger version 3.0.0 provides a means, via a context enricher, to add or 
retrieve attributes to the database of users for whom Ranger controls access. 
This permits syntax like "Dumbo" in $USER.aliases any Ranger policy condition, 
including row and tag filters.   This greatly enhances the ability to provide 
custom Attribute-based Access Control based on the specific business needs of 
one's organization.

I believe that the original assumption was that such attributes would be added 
to AD/LDAP and enter Ranger via regular user sync's. However, this process does 
not currently work with Azure AD, which many organizations use. Neither does it 
provide timely support for organizations for whom adding each new attribute to 
AD would be subject to prolonged scrutiny by overworked security teams.  

In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
written a RangerExternalUserStoreRetriever class which adds arbitrary 
attributes to Ranger users via external API calls, thus freeing additions to 
the UserStore from dependency on AD/LDAP.   We have also written a 
RangerRoleUserStoreRetriever class, which transforms role membership into user 
attributes, for ease of use in complex policy conditions.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/LICENSE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/NOTICE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
 PRE-CREATION 
  dev-support/spotbugsIncludeFile.xml 3621e8c08 
  plugin-nestedstructure/README.md ea878f6a2 


Diff: https://reviews.apache.org/r/74142/diff/3/

Changes: https://reviews.apache.org/r/74142/diff/2-3/


Testing
---


Thanks,

Barbara Eckman

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Oct. 25, 2022, 7:43 a.m., Madhan Neethiraj wrote:
> >

Did you really mean to remove writeToFile() and encodeSecrets() from the 
project?  I understand moving most of the HandleSecrets methods to getFromURL, 
but don't we want to give folks the tools to easily write to file secrets that 
are encoded the way we later decodethem?


> On Oct. 25, 2022, 7:43 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerRoleUserStoreRetriever.java
> > Lines 61 (patched)
> > 
> >
> > A comment here with details of hour RangerRoles contents are used to 
> > create RangerUserStore object - wth an example.

Done.


> On Oct. 25, 2022, 7:43 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerRoleUserStoreRetriever.java
> > Lines 64 (patched)
> > 
> >
> > Given roleName is initialized in init() method, consider moving 
> > compliing patter to this method - this will help avoid compiling on every 
> > call to retrieveUserStoreInfo().

done


> On Oct. 25, 2022, 7:43 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetBearerToken.java
> > Lines 62 (patched)
> > 
> >
> > Did you mean to check if response is null? Shouldn't this be "response 
> > == null"?

Done


> On Oct. 25, 2022, 7:43 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
> > Lines 85 (patched)
> > 
> >
> > Perhaps flattenedAttrMap.put() should be after the for loop at #81?

I think you realized that it was in the right place as is-- that's where it is 
in your patch, I think..


> On Oct. 25, 2022, 7:43 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/RangerExternalUserStoreRetriever.java
> > Lines 56 (patched)
> > 
> >
> > Since the user-store returned by a given instance of 
> > RangerExternalUserStoreRetriever always contains the same userAttrMap, it 
> > might be useful to instantiate RangerUserStore in init() method itself.

userStore is used in retrieveUserStoreInfo method as well as init, so it needs 
to be initiated outside.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224827
---


On Oct. 21, 2022, 9:09 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Oct. 21, 2022, 9:09 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
>  4e1d19556 
>   
> 

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/
---

(Updated Oct. 21, 2022, 9:09 p.m.)


Review request for ranger and madhan.


Changes
---

added a new diff.  I guess since the package name changed, or because of how I 
updated this repo from my working repo, the diff just shows deleted files and 
new files.  There is also a lot of refactoring between diff 1 and diff 2.  if 
this diff is not acceptable, let me know. (I might also need help on how to do 
better at updating this repo from my working repo.) I'm sorry if I'm causing 
undue inconvenience.


Bugs: Ranger-3855
https://issues.apache.org/jira/browse/Ranger-3855


Repository: ranger


Description
---

RangerExternalUserStoreRetriever class Ranger-3855

Ranger version 3.0.0 provides a means, via a context enricher, to add or 
retrieve attributes to the database of users for whom Ranger controls access. 
This permits syntax like "Dumbo" in $USER.aliases any Ranger policy condition, 
including row and tag filters.   This greatly enhances the ability to provide 
custom Attribute-based Access Control based on the specific business needs of 
one's organization.

I believe that the original assumption was that such attributes would be added 
to AD/LDAP and enter Ranger via regular user sync's. However, this process does 
not currently work with Azure AD, which many organizations use. Neither does it 
provide timely support for organizations for whom adding each new attribute to 
AD would be subject to prolonged scrutiny by overworked security teams.  

In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
written a RangerExternalUserStoreRetriever class which adds arbitrary 
attributes to Ranger users via external API calls, thus freeing additions to 
the UserStore from dependency on AD/LDAP.   We have also written a 
RangerRoleUserStoreRetriever class, which transforms role membership into user 
attributes, for ease of use in complex policy conditions.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
 4e1d19556 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromDataFile.java
 60c7f22f7 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromURL.java
 1b9335339 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
 c5e13dbba 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/LICENSE
  
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/NOTICE
  
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/README.md
 eaf9ae823 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerExternalUserStoreRetriever.java
 c7ab74bc7 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerRoleUserStoreRetriever.java
 9eb50faa3 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/TokenInputs.java
 b9e1f0185 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/pom.xml
 d2914dbc0 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetBearerToken.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/HandleSecrets.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/RangerExternalUserStoreRetriever.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74142/diff/2/

Changes: https://reviews.apache.org/r/74142/diff/1-2/


Testing
---


Thanks,

Barbara Eckman

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224760
---


On Sept. 26, 2022, 7:17 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Sept. 26, 2022, 7:17 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromDataFile.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromURL.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/LICENSE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/NOTICE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/README.md
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerExternalUserStoreRetriever.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerRoleUserStoreRetriever.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/TokenInputs.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/pom.xml
>  PRE-CREATION 
>   plugin-nestedstructure/README.md ea878f6a2 
> 
> 
> Diff: https://reviews.apache.org/r/74142/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224759
---


On Sept. 26, 2022, 7:17 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Sept. 26, 2022, 7:17 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromDataFile.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromURL.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/LICENSE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/NOTICE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/README.md
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerExternalUserStoreRetriever.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerRoleUserStoreRetriever.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/TokenInputs.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/pom.xml
>  PRE-CREATION 
>   plugin-nestedstructure/README.md ea878f6a2 
> 
> 
> Diff: https://reviews.apache.org/r/74142/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Oct. 7, 2022, 1:21 a.m., Madhan Neethiraj wrote:
> >

Thanks a lot, Madhan, for your comments!


> On Oct. 7, 2022, 1:21 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 48 (patched)
> > 
> >
> > Ranger libraries use HttpClient library for HTTP calls - 
> > https://hc.apache.org/httpcomponents-client-4.5.x/index.html#. To avoid 
> > additional dependency of unirest libraries, I suggest to use HttpClient 
> > library.

done.


> On Oct. 7, 2022, 1:21 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/pom.xml
> > Lines 18 (patched)
> > 
> >
> > Is this pom.xml file necessary? Please review and remove if unused.

it was really only needed for unirest, which I have removed in favor of apache 
httpclient.  So the pom.xml file has been removed.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224759
---


On Sept. 26, 2022, 7:17 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Sept. 26, 2022, 7:17 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromDataFile.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromURL.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/LICENSE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/NOTICE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/README.md
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerExternalUserStoreRetriever.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerRoleUserStoreRetriever.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/TokenInputs.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/pom.xml
>  PRE-CREATION 
>   plugin-nestedstructure/README.md ea878f6a2 
> 
> 
> Diff: https://reviews.apache.org/r/74142/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> >

Thanks a lot, Bosco, for your meticulous comments!  I did a lot of refactoring, 
to replace unirest with apache httpclient, and to bubble exceptions upward.  I 
welcome your feedback as always!


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 34 (patched)
> > 
> >
> > Could we document what this configFile should contain? If it is user 
> > entered, then can we validate that it has all the fields we are expecting?

Done: added description of required elements to the README file, and added 
validation of the required elements to the HandleSecrets class.


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 44 (patched)
> > 
> >
> > Does this have sensitive information? If so, we should print in debug 
> > logs
> 
> Barbara Eckman wrote:
> Is that a question? "Should we print in debug logs?"  I considered this 
> along with a Comcast security colleague and decided that putting this warning 
> in the README file was sufficient: "NOTE that if this code is run with debug 
> logging enabled, there will be a very high likelihood that sensitive content 
> will be emitted in the log.". If you don't think it's sufficient, I can 
> remove printing it in the debug logs, but it seems as though it would be nice 
> to see what's in there, if an error should occur that involves the file 
> contents.

I have removed the printing to debug log


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 50 (patched)
> > 
> >
> > Should strToken = null? So that the caller would know if the request 
> > failed?
> 
> Barbara Eckman wrote:
> I'm not against this, but I already raise an error if the request for 
> token fails, and execution should halt after that, shouldn't it? Or am I 
> missing your point?

Sorry, my last comment made no sense.  I changed strToken's initialization to 
null.


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 58 (patched)
> > 
> >
> > Should we check if the post was successfull? E.g. check for HTTP return 
> > code?

Done.


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 63 (patched)
> > 
> >
> > Instead of doing e.toString(), can we just pass "e" to the logger? So 
> > the stack trace will be printed?
> 
> Barbara Eckman wrote:
> done

I refactored to throw exceptions upward.  Should I print to logger.error before 
throwing the exception?


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 67 (patched)
> > 
> >
> > Would the response be null if the Unirest.post() throws an Exception? 
> > We could probably move this code within the try block above

Done


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromDataFile.java
> > Lines 35 (patched)
> > 
> >
> > Any reason this is class member attribute rather than defining it 
> > within the method getFromDataFile(). If i t is okay to have it class member 
> > attribute, then should we worry about multi-thread safety scenarios?

Done


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromDataFile.java
> > Lines 48 (patched)
> > 
> >
> > We should probably ident this properly. It seems as if though it is 
> > closing the try block.

Done


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromURL.java
> > Lines 39 (patched)
> > 

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 44 (patched)
> > 
> >
> > Does this have sensitive information? If so, we should print in debug 
> > logs

Is that a question? "Should we print in debug logs?"  I considered this along 
with a Comcast security colleague and decided that putting this warning in the 
README file was sufficient: "NOTE that if this code is run with debug logging 
enabled, there will be a very high likelihood that sensitive content will be 
emitted in the log.". If you don't think it's sufficient, I can remove printing 
it in the debug logs, but it seems as though it would be nice to see what's in 
there, if an error should occur that involves the file contents.


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 50 (patched)
> > 
> >
> > Should strToken = null? So that the caller would know if the request 
> > failed?

I'm not against this, but I already raise an error if the request for token 
fails, and execution should halt after that, shouldn't it? Or am I missing your 
point?


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 63 (patched)
> > 
> >
> > Instead of doing e.toString(), can we just pass "e" to the logger? So 
> > the stack trace will be printed?

done


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 64 (patched)
> > 
> >
> > Since we are printing using logger, do we need to print to stderr also?

done, removed print to stderr


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromDataFile.java
> > Lines 60 (patched)
> > 
> >
> > Can we pass the exception as ",e", so that we can print the stack trace?

done


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromURL.java
> > Lines 50 (patched)
> > 
> >
> > Do we need to print in stderr

done


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
> > Lines 41 (patched)
> > 
> >
> > Is it secure to print sensitive information?

see response to getBearerToken, line 44


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
> > Lines 50 (patched)
> > 
> >
> > Is it secure to print sensitive information?

see response to getBearerToken, line 44


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
> > Lines 66 (patched)
> > 
> >
> > Should pass the exception as parameter? So we can get the stack trace?

done


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
> > Lines 67 (patched)
> > 
> >
> > Do we need to print this in stderr?

done


> On Oct. 7, 2022, 1:30 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerExternalUserStoreRetriever.java
> > Lines 69 (patched)
> > 
> >
> > This seems to be internal comcast class. What happens if this is not 
> > available in the opensource?

Sorry, this was an oversight. It's fixed now.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Oct. 7, 2022, 1:21 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 19 (patched)
> > 
> >
> > By convention, package names are in all lowercase letters. Consider 
> > renaming this package to "externalretrievers".

done


> On Oct. 7, 2022, 1:21 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 41 (patched)
> > 
> >
> > How about using gson object instantiated at #32, instead of 
> > instantiating it for every call to getBearerToken()?

done


> On Oct. 7, 2022, 1:21 a.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
> > Lines 34 (patched)
> > 
> >
> > static members encodedSecrets and decodedSecrets are unused. Please 
> > review and remove.

done


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224759
---


On Sept. 26, 2022, 7:17 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Sept. 26, 2022, 7:17 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromDataFile.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromURL.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/LICENSE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/NOTICE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/README.md
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerExternalUserStoreRetriever.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerRoleUserStoreRetriever.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/TokenInputs.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/pom.xml
>  PRE-CREATION 
>   plugin-nestedstructure/README.md ea878f6a2 
> 
> 
> Diff: https://reviews.apache.org/r/74142/diff/1/
> 
> 
> Testing
> 

Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/
---

Review request for ranger and madhan.


Bugs: Ranger-3855
https://issues.apache.org/jira/browse/Ranger-3855


Repository: ranger


Description
---

RangerExternalUserStoreRetriever class Ranger-3855

Ranger version 3.0.0 provides a means, via a context enricher, to add or 
retrieve attributes to the database of users for whom Ranger controls access. 
This permits syntax like "Dumbo" in $USER.aliases any Ranger policy condition, 
including row and tag filters.   This greatly enhances the ability to provide 
custom Attribute-based Access Control based on the specific business needs of 
one's organization.

I believe that the original assumption was that such attributes would be added 
to AD/LDAP and enter Ranger via regular user sync's. However, this process does 
not currently work with Azure AD, which many organizations use. Neither does it 
provide timely support for organizations for whom adding each new attribute to 
AD would be subject to prolonged scrutiny by overworked security teams.  

In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
written a RangerExternalUserStoreRetriever class which adds arbitrary 
attributes to Ranger users via external API calls, thus freeing additions to 
the UserStore from dependency on AD/LDAP.   We have also written a 
RangerRoleUserStoreRetriever class, which transforms role membership into user 
attributes, for ease of use in complex policy conditions.


Diffs
-

  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromDataFile.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromURL.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/LICENSE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/NOTICE
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/README.md
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerExternalUserStoreRetriever.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerRoleUserStoreRetriever.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/TokenInputs.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/pom.xml
 PRE-CREATION 
  plugin-nestedstructure/README.md ea878f6a2 


Diff: https://reviews.apache.org/r/74142/diff/1/


Testing
---


Thanks,

Barbara Eckman

Re: Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74057/
---

(Updated Aug. 12, 2022, 5:58 p.m.)


Review request for ranger and Madhan Neethiraj.


Repository: ranger


Description
---

It would be nice to be able to do fine-grained access control (FGA) over nested 
structures, e.g., the JSON responses of API calls.  This requires the 
individual attributes in a JSON object to be first-class metadata objects which 
can be tagged and on which policies can be written.  We have built a plugin and 
the corresponding Apache Atlas metadata structures and tagsync-mapper to 
support TBAC/RBAC/ABAC FGA over JSON structures.   Our instigating use case was 
FGA over the JSON responses of API calls, but this plugin has potential value 
anywhere FGA over the individual attributes of nested structures is needed, eg 
JSON messages read from Kafka topics.


Diffs (updated)
-

  plugin-nestedstructure/NOTICE 8772a4fba 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/AccessResult.java
 bf4a32501 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/DataMasker.java
 f630799b0 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/FieldLevelAccess.java
 0d27d36ec 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/JsonManipulator.java
 57030f277 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskTypes.java
 6464cfc06 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskingException.java
 316973db7 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAccessType.java
 92f099051 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAuthorizer.java
 bd2f509e7 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureResource.java
 ff4706519 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureService.java
 15b6f21c9 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
 77767767c 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/ExampleClient.java
 af32aff99 
  
tagsync/src/main/java/org/apache/ranger/tagsync/nestedstructureplugin/AtlasNestedStructureResourceMapper.java
 71dd70b7f 
  
tagsync/src/test/java/org/apache/ranger/tagsync/nestedstructureplugin/ResourceTests.java
 05ca49e82 


Diff: https://reviews.apache.org/r/74057/diff/5/

Changes: https://reviews.apache.org/r/74057/diff/4-5/


Testing
---


File Attachments (updated)


0001-patch-with-edits-based-on-OS-review.patch
  
https://reviews.apache.org/media/uploaded/files/2022/07/26/9d048baa-1968-4201-b213-1b807ed02587__0001-patch-with-edits-based-on-OS-review.patch
original patch
  
https://reviews.apache.org/media/uploaded/files/2022/07/27/5d6d9df2-b661-4d03-bba3-3f08cb26c32e__3809-plugin-nestedstructure-RANGER-3828-with-tagsync-Atla.patch
RANGER-3828-3.patch
  
https://reviews.apache.org/media/uploaded/files/2022/07/28/6367376f-0582-450c-a1e5-e61d982b5ff2__RANGER-3828-3.patch
RANGER-3828-4.patch
  
https://reviews.apache.org/media/uploaded/files/2022/08/11/f4f107af-b79e-4c9e-a406-24881e4107ca__RANGER-3828-4.patch
0001-updated-license-header.patch
  
https://reviews.apache.org/media/uploaded/files/2022/08/12/e9d41650-3562-40cd-829f-589dc3083d5f__0001-updated-license-header.patch


Thanks,

Barbara Eckman

Re: Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

> On Aug. 11, 2022, 6:49 p.m., Madhan Neethiraj wrote:
> > Ship It!

thank you so much for your help with this project, Madhan!


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74057/#review224615
---


On Aug. 11, 2022, 6:06 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74057/
> ---
> 
> (Updated Aug. 11, 2022, 6:06 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> It would be nice to be able to do fine-grained access control (FGA) over 
> nested structures, e.g., the JSON responses of API calls.  This requires the 
> individual attributes in a JSON object to be first-class metadata objects 
> which can be tagged and on which policies can be written.  We have built a 
> plugin and the corresponding Apache Atlas metadata structures and 
> tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures.   Our 
> instigating use case was FGA over the JSON responses of API calls, but this 
> plugin has potential value anywhere FGA over the individual attributes of 
> nested structures is needed, eg JSON messages read from Kafka topics.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  db629c85d 
>   
> agents-common/src/main/resources/service-defs/ranger-servicedef-nestedstructure.json
>  PRE-CREATION 
>   dev-support/ranger-pmd-ruleset.xml 88d77f236 
>   plugin-nestedstructure/.gitignore PRE-CREATION 
>   plugin-nestedstructure/LICENSE PRE-CREATION 
>   plugin-nestedstructure/NOTICE PRE-CREATION 
>   plugin-nestedstructure/README.md PRE-CREATION 
>   plugin-nestedstructure/conf/log4j.properties PRE-CREATION 
>   plugin-nestedstructure/conf/ranger-nestedstructure-audit.xml PRE-CREATION 
>   plugin-nestedstructure/conf/ranger-nestedstructure-policymgr-ssl.xml 
> PRE-CREATION 
>   plugin-nestedstructure/conf/ranger-nestedstructure-security.xml 
> PRE-CREATION 
>   plugin-nestedstructure/pom.xml PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/AccessResult.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/DataMasker.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/FieldLevelAccess.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/JsonManipulator.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskTypes.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskingException.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAccessType.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAuditHandler.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAuthorizer.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureResource.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureService.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/ExampleClient.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestDataMasker.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestJsonManipulator.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestNestedStructureAuthorizer.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
>  PRE-CREATION 
>   plugin-nestedstructure/src/test/resources/servicedef-nestedstructure.json 
> PRE-CREATION 
>   plugin-nestedstructure/src/test/resources/test_customer_records.json 
> 

Re: Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

> On Aug. 11, 2022, 8:47 p.m., Abhishek  Kumar wrote:
> >

thank you for catching these header slip-ups, which I thought I had already 
fixed.


> On Aug. 11, 2022, 8:47 p.m., Abhishek  Kumar wrote:
> > plugin-nestedstructure/NOTICE
> > Lines 3 (patched)
> > 
> >
> > Please keep the header consistent with the Apache License header.

removed Comcast copyright


> On Aug. 11, 2022, 8:47 p.m., Abhishek  Kumar wrote:
> > plugin-nestedstructure/NOTICE
> > Lines 18 (patched)
> > 
> >
> > Please keep the header consistent with the Apache License header.

I am under the impression that the NOTICE file permits such attributions.


> On Aug. 11, 2022, 8:47 p.m., Abhishek  Kumar wrote:
> > plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/AccessResult.java
> > Lines 2 (patched)
> > 
> >
> > Please keep the header consistent with the Apache License header.

done


> On Aug. 11, 2022, 8:47 p.m., Abhishek  Kumar wrote:
> > plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/DataMasker.java
> > Lines 2 (patched)
> > 
> >
> > Please keep the header consistent with the Apache License header.

done


> On Aug. 11, 2022, 8:47 p.m., Abhishek  Kumar wrote:
> > plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/FieldLevelAccess.java
> > Lines 2 (patched)
> > 
> >
> > Please keep the header consistent with the Apache License header.

done


> On Aug. 11, 2022, 8:47 p.m., Abhishek  Kumar wrote:
> > plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/JsonManipulator.java
> > Lines 2 (patched)
> > 
> >
> > Please keep the header consistent with the Apache License header.

done


> On Aug. 11, 2022, 8:47 p.m., Abhishek  Kumar wrote:
> > plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskTypes.java
> > Lines 2 (patched)
> > 
> >
> > Please keep the header consistent with the Apache License header.

done


> On Aug. 11, 2022, 8:47 p.m., Abhishek  Kumar wrote:
> > plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskingException.java
> > Lines 2 (patched)
> > 
> >
> > Please keep the header consistent with the Apache License header.

done


> On Aug. 11, 2022, 8:47 p.m., Abhishek  Kumar wrote:
> > plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAccessType.java
> > Lines 2 (patched)
> > 
> >
> > Please keep the header consistent with the Apache License header.

done


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74057/#review224618
---


On Aug. 11, 2022, 6:06 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74057/
> ---
> 
> (Updated Aug. 11, 2022, 6:06 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> It would be nice to be able to do fine-grained access control (FGA) over 
> nested structures, e.g., the JSON responses of API calls.  This requires the 
> individual attributes in a JSON object to be first-class metadata objects 
> which can be tagged and on which policies can be written.  We have built a 
> plugin and the corresponding Apache Atlas metadata structures and 
> tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures.   Our 
> instigating use case was FGA over the JSON responses of API calls, but this 
> plugin has potential value anywhere FGA over the individual attributes of 
> nested structures is needed, eg JSON messages read from Kafka topics.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  db629c85d 
>   
> agents-common/src/main/resources/service-defs/ranger-servicedef-nestedstructure.json
>  PRE-CREATION 
>   dev-support/ranger-pmd-ruleset.xml 88d77f236 
>   

Re: Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

> On Aug. 11, 2022, 9:23 p.m., Abhishek  Kumar wrote:
> > tagsync/src/test/java/org/apache/ranger/tagsync/nestedstructureplugin/ResourceTests.java
> > Lines 45 (patched)
> > 
> >
> > Please avoid System.out.println across test methods, log4j may be used 
> > for debug logs.

changed to log4j debug


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74057/#review224619
---


On Aug. 11, 2022, 6:06 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74057/
> ---
> 
> (Updated Aug. 11, 2022, 6:06 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> It would be nice to be able to do fine-grained access control (FGA) over 
> nested structures, e.g., the JSON responses of API calls.  This requires the 
> individual attributes in a JSON object to be first-class metadata objects 
> which can be tagged and on which policies can be written.  We have built a 
> plugin and the corresponding Apache Atlas metadata structures and 
> tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures.   Our 
> instigating use case was FGA over the JSON responses of API calls, but this 
> plugin has potential value anywhere FGA over the individual attributes of 
> nested structures is needed, eg JSON messages read from Kafka topics.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  db629c85d 
>   
> agents-common/src/main/resources/service-defs/ranger-servicedef-nestedstructure.json
>  PRE-CREATION 
>   dev-support/ranger-pmd-ruleset.xml 88d77f236 
>   plugin-nestedstructure/.gitignore PRE-CREATION 
>   plugin-nestedstructure/LICENSE PRE-CREATION 
>   plugin-nestedstructure/NOTICE PRE-CREATION 
>   plugin-nestedstructure/README.md PRE-CREATION 
>   plugin-nestedstructure/conf/log4j.properties PRE-CREATION 
>   plugin-nestedstructure/conf/ranger-nestedstructure-audit.xml PRE-CREATION 
>   plugin-nestedstructure/conf/ranger-nestedstructure-policymgr-ssl.xml 
> PRE-CREATION 
>   plugin-nestedstructure/conf/ranger-nestedstructure-security.xml 
> PRE-CREATION 
>   plugin-nestedstructure/pom.xml PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/AccessResult.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/DataMasker.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/FieldLevelAccess.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/JsonManipulator.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskTypes.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskingException.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAccessType.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAuditHandler.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAuthorizer.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureResource.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureService.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/ExampleClient.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestDataMasker.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestJsonManipulator.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestNestedStructureAuthorizer.java
>  PRE-CREATION 
>   
> 

Re: Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74057/
---

(Updated Aug. 11, 2022, 6:06 p.m.)


Review request for ranger and Madhan Neethiraj.


Repository: ranger


Description
---

It would be nice to be able to do fine-grained access control (FGA) over nested 
structures, e.g., the JSON responses of API calls.  This requires the 
individual attributes in a JSON object to be first-class metadata objects which 
can be tagged and on which policies can be written.  We have built a plugin and 
the corresponding Apache Atlas metadata structures and tagsync-mapper to 
support TBAC/RBAC/ABAC FGA over JSON structures.   Our instigating use case was 
FGA over the JSON responses of API calls, but this plugin has potential value 
anywhere FGA over the individual attributes of nested structures is needed, eg 
JSON messages read from Kafka topics.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 db629c85d 
  
agents-common/src/main/resources/service-defs/ranger-servicedef-nestedstructure.json
 PRE-CREATION 
  dev-support/ranger-pmd-ruleset.xml 88d77f236 
  plugin-nestedstructure/.gitignore PRE-CREATION 
  plugin-nestedstructure/LICENSE PRE-CREATION 
  plugin-nestedstructure/NOTICE PRE-CREATION 
  plugin-nestedstructure/README.md PRE-CREATION 
  plugin-nestedstructure/conf/log4j.properties PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-audit.xml PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-policymgr-ssl.xml 
PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-security.xml PRE-CREATION 
  plugin-nestedstructure/pom.xml PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/AccessResult.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/DataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/FieldLevelAccess.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/JsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskTypes.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskingException.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAccessType.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAuditHandler.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAuthorizer.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureResource.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureService.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/ExampleClient.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestDataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestJsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestNestedStructureAuthorizer.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
 PRE-CREATION 
  plugin-nestedstructure/src/test/resources/servicedef-nestedstructure.json 
PRE-CREATION 
  plugin-nestedstructure/src/test/resources/test_customer_records.json 
PRE-CREATION 
  pom.xml 0945f4b1d 
  
tagsync/src/main/java/org/apache/ranger/tagsync/nestedstructureplugin/AtlasNestedStructureResourceMapper.java
 PRE-CREATION 
  
tagsync/src/test/java/org/apache/ranger/tagsync/nestedstructureplugin/ResourceTests.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74057/diff/4/

Changes: https://reviews.apache.org/r/74057/diff/3-4/


Testing
---


File Attachments


0001-patch-with-edits-based-on-OS-review.patch
  
https://reviews.apache.org/media/uploaded/files/2022/07/26/9d048baa-1968-4201-b213-1b807ed02587__0001-patch-with-edits-based-on-OS-review.patch
original 

Re: Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74057/
---

(Updated Aug. 11, 2022, 5:44 p.m.)


Review request for ranger and Madhan Neethiraj.


Changes
---

added 3828-4.patch


Repository: ranger


Description
---

It would be nice to be able to do fine-grained access control (FGA) over nested 
structures, e.g., the JSON responses of API calls.  This requires the 
individual attributes in a JSON object to be first-class metadata objects which 
can be tagged and on which policies can be written.  We have built a plugin and 
the corresponding Apache Atlas metadata structures and tagsync-mapper to 
support TBAC/RBAC/ABAC FGA over JSON structures.   Our instigating use case was 
FGA over the JSON responses of API calls, but this plugin has potential value 
anywhere FGA over the individual attributes of nested structures is needed, eg 
JSON messages read from Kafka topics.


Diffs
-

  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 db629c85d 
  
agents-common/src/main/resources/service-defs/ranger-servicedef-nestedstructure.json
 PRE-CREATION 
  dev-support/ranger-pmd-ruleset.xml 88d77f236 
  plugin-nestedstructure/.gitignore PRE-CREATION 
  plugin-nestedstructure/LICENSE PRE-CREATION 
  plugin-nestedstructure/NOTICE PRE-CREATION 
  plugin-nestedstructure/README.md PRE-CREATION 
  plugin-nestedstructure/conf/log4j.properties PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-audit.xml PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-policymgr-ssl.xml 
PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-security.xml PRE-CREATION 
  plugin-nestedstructure/pom.xml PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/AccessResult.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/DataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/FieldLevelAccess.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/JsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskTypes.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskingException.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAccessType.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAuthorizer.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureResource.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureService.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/ExampleClient.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestDataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestJsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestNestedStructureAuthorizer.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
 PRE-CREATION 
  plugin-nestedstructure/src/test/resources/servicedef-nestedstructure.json 
PRE-CREATION 
  plugin-nestedstructure/src/test/resources/test_customer_records.json 
PRE-CREATION 
  pom.xml 0945f4b1d 
  
tagsync/src/main/java/org/apache/ranger/tagsync/nestedstructureplugin/AtlasNestedStructureResourceMapper.java
 PRE-CREATION 
  
tagsync/src/test/java/org/apache/ranger/tagsync/nestedstructureplugin/ResourceTests.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74057/diff/3/


Testing
---


File Attachments (updated)


0001-patch-with-edits-based-on-OS-review.patch
  
https://reviews.apache.org/media/uploaded/files/2022/07/26/9d048baa-1968-4201-b213-1b807ed02587__0001-patch-with-edits-based-on-OS-review.patch
original patch
  

Re: Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74057/
---

(Updated July 28, 2022, 2 p.m.)


Review request for ranger and Madhan Neethiraj.


Changes
---

added Madhan's patch


Repository: ranger


Description
---

It would be nice to be able to do fine-grained access control (FGA) over nested 
structures, e.g., the JSON responses of API calls.  This requires the 
individual attributes in a JSON object to be first-class metadata objects which 
can be tagged and on which policies can be written.  We have built a plugin and 
the corresponding Apache Atlas metadata structures and tagsync-mapper to 
support TBAC/RBAC/ABAC FGA over JSON structures.   Our instigating use case was 
FGA over the JSON responses of API calls, but this plugin has potential value 
anywhere FGA over the individual attributes of nested structures is needed, eg 
JSON messages read from Kafka topics.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 db629c85d 
  
agents-common/src/main/resources/service-defs/ranger-servicedef-nestedstructure.json
 PRE-CREATION 
  dev-support/ranger-pmd-ruleset.xml 88d77f236 
  plugin-nestedstructure/.gitignore PRE-CREATION 
  plugin-nestedstructure/LICENSE PRE-CREATION 
  plugin-nestedstructure/NOTICE PRE-CREATION 
  plugin-nestedstructure/README.md PRE-CREATION 
  plugin-nestedstructure/conf/log4j.properties PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-audit.xml PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-policymgr-ssl.xml 
PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-security.xml PRE-CREATION 
  plugin-nestedstructure/pom.xml PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/AccessResult.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/DataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/FieldLevelAccess.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/JsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskTypes.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/MaskingException.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAccessType.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureAuthorizer.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureResource.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/NestedStructureService.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/ExampleClient.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestDataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestJsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestNestedStructureAuthorizer.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
 PRE-CREATION 
  plugin-nestedstructure/src/test/resources/servicedef-nestedstructure.json 
PRE-CREATION 
  plugin-nestedstructure/src/test/resources/test_customer_records.json 
PRE-CREATION 
  pom.xml 0945f4b1d 
  
tagsync/src/main/java/org/apache/ranger/tagsync/nestedstructureplugin/AtlasNestedStructureResourceMapper.java
 PRE-CREATION 
  
tagsync/src/test/java/org/apache/ranger/tagsync/nestedstructureplugin/ResourceTests.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74057/diff/3/

Changes: https://reviews.apache.org/r/74057/diff/2-3/


Testing
---


File Attachments (updated)


0001-patch-with-edits-based-on-OS-review.patch
  
https://reviews.apache.org/media/uploaded/files/2022/07/26/9d048baa-1968-4201-b213-1b807ed02587__0001-patch-with-edits-based-on-OS-review.patch
original patch
  

Re: Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74057/
---

(Updated July 27, 2022, 3:09 p.m.)


Review request for ranger and Madhan Neethiraj.


Repository: ranger


Description
---

It would be nice to be able to do fine-grained access control (FGA) over nested 
structures, e.g., the JSON responses of API calls.  This requires the 
individual attributes in a JSON object to be first-class metadata objects which 
can be tagged and on which policies can be written.  We have built a plugin and 
the corresponding Apache Atlas metadata structures and tagsync-mapper to 
support TBAC/RBAC/ABAC FGA over JSON structures.   Our instigating use case was 
FGA over the JSON responses of API calls, but this plugin has potential value 
anywhere FGA over the individual attributes of nested structures is needed, eg 
JSON messages read from Kafka topics.


Diffs
-

  plugin-nestedstructure/CONTRIBUTING PRE-CREATION 
  plugin-nestedstructure/LICENSE PRE-CREATION 
  plugin-nestedstructure/NOTICE PRE-CREATION 
  plugin-nestedstructure/README.md PRE-CREATION 
  plugin-nestedstructure/conf/log4j.properties PRE-CREATION 
  plugin-nestedstructure/conf/nestedstructure_servicedef.json PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-audit.xml PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-policymgr-ssl.xml 
PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-security.xml PRE-CREATION 
  plugin-nestedstructure/pom.xml PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/AccessResult.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/DataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/ExampleClient.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/FieldLevelAccess.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/JsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/MaskTypes.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/MaskingException.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructureAccessType.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructureAuthorizer.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructure_Resource.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructure_Service.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/RecordFilterJavaScript.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestDataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestJsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
 PRE-CREATION 
  pom.xml 0945f4b1d 
  
tagsync/src/main/java/org/apache/ranger/tagsync/nestedstructureplugin/AtlasNestedStructureResourceMapper.java
 PRE-CREATION 
  
tagsync/src/test/java/org/apache/ranger/tagsync/nestedstructureplugin/ResourceTests.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74057/diff/2/


Testing
---


File Attachments (updated)


0001-patch-with-edits-based-on-OS-review.patch
  
https://reviews.apache.org/media/uploaded/files/2022/07/26/9d048baa-1968-4201-b213-1b807ed02587__0001-patch-with-edits-based-on-OS-review.patch
original patch
  
https://reviews.apache.org/media/uploaded/files/2022/07/27/5d6d9df2-b661-4d03-bba3-3f08cb26c32e__3809-plugin-nestedstructure-RANGER-3828-with-tagsync-Atla.patch


Thanks,

Barbara Eckman

Re: Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74057/
---

(Updated July 26, 2022, 7:13 p.m.)


Review request for ranger and Madhan Neethiraj.


Repository: ranger


Description
---

It would be nice to be able to do fine-grained access control (FGA) over nested 
structures, e.g., the JSON responses of API calls.  This requires the 
individual attributes in a JSON object to be first-class metadata objects which 
can be tagged and on which policies can be written.  We have built a plugin and 
the corresponding Apache Atlas metadata structures and tagsync-mapper to 
support TBAC/RBAC/ABAC FGA over JSON structures.   Our instigating use case was 
FGA over the JSON responses of API calls, but this plugin has potential value 
anywhere FGA over the individual attributes of nested structures is needed, eg 
JSON messages read from Kafka topics.


Diffs (updated)
-

  plugin-nestedstructure/CONTRIBUTING PRE-CREATION 
  plugin-nestedstructure/LICENSE PRE-CREATION 
  plugin-nestedstructure/NOTICE PRE-CREATION 
  plugin-nestedstructure/README.md PRE-CREATION 
  plugin-nestedstructure/conf/log4j.properties PRE-CREATION 
  plugin-nestedstructure/conf/nestedstructure_servicedef.json PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-audit.xml PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-policymgr-ssl.xml 
PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-security.xml PRE-CREATION 
  plugin-nestedstructure/pom.xml PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/AccessResult.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/DataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/ExampleClient.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/FieldLevelAccess.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/JsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/MaskTypes.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/MaskingException.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructureAccessType.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructureAuthorizer.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructure_Resource.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructure_Service.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/RecordFilterJavaScript.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestDataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestJsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
 PRE-CREATION 
  pom.xml 0945f4b1d 
  
tagsync/src/main/java/org/apache/ranger/tagsync/nestedstructureplugin/AtlasNestedStructureResourceMapper.java
 PRE-CREATION 
  
tagsync/src/test/java/org/apache/ranger/tagsync/nestedstructureplugin/ResourceTests.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74057/diff/2/

Changes: https://reviews.apache.org/r/74057/diff/1-2/


Testing
---


File Attachments (updated)


0001-patch-with-edits-based-on-OS-review.patch
  
https://reviews.apache.org/media/uploaded/files/2022/07/26/9d048baa-1968-4201-b213-1b807ed02587__0001-patch-with-edits-based-on-OS-review.patch


Thanks,

Barbara Eckman

Re: Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

> On July 18, 2022, 7:01 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructureAuthorizer.java
> > Lines 210 (patched)
> > 
> >
> > This call to isAccessAllowed() would return isAllowed=true only when 
> > access is allowed for the whole schema, The intent of this method seems to 
> > return true if user has the requested access to even only of field in the 
> > schema.
> > 
> > To address this, resource-match scope should be set as shown below:
> >   
> > request.setResourceMatchingScope(RangerAccessRequest.ResourceMatchingScope.SELF_OR_DESCENDANTS);

done


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74057/#review224567
---


On July 13, 2022, 11:03 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74057/
> ---
> 
> (Updated July 13, 2022, 11:03 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> It would be nice to be able to do fine-grained access control (FGA) over 
> nested structures, e.g., the JSON responses of API calls.  This requires the 
> individual attributes in a JSON object to be first-class metadata objects 
> which can be tagged and on which policies can be written.  We have built a 
> plugin and the corresponding Apache Atlas metadata structures and 
> tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures.   Our 
> instigating use case was FGA over the JSON responses of API calls, but this 
> plugin has potential value anywhere FGA over the individual attributes of 
> nested structures is needed, eg JSON messages read from Kafka topics.
> 
> 
> Diffs
> -
> 
>   plugin-nestedstructure/CONTRIBUTING PRE-CREATION 
>   plugin-nestedstructure/LICENSE PRE-CREATION 
>   plugin-nestedstructure/NOTICE PRE-CREATION 
>   plugin-nestedstructure/README.md PRE-CREATION 
>   plugin-nestedstructure/conf/log4j.properties PRE-CREATION 
>   plugin-nestedstructure/conf/nestedstructure_servicedef.json PRE-CREATION 
>   plugin-nestedstructure/conf/ranger-nestedstructure-audit.xml PRE-CREATION 
>   plugin-nestedstructure/conf/ranger-nestedstructure-policymgr-ssl.xml 
> PRE-CREATION 
>   plugin-nestedstructure/conf/ranger-nestedstructure-security.xml 
> PRE-CREATION 
>   plugin-nestedstructure/pom.xml PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/AccessResult.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/DataMasker.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/ExampleClient.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/FieldLevelAccess.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/JsonManipulator.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/MaskTypes.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/MaskingException.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructureAccessType.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructureAuthorizer.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructure_Resource.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructure_Service.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/RecordFilterJavaScript.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestDataMasker.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestJsonManipulator.java
>  PRE-CREATION 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
>  PRE-CREATION 
>   pom.xml 0945f4b1d 
>   
> 

Re: Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > Barbara - thank you for your contribution to Apache Ranger. This plugin to 
> > authorize access to Json records/fields, including row-filter and 
> > field-masking will be useful widely in any application serving complex Json 
> > records.
> > 
> > Here are my initial comments on reviewing the patch. I will review the 
> > patch further and add further comments if necessary.

Madhan, thanks for your very careful review!


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/CONTRIBUTING
> > Lines 1 (patched)
> > 
> >
> > Please review if this is needed in Apache Ranger repo. If not, please 
> > remove.

removed


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/NOTICE
> > Lines 1 (patched)
> > 
> >
> > Ranger => Apache Ranger

done


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/conf/log4j.properties
> > Lines 1 (patched)
> > 
> >
> > License header is missing. Please add.

done


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/conf/nestedstructure_servicedef.json
> > Lines 1 (patched)
> > 
> >
> > Consider moving service-def to 
> > agents-common/src/main/resources/service-defs, alog with rest of 
> > service-defs. This will also enable Ranger to load this service-def at 
> > startup automatically in EmbeddedServiceDefsUtil.

done


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/pom.xml
> > Lines 2 (patched)
> > 
> >
> > License header is missing. Please add.

done


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/AccessResult.java
> > Lines 2 (patched)
> > 
> >
> > Please review Apache guidelines on source code headers at 
> > https://www.apache.org/legal/src-headers, specifically about Copyright in 
> > sources and update.

done on all .java files


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/AccessResult.java
> > Lines 26 (patched)
> > 
> >
> > Apache strongly recommends to avoid author tags in source code - see 
> > https://www.apache.org/foundation/records/minutes/2004/board_minutes_2004_09_22.txt
> >  (Recommend strongly that @author is avoided; but leave it to each PMC to 
> > make the final call with their respective communities).
> > 
> > To be consistent with rest of Apache Ranger source code, I suggest to 
> > remove @author tag from sources.

removed all @author tags


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/AccessResult.java
> > Lines 30 (patched)
> > 
> >
> > Instance members are assigned value only in the constructor. Consider 
> > marking these as final.

done


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/AccessResult.java
> > Lines 34 (patched)
> > 
> >
> > Default constructor is unused. Please review and remove.

done


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/DataMasker.java
> > Lines 53 (patched)
> > 
> >
> > Consider marking SUPPORTED_DATE_FORMATS as final, as it is is not 
> > reassigned after initialization.

done


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/DataMasker.java
> > Lines 120 (patched)
> > 
> >
> > Consider moving customMaskValue to 'case MaskTypes.CUSTOM', as this is 
> > unused in other cases.

done


> On July 16, 2022, 6:09 a.m., Madhan Neethiraj wrote:
> > plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/DataMasker.java
> > Lines 210 (patched)
> > 

Review Request 74057: Plugin for Fine-grained Access Control over nested structures

[Barbara Eckman via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74057/
---

Review request for ranger.


Repository: ranger


Description
---

It would be nice to be able to do fine-grained access control (FGA) over nested 
structures, e.g., the JSON responses of API calls.  This requires the 
individual attributes in a JSON object to be first-class metadata objects which 
can be tagged and on which policies can be written.  We have built a plugin and 
the corresponding Apache Atlas metadata structures and tagsync-mapper to 
support TBAC/RBAC/ABAC FGA over JSON structures.   Our instigating use case was 
FGA over the JSON responses of API calls, but this plugin has potential value 
anywhere FGA over the individual attributes of nested structures is needed, eg 
JSON messages read from Kafka topics.


Diffs
-

  plugin-nestedstructure/CONTRIBUTING PRE-CREATION 
  plugin-nestedstructure/LICENSE PRE-CREATION 
  plugin-nestedstructure/NOTICE PRE-CREATION 
  plugin-nestedstructure/README.md PRE-CREATION 
  plugin-nestedstructure/conf/log4j.properties PRE-CREATION 
  plugin-nestedstructure/conf/nestedstructure_servicedef.json PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-audit.xml PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-policymgr-ssl.xml 
PRE-CREATION 
  plugin-nestedstructure/conf/ranger-nestedstructure-security.xml PRE-CREATION 
  plugin-nestedstructure/pom.xml PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/AccessResult.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/DataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/ExampleClient.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/FieldLevelAccess.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/JsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/MaskTypes.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/MaskingException.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructureAccessType.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructureAuthorizer.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructure_Resource.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructure_Service.java
 PRE-CREATION 
  
plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/RecordFilterJavaScript.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestDataMasker.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestJsonManipulator.java
 PRE-CREATION 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
 PRE-CREATION 
  pom.xml 0945f4b1d 
  
tagsync/src/main/java/org/apache/ranger/tagsync/nestedstructureplugin/AtlasNestedStructureResourceMapper.java
 PRE-CREATION 
  
tagsync/src/test/java/org/apache/ranger/tagsync/nestedstructureplugin/ResourceTests.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74057/diff/1/


Testing
---


Thanks,

Barbara Eckman