[jira] [Commented] (RANGER-3960) RANGER - Upgrade spring-security version to 5.7.5

[Pradeep Agrawal (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630725#comment-17630725
 ] 

Pradeep Agrawal commented on RANGER-3960:
-

Master branch Commit link : 
[https://github.com/apache/ranger/commit/73f1a3b22848e43da0d1aad86ea59dd491c568ad]

2.4 branch commit link : 
[https://github.com/apache/ranger/commit/731be8363bc1db15b2a2a999c3d56e3d2eb27b8e]

 

> RANGER - Upgrade spring-security version to 5.7.5
> -
>
> Key: RANGER-3960
> URL: https://issues.apache.org/jira/browse/RANGER-3960
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Affects Versions: 3.0.0
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: 
> 0001-RANGER-3960-Upgrade-spring-security-version-to-5.7.5.patch
>
>
> Currently ranger is pulling spring-security version-5.7.2, upgrade it to 5.7.5



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Barbara Eckman via Review Board]

> On Oct. 15, 2022, 4:17 p.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
> > Lines 44 (patched)
> > 
> >
> > Yes, it was a question. I was not sure whether we are printing would 
> > have sensitive information. If it doesn't the suggestion is not to print 
> > them. The reason being, it is common for applications to be configured to 
> > DEBUG level during troubleshooting sessions and also in some cases, these 
> > logs are sent to external systems like DataDog (in the cloud) or other log 
> > aggregation tools and it would be difficult to enforce any policies in 
> > those tools.

I removed debug logging for sensitive info everywhere. It will make debugging 
harder but I agree it is necessary.


- Barbara


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224797
---


On Nov. 3, 2022, 5:36 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Nov. 3, 2022, 5:36 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/AllRangerUserStoreRetrievers.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromDataFile.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/GetFromURL.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/LICENSE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/NOTICE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalretrievers/README.md
>  PRE-CREATION 
>   dev-support/spotbugsIncludeFile.xml 3621e8c08 
>   plugin-nestedstructure/README.md ea878f6a2 
> 
> 
> Diff: https://reviews.apache.org/r/74142/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

[jira] [Resolved] (RANGER-3787) Non-daemon threads started by ElasticSearchAuditDestination cause Spark application hanging

[Madhan Neethiraj (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved RANGER-3787.
--
Fix Version/s: 3.0.0
   2.4.0
   Resolution: Fixed

[~zhouyifan279] - thank you for the fix. The patch is now merged in following 
branches:

master:
{noformat}
commit c3e2324ddc05e990721bcc70f02f456e344c2314 (origin/master, origin/HEAD)
Author: zhouyifan279 
Date:   Tue Jun 14 10:36:42 2022 +0800

RANGER-3787: non-daemon threads started by ElasticSearchAuditDestination 
cause Spark application hanging

Signed-off-by: Bhavik Patel 
{noformat}
 

ranger-2.4:
{noformat}
commit 01b5481b3a78344e2cedeb84f64fb7339de03595 (HEAD -> ranger-2.4, 
origin/ranger-2.4)
Author: zhouyifan279 
Date:   Tue Jun 14 10:36:42 2022 +0800

RANGER-3787: non-daemon threads started by ElasticSearchAuditDestination 
cause Spark application hanging

Signed-off-by: Bhavik Patel 
(cherry picked from commit c3e2324ddc05e990721bcc70f02f456e344c2314)
{noformat}

> Non-daemon threads started by ElasticSearchAuditDestination cause Spark 
> application hanging
> ---
>
> Key: RANGER-3787
> URL: https://issues.apache.org/jira/browse/RANGER-3787
> Project: Ranger
>  Issue Type: Improvement
>  Components: audit
>Affects Versions: 3.0.0
>Reporter: Zhou Yifan
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When using [kyuubi-spark-authz 
> plugin|https://github.com/apache/incubator-kyuubi/tree/master/extensions/spark/kyuubi-spark-authz]
>  (which extends RangerBasePlugin) to save audit log to ElasticSearch in Spark 
> application, I found that if Spark application was submitted in local or 
> client mode, it hanged forever even after main thread exited. But if I turn 
> off `xasecure.audit.destination.elasticsearch`, Spark application exits 
> normally.
>  
> Here is my `ranger-spark-audit.xml`:
> {code:java}
>     
>
>         xasecure.audit.destination.elasticsearch
>         enabled
>         
>         xasecure.audit.destination.elasticsearch.urls
>         es-master-1,es-master-2,es-master-3
>         
>         xasecure.audit.destination.elasticsearch.port
>         9200
>     
>  {code}
> In `jstack` output, I found 2 kinds of non-daemon threads started by 
> ElasticSearch RestHighLevelClient:
> {code:java}
> "I/O dispatcher 1" #64 prio=5 os_prio=31 tid=0x7f8717d89800 nid=0x14303 
> runnable [0x754bc000]
>    java.lang.Thread.State: RUNNABLE
>         at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
>         at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
>         at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
>         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
>         - locked <0x0007808a7b58> (a sun.nio.ch.Util$3)
>         - locked <0x0007808a7b48> (a 
> java.util.Collections$UnmodifiableSet)
>         - locked <0x00078088b318> (a sun.nio.ch.KQueueSelectorImpl)
>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
>         at 
> org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:255)
>         at 
> org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
>         at 
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
>         at java.lang.Thread.run(Thread.java:750)"pool-22-thread-1" #63 prio=5 
> os_prio=31 tid=0x7f86c77d1800 nid=0xb503 runnable [0x753b9000]
>    java.lang.Thread.State: RUNNABLE
>         at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
>         at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
>         at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
>         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
>         - locked <0x000780832dc0> (a sun.nio.ch.Util$3)
>         - locked <0x000780832db0> (a 
> java.util.Collections$UnmodifiableSet)
>         - locked <0x000780832b60> (a sun.nio.ch.KQueueSelectorImpl)
>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
>         at 
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:343)
>         at 
> org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
>         at 
> org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
>         at java.lang.Thread.run(Thread.java:750) {code}
>  
> h3. Reproduce steps
> Build kyuubi-spark-authz plugin:
> {code:java}
> git clone g...@github.com:zhouyif

Re: Review Request 74196: RANGER-3960: Upgrade spring-security version to 5.7.5

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74196/#review224857
---


Ship it!




Ship It!

- Madhan Neethiraj


On Nov. 8, 2022, 2:29 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74196/
> ---
> 
> (Updated Nov. 8, 2022, 2:29 p.m.)
> 
> 
> Review request for ranger, bhavik patel, Abhay Kulkarni, Madhan Neethiraj, 
> Nikhil P, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: CDPD-3960
> https://issues.apache.org/jira/browse/CDPD-3960
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Here I am proposing to Upgrade Spring Security version to 5.7.5 and Spring 
> Framework to 5.3.23
> 
> 
> Diffs
> -
> 
>   pom.xml fc2c2a585 
> 
> 
> Diff: https://reviews.apache.org/r/74196/diff/1/
> 
> 
> Testing
> ---
> 
> Tested ranger admin installation, password change, CRUD operation on Ranger 
> service, policy, users and group.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Review Request 74196: RANGER-3960: Upgrade spring-security version to 5.7.5

[Pradeep Agrawal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74196/
---

Review request for ranger, bhavik patel, Abhay Kulkarni, Madhan Neethiraj, 
Nikhil P, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
Polavarapu, and Velmurugan Periasamy.


Bugs: CDPD-3960
https://issues.apache.org/jira/browse/CDPD-3960


Repository: ranger


Description
---

Here I am proposing to Upgrade Spring Security version to 5.7.5 and Spring 
Framework to 5.3.23


Diffs
-

  pom.xml fc2c2a585 


Diff: https://reviews.apache.org/r/74196/diff/1/


Testing
---

Tested ranger admin installation, password change, CRUD operation on Ranger 
service, policy, users and group.


Thanks,

Pradeep Agrawal

[jira] [Updated] (RANGER-3960) RANGER - Upgrade spring-security version to 5.7.5

[Pradeep Agrawal (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pradeep Agrawal updated RANGER-3960:

Attachment: 0001-RANGER-3960-Upgrade-spring-security-version-to-5.7.5.patch

> RANGER - Upgrade spring-security version to 5.7.5
> -
>
> Key: RANGER-3960
> URL: https://issues.apache.org/jira/browse/RANGER-3960
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Affects Versions: 3.0.0
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: 
> 0001-RANGER-3960-Upgrade-spring-security-version-to-5.7.5.patch
>
>
> Currently ranger is pulling spring-security version-5.7.2, upgrade it to 5.7.5



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Review Request 74194: RANGER-3962: Add preload directive to HSTS header

[Mahesh Bandal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74194/
---

Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Kishor 
Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, 
Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.


Bugs: RANGER-3962
https://issues.apache.org/jira/browse/RANGER-3962


Repository: ranger


Description
---

"Preload" directive is absent in HSTS header. As security its recommended to 
have within HSTS header.


Diffs
-

  kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSMDCFilter.java 
1174f0bd6 
  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSecurityContextFormationFilter.java
 9f83daf9a 
  security-admin/src/main/webapp/login.jsp df234efd9 


Diff: https://reviews.apache.org/r/74194/diff/1/


Testing
---

1. Ranger maven build successful -> mvn clean compile verify test install
2. Ranger Setup & install successful
3. Performed sanity testing.


Thanks,

Mahesh Bandal

Review Request 74192: RANGER-3961: AuditFileSpool logs out all events that were not audited successfully

[Mahesh Bandal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74192/
---

Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Kishor 
Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, 
Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.


Bugs: RANGER-3961
https://issues.apache.org/jira/browse/RANGER-3961


Repository: ranger


Description
---

At AuditFileSpool.java#L904, the variable lines holds all the events that are 
in current batch, that is 1000 by default (~0.5MB of strings). The batch can be 
configured even higher that makes things worse.

If there is an issue in audit, a lot of huge strings will be logged out.

Suggesting to remove lines variable and/or add lines.size() to know how many 
events were not sent.


Diffs
-

  agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileSpool.java 
edce2461a 


Diff: https://reviews.apache.org/r/74192/diff/1/


Testing
---

1. Ranger maven build successful -> mvn clean compile verify test install
2. Ranger Setup & install successful
3. Performed sanity testing.


Thanks,

Mahesh Bandal