Review Request 74934: RANGER-4737 : The inactivityTimeout is getting reset when user updates its profile from UserProfile page

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74934/
---

Review request for ranger, Brijesh Bhalala and Dhaval Rajpara.


Bugs: RANGER-4737
https://issues.apache.org/jira/browse/RANGER-4737


Repository: ranger


Description
---

In Ranger Admin with React JS, the inactivityTimeout is getting reset to 
default value of 15 mins only when user updates its profile from UserProfile 
page.


Diffs
-

  
security-admin/src/main/webapp/react-webapp/src/views/UserGroupRoleListing/users_details/UserFormComp.jsx
 cb0544c8a77a877a6dd4c5f7670c6c11eaeed886 
  security-admin/src/main/webapp/react-webapp/src/views/UserProfile.jsx 
d5f3018e54703b3523769c22b4dd3535df0ad71c 


Diff: https://reviews.apache.org/r/74934/diff/1/


Testing
---

Tested changes on a cluster setup with Ranger Admin build with React JS code 
base.


Successful completion of build command :
mvn clean compile package


Thanks,

Mugdha Varadkar

[jira] [Updated] (RANGER-4737) The inactivityTimeout is getting reset when user updates its profile from UserProfile page

[Mugdha Varadkar (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mugdha Varadkar updated RANGER-4737:

Attachment: 0001-RANGER-4737.patch

> The inactivityTimeout is getting reset when user updates its profile from 
> UserProfile page
> --
>
> Key: RANGER-4737
> URL: https://issues.apache.org/jira/browse/RANGER-4737
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Mugdha Varadkar
>Assignee: Mugdha Varadkar
>Priority: Major
>  Labels: ranger-react
> Attachments: 0001-RANGER-4737.patch
>
>
> In Ranger Admin with React JS, the inactivityTimeout is getting reset to 
> default value of 15 mins only when user updates its profile from UserProfile 
> page.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (RANGER-4506) Illegal read lock usage in getMetadata/getKeyVersion

[Vikas Kumar (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vikas Kumar resolved RANGER-4506.
-
Resolution: Not A Problem

Resolving this issue as we found it as not an issue and proper explanation has 
been provided. Thanks.

> Illegal read lock usage in getMetadata/getKeyVersion
> 
>
> Key: RANGER-4506
> URL: https://issues.apache.org/jira/browse/RANGER-4506
> Project: Ranger
>  Issue Type: Bug
>  Components: kms
>Reporter: Jianchun Xu
>Assignee: Vikas Kumar
>Priority: Major
>
> RangerKeyStoreProvider illegally writes to key store under Read lock. This 
> happens in both getMetadata and getKeyVersion.
> E.g. in following getKeyVersion, under Read lock, the code calls 
> `dbStore.engineLoad(null, masterKey)` which reloads all the keys. Since 
> multiple threads can hold the Read lock, multiple threads can enter and 
> reload all the keys. Thus the 2nd `dbStore.engineContainsAlias(versionName)` 
> test and following `dbStore.engineGetDecryptedZoneKeyByte(versionName)` can 
> both get wrong result if another thread is reloading keys.
> [https://github.com/apache/ranger/blob/master/kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java#L331]
> {code:java}
> @Override
> public KeyVersion getKeyVersion(String versionName) throws IOException {
> if (logger.isDebugEnabled()) {
> logger.debug("==> getKeyVersion({})", versionName);
> }
> KeyVersion ret = null;
> try (AutoClosableReadLock ignored = new AutoClosableReadLock(lock)) {
> if (keyVaultEnabled) {
> try {
> boolean versionNameExists = 
> dbStore.engineContainsAlias(versionName);
> if (!versionNameExists) {
> dbStore.engineLoad(null, masterKey);
> versionNameExists = 
> dbStore.engineContainsAlias(versionName);
> }
> if (versionNameExists) {
> byte[] decryptKeyByte;
> try {
> decryptKeyByte = 
> dbStore.engineGetDecryptedZoneKeyByte(versionName);
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74897: RANGER-4719: Policy condition expressions are split by the Ranger UI on commas

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74897/#review226299
---




agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java
Lines 719 (patched)


@Himanshu - how does adding "engineName" to evaluator options help resolve 
this issue? Can you please add details? Thanks!


- Madhan Neethiraj


On March 6, 2024, 11:57 a.m., Himanshu Maurya wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74897/
> ---
> 
> (Updated March 6, 2024, 11:57 a.m.)
> 
> 
> Review request for ranger, bhavik patel, Dhaval Shah, Dineshkumar Yadav, 
> Harshal Chavan, Kishor Gollapalliwar, Madhan Neethiraj, Mehul Parikh, Nitin 
> Galave, Pradeep Agrawal, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4719
> https://issues.apache.org/jira/browse/RANGER-4719
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> While using attributes with default values introduced in RANGER-3997 like 
> GET_USER_ATTR('state', 'null') in policy condition.
> It is observed that ranger is splitting condition string in to parts 
> separated by comma of the parameters passed to GET_USER_ATTR(), due to this 
> it is not taking default value and conditions are converted to invalid 
> strings.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 
> ea76e6c33 
> 
> 
> Diff: https://reviews.apache.org/r/74897/diff/2/
> 
> 
> Testing
> ---
> 
> Done the required code changes and installed ranger.
> Validated the policy conditions with default comma separated parameters 
> passed with attributes.
> 
> 
> Thanks,
> 
> Himanshu Maurya
> 
>

[jira] [Comment Edited] (RANGER-4506) Illegal read lock usage in getMetadata/getKeyVersion

[Madhan Neethiraj (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824525#comment-17824525
 ] 

Madhan Neethiraj edited comment on RANGER-4506 at 3/7/24 9:50 PM:
--

[~jianchun], [~vikkumar]   - thank you both for the detailed reporting and 
analysis of the issue.  Fix in RANGER-3989 required sizable updates to prevent 
issues in KMS handling of simultaneous calls. [~vikkumar]  helped review that 
fix. [~jianchun] - it's good to have you as well review Ranger KMS! Thank you.


was (Author: madhan.neethiraj):
[~jianchun], [~vikkumar]   - thank you both for the detailed reporting and 
analysis of the issue.  Fix in RANGER-3989 required sizable updates to prevent 
issues in handling simultaneous calls in KMS. [~vikkumar]  heled review that 
fix. [~jianchun] - it's good to have you as well review Ranger KMS! Thank you.

> Illegal read lock usage in getMetadata/getKeyVersion
> 
>
> Key: RANGER-4506
> URL: https://issues.apache.org/jira/browse/RANGER-4506
> Project: Ranger
>  Issue Type: Bug
>  Components: kms
>Reporter: Jianchun Xu
>Assignee: Vikas Kumar
>Priority: Major
>
> RangerKeyStoreProvider illegally writes to key store under Read lock. This 
> happens in both getMetadata and getKeyVersion.
> E.g. in following getKeyVersion, under Read lock, the code calls 
> `dbStore.engineLoad(null, masterKey)` which reloads all the keys. Since 
> multiple threads can hold the Read lock, multiple threads can enter and 
> reload all the keys. Thus the 2nd `dbStore.engineContainsAlias(versionName)` 
> test and following `dbStore.engineGetDecryptedZoneKeyByte(versionName)` can 
> both get wrong result if another thread is reloading keys.
> [https://github.com/apache/ranger/blob/master/kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java#L331]
> {code:java}
> @Override
> public KeyVersion getKeyVersion(String versionName) throws IOException {
> if (logger.isDebugEnabled()) {
> logger.debug("==> getKeyVersion({})", versionName);
> }
> KeyVersion ret = null;
> try (AutoClosableReadLock ignored = new AutoClosableReadLock(lock)) {
> if (keyVaultEnabled) {
> try {
> boolean versionNameExists = 
> dbStore.engineContainsAlias(versionName);
> if (!versionNameExists) {
> dbStore.engineLoad(null, masterKey);
> versionNameExists = 
> dbStore.engineContainsAlias(versionName);
> }
> if (versionNameExists) {
> byte[] decryptKeyByte;
> try {
> decryptKeyByte = 
> dbStore.engineGetDecryptedZoneKeyByte(versionName);
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-4506) Illegal read lock usage in getMetadata/getKeyVersion

[Madhan Neethiraj (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824525#comment-17824525
 ] 

Madhan Neethiraj commented on RANGER-4506:
--

[~jianchun], [~vikkumar]   - thank you both for the detailed reporting and 
analysis of the issue.  Fix in RANGER-3989 required sizable updates to prevent 
issues in handling simultaneous calls in KMS. [~vikkumar]  heled review that 
fix. [~jianchun] - it's good to have you as well review Ranger KMS! Thank you.

> Illegal read lock usage in getMetadata/getKeyVersion
> 
>
> Key: RANGER-4506
> URL: https://issues.apache.org/jira/browse/RANGER-4506
> Project: Ranger
>  Issue Type: Bug
>  Components: kms
>Reporter: Jianchun Xu
>Assignee: Vikas Kumar
>Priority: Major
>
> RangerKeyStoreProvider illegally writes to key store under Read lock. This 
> happens in both getMetadata and getKeyVersion.
> E.g. in following getKeyVersion, under Read lock, the code calls 
> `dbStore.engineLoad(null, masterKey)` which reloads all the keys. Since 
> multiple threads can hold the Read lock, multiple threads can enter and 
> reload all the keys. Thus the 2nd `dbStore.engineContainsAlias(versionName)` 
> test and following `dbStore.engineGetDecryptedZoneKeyByte(versionName)` can 
> both get wrong result if another thread is reloading keys.
> [https://github.com/apache/ranger/blob/master/kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java#L331]
> {code:java}
> @Override
> public KeyVersion getKeyVersion(String versionName) throws IOException {
> if (logger.isDebugEnabled()) {
> logger.debug("==> getKeyVersion({})", versionName);
> }
> KeyVersion ret = null;
> try (AutoClosableReadLock ignored = new AutoClosableReadLock(lock)) {
> if (keyVaultEnabled) {
> try {
> boolean versionNameExists = 
> dbStore.engineContainsAlias(versionName);
> if (!versionNameExists) {
> dbStore.engineLoad(null, masterKey);
> versionNameExists = 
> dbStore.engineContainsAlias(versionName);
> }
> if (versionNameExists) {
> byte[] decryptKeyByte;
> try {
> decryptKeyByte = 
> dbStore.engineGetDecryptedZoneKeyByte(versionName);
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-4506) Illegal read lock usage in getMetadata/getKeyVersion

[Jianchun Xu (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824486#comment-17824486
 ] 

Jianchun Xu commented on RANGER-4506:
-

[~vikkumar] Thanks for looking. With your info and re-checking now I realized 
the issue was fixed in 
[https://github.com/apache/ranger/commit/6e7cca01a041dc68e961fe13ef699ee799f8a890]
 (Jan 16 2023). I had discovered the issue on an earlier version. Prior to 
above commit, "dbStore.engineLoad" implementation does not operate on a temp 
map; instead it directly operates on ["synchronized (keyEntries)", 
"keyEntries.clear()", 
...|https://github.com/apache/ranger/commit/6e7cca01a041dc68e961fe13ef699ee799f8a890#diff-285d977276d43d86adf274f6817e023b704ada82bd505b0aed83edba133d2d45].
 (before-change RangerKeyStore.java, engineLoad). After above commit it should 
be ok as in your analysis.

> Illegal read lock usage in getMetadata/getKeyVersion
> 
>
> Key: RANGER-4506
> URL: https://issues.apache.org/jira/browse/RANGER-4506
> Project: Ranger
>  Issue Type: Bug
>  Components: kms
>Reporter: Jianchun Xu
>Assignee: Vikas Kumar
>Priority: Major
>
> RangerKeyStoreProvider illegally writes to key store under Read lock. This 
> happens in both getMetadata and getKeyVersion.
> E.g. in following getKeyVersion, under Read lock, the code calls 
> `dbStore.engineLoad(null, masterKey)` which reloads all the keys. Since 
> multiple threads can hold the Read lock, multiple threads can enter and 
> reload all the keys. Thus the 2nd `dbStore.engineContainsAlias(versionName)` 
> test and following `dbStore.engineGetDecryptedZoneKeyByte(versionName)` can 
> both get wrong result if another thread is reloading keys.
> [https://github.com/apache/ranger/blob/master/kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java#L331]
> {code:java}
> @Override
> public KeyVersion getKeyVersion(String versionName) throws IOException {
> if (logger.isDebugEnabled()) {
> logger.debug("==> getKeyVersion({})", versionName);
> }
> KeyVersion ret = null;
> try (AutoClosableReadLock ignored = new AutoClosableReadLock(lock)) {
> if (keyVaultEnabled) {
> try {
> boolean versionNameExists = 
> dbStore.engineContainsAlias(versionName);
> if (!versionNameExists) {
> dbStore.engineLoad(null, masterKey);
> versionNameExists = 
> dbStore.engineContainsAlias(versionName);
> }
> if (versionNameExists) {
> byte[] decryptKeyByte;
> try {
> decryptKeyByte = 
> dbStore.engineGetDecryptedZoneKeyByte(versionName);
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74926: RANGER-4076: Support Java 17 for build and runtime

[Rakesh Gupta]

> On March 6, 2024, 6:30 a.m., Kishor Gollapalliwar wrote:
> > pom.xml
> > Lines 428 (patched)
> > 
> >
> > Are we planning to make changes in this class later to support build, 
> > if not instead of skipping class, shouldn't we remove it ?

Yes, I have create separate task Jira(RANGER-4692) to handle.


- Rakesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74926/#review226290
---


On March 5, 2024, 7:10 a.m., Rakesh Gupta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74926/
> ---
> 
> (Updated March 5, 2024, 7:10 a.m.)
> 
> 
> Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay 
> Kulkarni, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, 
> sanket shelar, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4076
> https://issues.apache.org/jira/browse/RANGER-4076
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.
> 
> This patch enables manual and Docker-based build of Apache Ranger. It ensures 
> compatibility with Java 8, Java 11, and Java 17 for both build and runtime 
> environments.
> (I encountered a TestCase Failure in the KnoxRangerTest class, so currently I 
> had excluded that class from jdk17 build)
> 
> 
> Diffs
> -
> 
>   agents-common/pom.xml 12e093f78 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/GraalScriptEngineCreator.java
>  512d8d3ca 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java
>  db620df92 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ScriptEngineUtil.java
>  8d76c1d81 
>   dev-support/ranger-docker/.env 60bc4a9f2 
>   dev-support/ranger-docker/Dockerfile.ranger-build 9a192f152 
>   dev-support/ranger-docker/docker-compose.ranger-build.yml 3dd4a4a06 
>   distro/src/main/assembly/admin-web.xml 245d9ca09 
>   docs/src/site/resources/index.js bb876f28d 
>   hdfs-agent/pom.xml dece8f46f 
>   kms/pom.xml 2739bb81a 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 
> 7188b19b2 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 
> b6fc32950 
>   plugin-nestedstructure/pom.xml 0e208f99c 
>   
> plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
>  77767767c 
>   
> plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
>  9cb161b8d 
>   pom.xml cec2390f2 
>   ranger-tools/pom.xml cac8d7ba3 
>   
> ranger-tools/src/test/java/org/apache/ranger/policyengine/RangerPolicyEnginePerformanceTest.java
>  187d20227 
>   security-admin/pom.xml fc59287d0 
> 
> 
> Diff: https://reviews.apache.org/r/74926/diff/1/
> 
> 
> Testing
> ---
> 
> This patch has been successfully tested and verified with Java versions 8, 
> 11, and 17. The verification process included both manual and Docker setup. 
> Now we are able to build and run on same jdk.
> 
> 
> Thanks,
> 
> Rakesh Gupta
> 
>

[jira] [Updated] (RANGER-4739) Unit test failures in KnoxRangerTest for JDK 17

[Rakesh Gupta (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rakesh Gupta updated RANGER-4739:
-
Parent: (was: RANGER-4076)
Issue Type: Task  (was: Sub-task)

> Unit test failures in KnoxRangerTest for JDK 17
> ---
>
> Key: RANGER-4739
> URL: https://issues.apache.org/jira/browse/RANGER-4739
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Reporter: Rakesh Gupta
>Assignee: Rakesh Gupta
>Priority: Major
>
> Build in master branch (mvn clean compile package install) fails due to 
> KnoxRangerTest class:
> {code:java}
> ERROR Error in generating certificate: java.lang.IllegalAccessException: 
> class org.apache.knox.gateway.util.X509CertificateUtil cannot access class 
> sun.security.x509.X509CertInfo (in module java.base) because module java.base 
> does not export sun.security.x509 to unnamed module @148080bb
> java.lang.IllegalAccessException: class 
> org.apache.knox.gateway.util.X509CertificateUtil cannot access class 
> sun.security.x509.X509CertInfo (in module java.base) because module java.base 
> does not export sun.security.x509 to unnamed module @148080bb
>   at 
> java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:392)
>   at 
> java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:674)
>   at 
> java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:489)
>   at 
> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
>   at 
> org.apache.knox.gateway.util.X509CertificateUtil.generateCertificate(X509CertificateUtil.java:69)
>   at 
> org.apache.knox.gateway.services.security.impl.DefaultKeystoreService.addCertForGateway(DefaultKeystoreService.java:186)
>   at 
> org.apache.knox.gateway.services.security.impl.DefaultKeystoreService.addSelfSignedCertForGateway(DefaultKeystoreService.java:166)
>   at 
> org.apache.knox.gateway.services.security.impl.DefaultKeystoreService.addSelfSignedCertForGateway(DefaultKeystoreService.java:160)
>   at 
> org.apache.knox.gateway.services.security.impl.JettySSLService.init(JettySSLService.java:88)
>   at 
> org.apache.knox.gateway.services.DefaultGatewayServices.init(DefaultGatewayServices.java:104)
>   at 
> org.apache.knox.gateway.GatewayTestDriver.setupGateway(GatewayTestDriver.java:162)
>   at 
> org.apache.ranger.services.knox.KnoxRangerTest.setupSuite(KnoxRangerTest.java:60)
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.266 
> s <<< FAILURE! - in org.apache.ranger.services.knox.KnoxRangerTest
> [ERROR] org.apache.ranger.services.knox.KnoxRangerTest  Time elapsed: 7.266 s 
>  <<< ERROR!
> java.lang.NullPointerException: Cannot invoke 
> "java.security.cert.Certificate.getEncoded()" because "entry.chain[i]" is null
>   at 
> java.base/sun.security.provider.JavaKeyStore.engineStore(JavaKeyStore.java:606)
>   at 
> java.base/sun.security.util.KeyStoreDelegator.engineStore(KeyStoreDelegator.java:190)
>   at java.base/java.security.KeyStore.store(KeyStore.java:1404)
>   at 
> org.apache.knox.gateway.services.security.impl.DefaultKeystoreService.writeKeyStoreToFile(DefaultKeystoreService.java:503)
>   at 
> org.apache.knox.gateway.services.security.impl.DefaultKeystoreService.addCertForGateway(DefaultKeystoreService.java:194)
>   at 
> org.apache.knox.gateway.services.security.impl.DefaultKeystoreService.addSelfSignedCertForGateway(DefaultKeystoreService.java:166)
>   at 
> org.apache.knox.gateway.services.security.impl.DefaultKeystoreService.addSelfSignedCertForGateway(DefaultKeystoreService.java:160)
>   at 
> org.apache.knox.gateway.services.security.impl.JettySSLService.init(JettySSLService.java:88)
>   at 
> org.apache.knox.gateway.services.DefaultGatewayServices.init(DefaultGatewayServices.java:104)
>   at 
> org.apache.knox.gateway.GatewayTestDriver.setupGateway(GatewayTestDriver.java:162)
>   at 
> org.apache.ranger.services.knox.KnoxRangerTest.setupSuite(KnoxRangerTest.java:60)
>   at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> [INFO] 
> [INFO] Results:
> [INFO] 
> [ERROR] Errors: 
> [ERROR]   KnoxRangerTest.setupSuite:60 » NullPointer Cannot invoke 
> "java.security.cert.Certificate.getEncoded()" because "entry.chain[i]" is null
> [INFO] 
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {code}
> Even after handling above JDK 17 error through config "--add-exports 
> java.base/sun.security.x509=ALL-UNNAMED" , its give Error
> {code:java}
> ERROR Error in generating certificate: java.lang.NoSuchFieldException: 
> md5WithRSAEncryption_oid
> java.lang.NoSuchFieldException: md5WithRSAEncryption_oid
>   at java.base/java.lang.Class.getDeclare

Re: [PR] Resolve ranger-examples-distro module compile error [ranger]

[via GitHub]

vikaskumareinstein commented on PR #71:
URL: https://github.com/apache/ranger/pull/71#issuecomment-1983558014

   While trying to build the Apache Ranger on Ubuntu 20.04, Using Java 1.8, I 
came across this issue. Not sure, whether this issue is related to maven or 
Ranger. It is failing on **Apache Ranger Distribution ** step. I am attaching 
the screenshot for reference.
   
   
![apache_ranger_issue](https://github.com/apache/ranger/assets/34993771/55ee31e6-136d-4589-a4fa-a7ba0ead3e4c)
   
   
   
   
   
   ```
   [ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-assembly-plugin:2.6:single (default) on project 
ranger-distro: Failed to create assem
   bly: Error creating assembly archive schema-registry-plugin: Problem 
creating jar: jar:file:/home/swapnendu_b/apache-ranger-2.4.0/distro/target/ran
   
ger-distro-2.4.0.jar!/META-INF/maven/org.apache.ranger/ranger-distro/pom.properties:
 JAR entry META-INF/maven/org.apache.ranger/ranger-distro/pom.p
   roperties not found in 
/home/swapnendu_b/apache-ranger-2.4.0/distro/target/ranger-distro-2.4.0.jar -> 
[Help 1]
   [ERROR] 
   [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
   [ERROR] Re-run Maven using the -X switch to enable full debug logging.
   [ERROR] 
   [ERROR] For more information about the errors and possible solutions, please 
read the following articles:
   [ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
   [ERROR] 
   [ERROR] After correcting the problems, you can resume the build with the 
command
   [ERROR]   mvn  -rf :ranger-distro
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (RANGER-4506) Illegal read lock usage in getMetadata/getKeyVersion

[Vikas Kumar (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824400#comment-17824400
 ] 

Vikas Kumar commented on RANGER-4506:
-

*Update:*

[~jianchun] , thanks for your review.

I analysed the code considering  the points you raised, following is my 
analysis:

let's take the example of getkeyVersion() method that you pointed:

*Step1:* Thread1 (let's call it {*}T1{*}) enters the method with input 
*zonekey1@1*

*Step2:* T1 acquires the ReadLock and checks 
*_dbStore.engineContainsAlias(versionName);_* a get/read operation on 
ConcurrentHashMap (keyEntries)

*Step3:* zonekey1@1 not found in map and calls the engineLoad. This method does 
DB operation, reads all keys and updates the underlying keyEntries map. Please 
note, it creates a temp map while iterating through the all keys and at last 
assigns the new map to keyEntries reference ( that is marked volatile).

*Step4:* Now next is, *engineGetDecryptedZoneKeyByte,* it again gets the key 
from the same map.

           Now if another thread *T2 for zonekey2@1* acquired Read lock and 
in-process of updating the keyEntries map while *T1* is reading that map, then 
*T1* may get different view.

_But *T2* is simply refreshing the map. This refresh will not change anything 
for T1, zonekey1@1 and their corresponding value will be changed only through 
create/delete/rollover API. And these API methods are protected with the 
corresponding write lock. So keyentry for T1 will be same even if another 
thread (T2) updates the "keyEntries" map reference._ 

However, inconsistencies may occur with multiple KMS instances. Like assume 
above explained operations are executing on KMS-1 and KMS-2 got request to 
delete that key in between and got processed. This write lock (this is not a 
distributed lock) will not stop another instance from deleting the key.

In that case, at step4 at KMS-1, T1 thread inside  
*engineGetDecryptedZoneKeyByte() will get null as key and thus NPE.*

*The request will fail, and it should fail.*

As per my understanding, purpose of introducing this lock is to 
protect/safeguard  *DB read/write* operations, not the in-memory map.

[~jianchun] , Please go through my analysis and let me know if I missed 
anything. 

Tagging [~madhan] ( author of this change) for his review. Thanks.

> Illegal read lock usage in getMetadata/getKeyVersion
> 
>
> Key: RANGER-4506
> URL: https://issues.apache.org/jira/browse/RANGER-4506
> Project: Ranger
>  Issue Type: Bug
>  Components: kms
>Reporter: Jianchun Xu
>Assignee: Vikas Kumar
>Priority: Major
>
> RangerKeyStoreProvider illegally writes to key store under Read lock. This 
> happens in both getMetadata and getKeyVersion.
> E.g. in following getKeyVersion, under Read lock, the code calls 
> `dbStore.engineLoad(null, masterKey)` which reloads all the keys. Since 
> multiple threads can hold the Read lock, multiple threads can enter and 
> reload all the keys. Thus the 2nd `dbStore.engineContainsAlias(versionName)` 
> test and following `dbStore.engineGetDecryptedZoneKeyByte(versionName)` can 
> both get wrong result if another thread is reloading keys.
> [https://github.com/apache/ranger/blob/master/kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java#L331]
> {code:java}
> @Override
> public KeyVersion getKeyVersion(String versionName) throws IOException {
> if (logger.isDebugEnabled()) {
> logger.debug("==> getKeyVersion({})", versionName);
> }
> KeyVersion ret = null;
> try (AutoClosableReadLock ignored = new AutoClosableReadLock(lock)) {
> if (keyVaultEnabled) {
> try {
> boolean versionNameExists = 
> dbStore.engineContainsAlias(versionName);
> if (!versionNameExists) {
> dbStore.engineLoad(null, masterKey);
> versionNameExists = 
> dbStore.engineContainsAlias(versionName);
> }
> if (versionNameExists) {
> byte[] decryptKeyByte;
> try {
> decryptKeyByte = 
> dbStore.engineGetDecryptedZoneKeyByte(versionName);
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4742) Audits logged for Ranger trino plugin does not contain cluster name and query fields

[Abhishek (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek updated RANGER-4742:
-
Labels: Trino  (was: )

> Audits logged for Ranger trino plugin does not contain cluster name and query 
> fields
> 
>
> Key: RANGER-4742
> URL: https://issues.apache.org/jira/browse/RANGER-4742
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Abhishek
>Assignee: Pradeep Agrawal
>Priority: Major
>  Labels: Trino
>
> {color:#22}The audits logged for Ranger Trino plugin does not contain the 
> following fields :-{color}
> {color:#22}1. Cluster name{color}
> {color:#22}2. Query - In Hive (or any other plugin audits), there is a 
> field which contains the field for the operation performed, for e.g, "Select 
> * from table_name;". This field is not present in the trino audits{color}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (RANGER-4742) Audits logged for Ranger trino plugin does not contain cluster name and query fields

[Abhishek (Jira)]

Abhishek created RANGER-4742:


 Summary: Audits logged for Ranger trino plugin does not contain 
cluster name and query fields
 Key: RANGER-4742
 URL: https://issues.apache.org/jira/browse/RANGER-4742
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Reporter: Abhishek
Assignee: Pradeep Agrawal


{color:#22}The audits logged for Ranger Trino plugin does not contain the 
following fields :-{color}
{color:#22}1. Cluster name{color}
{color:#22}2. Query - In Hive (or any other plugin audits), there is a 
field which contains the field for the operation performed, for e.g, "Select * 
from table_name;". This field is not present in the trino audits{color}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-4719) Policy condition expressions are split by the Ranger UI on commas

[Himanshu Maurya (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824347#comment-17824347
 ] 

Himanshu Maurya commented on RANGER-4719:
-

Hi [~madhan] 

I have updated the patch in the review board :- 
[https://reviews.apache.org/r/74897/]
Kindly review and merge this

Thanks and Regards

>  Policy condition expressions are split by the Ranger UI on commas
> --
>
> Key: RANGER-4719
> URL: https://issues.apache.org/jira/browse/RANGER-4719
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Himanshu Maurya
>Assignee: Himanshu Maurya
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: Screenshot 2024-02-15 at 5.07.02 PM.png, Screenshot 
> 2024-02-15 at 5.20.36 PM.png, Screenshot 2024-02-15 at 5.27.54 PM.png, 
> Screenshot 2024-02-20 at 5.26.46 PM.png, image-2024-02-20-17-28-17-767.png, 
> image-2024-02-20-17-28-28-558.png, image-2024-02-20-17-39-05-432.png
>
>
> While using attributes with default values introduced in 
> [RANGER-3997|https://issues.apache.org/jira/browse/RANGER-3997] like 
> GET_USER_ATTR('state', 'null') in policy condition.
> It is observed that ranger is splitting condition string in to 2 parts 
> separated by comma of the parameters passed to GET_USER_ATTR(), due to this 
> it is not taking default value and conditions are converted to invalid 
> strings.
> Kindly refer to screenshots attached below.
> !Screenshot 2024-02-15 at 5.07.02 PM.png|width=645,height=376!
>  
> !Screenshot 2024-02-15 at 5.20.36 PM.png|width=645,height=112!
>  
> !Screenshot 2024-02-15 at 5.27.54 PM.png|width=642,height=176!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4741) Hive plugin optimization to avoid excessive metastore API calls

[Madhan Neethiraj (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated RANGER-4741:
-
Attachment: RANGER-4741.patch

> Hive plugin optimization to avoid excessive metastore API calls
> ---
>
> Key: RANGER-4741
> URL: https://issues.apache.org/jira/browse/RANGER-4741
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: RANGER-4741.patch
>
>
> Authorizing access to tables with large number of columns can take a long 
> time, as shown below. Time taken to for a table with 400 columns takes about 
> 100 seconds.
> {noformat}
> 0: jdbc:hive2://localhost:1> SELECT * FROM large_tbl_4000;
> ...
> No rows selected (98.674 seconds)
> 0: jdbc:hive2://localhost:1> SELECT * FROM large_tbl_1000;
> ...
> No rows selected (10.4 seconds)
> {noformat}
>  
> For each column referenced in the query, Ranger Hive authorizer calls 
> metastore API to obtain owner of the table. Optimizing to call the metastore 
> API once per table can significantly reduce the time taken to authorize 
> queries.
> Here is the time taken to query the same tables with the Ranger Hive 
> authorizer optimized to call metastore API only once per table referenced in 
> the query:
> {noformat}
> 0: jdbc:hive2://localhost:1> SELECT * FROM large_tbl_4000;
> ...
> No rows selected (1.328 seconds)
> 0: jdbc:hive2://localhost:1> SELECT * FROM large_tbl_1000;
> ...
> No rows selected (0.194 seconds)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Review Request 74929: RANGER-4741: Hive plugin optimization to avoid excessive metastore API calls

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74929/
---

Review request for ranger.


Bugs: RANGER-4741
https://issues.apache.org/jira/browse/RANGER-4741


Repository: ranger


Description
---

updated Ranger Hive authorizer to get owner of a table only once per query


Diffs
-

  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
 9b25e2b8a 


Diff: https://reviews.apache.org/r/74929/diff/1/


Testing
---

- verified that the optimization results in significantly reduced time to 
authorize access to tables with large number of columns. For a table with 4000 
columns, the time taked reduced from 100 seconds to 1.5 seconds
- verified that all existing tests pass successfully


Thanks,

Madhan Neethiraj

[jira] [Created] (RANGER-4741) Hive plugin optimization to avoid excessive metastore API calls

[Madhan Neethiraj (Jira)]

Madhan Neethiraj created RANGER-4741:


 Summary: Hive plugin optimization to avoid excessive metastore API 
calls
 Key: RANGER-4741
 URL: https://issues.apache.org/jira/browse/RANGER-4741
 Project: Ranger
  Issue Type: Improvement
  Components: plugins
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


Authorizing access to tables with large number of columns can take a long time, 
as shown below. Time taken to for a table with 400 columns takes about 100 
seconds.
{noformat}
0: jdbc:hive2://localhost:1> SELECT * FROM large_tbl_4000;
...
No rows selected (98.674 seconds)


0: jdbc:hive2://localhost:1> SELECT * FROM large_tbl_1000;
...
No rows selected (10.4 seconds)
{noformat}
 

For each column referenced in the query, Ranger Hive authorizer calls metastore 
API to obtain owner of the table. Optimizing to call the metastore API once per 
table can significantly reduce the time taken to authorize queries.

Here is the time taken to query the same tables with the Ranger Hive authorizer 
optimized to call metastore API only once per table referenced in the query:
{noformat}
0: jdbc:hive2://localhost:1> SELECT * FROM large_tbl_4000;
...
No rows selected (1.328 seconds)


0: jdbc:hive2://localhost:1> SELECT * FROM large_tbl_1000;
...
No rows selected (0.194 seconds)
{noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)