[jira] [Commented] (RANGER-4170) keep policy-cache snapshot for each update

[Madhan Neethiraj (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707783#comment-17707783
 ] 

Madhan Neethiraj commented on RANGER-4170:
--

[~ximz]  - can you update the Jira with the version of Ranger this issue was 
seen in? There were few fixes related to cache in 2.2.0/2.3.0 releases: 
RANGER-3343, RANGER-3253, RANGER-3371, RANGER-3554.

> keep policy-cache snapshot for each update
> --
>
> Key: RANGER-4170
> URL: https://issues.apache.org/jira/browse/RANGER-4170
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Xiaomin Zhang
>Priority: Major
>
> We observed strange issue that plugin is not able to find the correct policy 
> hence failing the access request.  This is intermittent and not reproduced at 
> will.  This issue seems gone after restarting the component or RangerAdmin or 
> make some slight changes to the affected policy, or just wait for some while 
> without doing any thing.  
> Since policy-cache snapshot at the issue time is critical to debug this 
> issue, but there is a major gap here to keep the snapshot. We need some 
> improvement in component plugin to keep every policy-cache snapshot when it's 
> updated, so that we can trace back to find the policy-cache at the point of 
> time when issue recurs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (RANGER-4170) keep policy-cache snapshot for each update

[Xiaomin Zhang (Jira)]

Xiaomin Zhang created RANGER-4170:
-

 Summary: keep policy-cache snapshot for each update
 Key: RANGER-4170
 URL: https://issues.apache.org/jira/browse/RANGER-4170
 Project: Ranger
  Issue Type: Improvement
  Components: plugins
Reporter: Xiaomin Zhang


We observed strange issue that plugin is not able to find the correct policy 
hence failing the access request.  This is intermittent and not reproduced at 
will.  This issue seems gone after restarting the component or RangerAdmin or 
make some slight changes to the affected policy, or just wait for some while 
without doing any thing.  

Since policy-cache snapshot at the issue time is critical to debug this issue, 
but there is a major gap here to keep the snapshot. We need some improvement in 
component plugin to keep every policy-cache snapshot when it's updated, so that 
we can trace back to find the policy-cache at the point of time when issue 
recurs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (RANGER-4159) Ranger audit metrics deletion is failing

[Dineshkumar Yadav (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dineshkumar Yadav resolved RANGER-4159.
---
  Assignee: Dineshkumar Yadav
Resolution: Invalid

> Ranger audit metrics deletion is failing
> 
>
> Key: RANGER-4159
> URL: https://issues.apache.org/jira/browse/RANGER-4159
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Dineshkumar Yadav
>Assignee: Dineshkumar Yadav
>Priority: Major
>
> Ranger audit metrics deletion is failing



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (RANGER-4169) Add pull request template

[Abhishek Kumar (Jira)]

Abhishek Kumar created RANGER-4169:
--

 Summary: Add pull request template
 Key: RANGER-4169
 URL: https://issues.apache.org/jira/browse/RANGER-4169
 Project: Ranger
  Issue Type: Task
  Components: Ranger
Reporter: Abhishek Kumar
Assignee: Abhishek Kumar


Add a pull request template to help developers contributing via PRs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (RANGER-4168) Unit test cases are missing for TagDBStore.java

[Ramachandran (Jira)]

Ramachandran created RANGER-4168:


 Summary: Unit test cases are missing for  TagDBStore.java
 Key: RANGER-4168
 URL: https://issues.apache.org/jira/browse/RANGER-4168
 Project: Ranger
  Issue Type: Task
  Components: Ranger
Reporter: Ramachandran
Assignee: Ramachandran
 Fix For: 3.0.0


This also needs to be added as part of improvement Task which will help the 
developer whenever they make code changes



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74299: RANGER-4029 Ranger cannot build with HBase 2.5.x versions because preBalance coprocessor hook syntax changed in HBASE-26147

[bhavik patel]

> On Feb. 14, 2023, 3:14 p.m., Ramesh Mani wrote:
> > hbase-compat-2.4/pom.xml
> > Lines 20 (patched)
> > 
> >
> > Could you please elaborate why hbase-compact-2.4 and hbase-compact-2.5 
> > module is created?
> 
> Rajeshbabu Chintaguntla wrote:
> As part of HBASE-26147 preBalance and postBalance coprocessor hooks 
> syntax changed because of that building ranger with 2.5.x is failing. If we 
> fix the syntax then we cannot build with versions below 2.5.0. So we need to 
> have both compatible implementation in our code and depending upon the 
> profile selected the hbase ragner plugin gets build.
> 
> -  public void preBalance(ObserverContext c)
> +  public void preBalance(ObserverContext 
> c, BalanceRequest request)
>throws IOException {
> 
> 
> 
>  @Override
>  public void postBalance(final 
> ObserverContext ctx,
> -List plans) throws IOException {
> +BalanceRequest request, List plans) throws 
> IOException {
>if (!plans.isEmpty()) {
> 
> 
> Already 3 point releases went in 2.5.x release lines with the 
> compatibilty issue.
> 
> 
> hbase-compact-2.4 is for the balance hooks implementations of HBase 
> versions below 2.5.0
> hbase-compact-2.5 is for the balance hooks implementations of HBase 
> versions 2.5.0+.

I think it's better to have only one of hbase version-2.5.x


- bhavik


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74299/#review225187
---


On Feb. 9, 2023, 12:38 p.m., Rajeshbabu Chintaguntla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74299/
> ---
> 
> (Updated Feb. 9, 2023, 12:38 p.m.)
> 
> 
> Review request for ranger, bhavik patel, Madhan Neethiraj, Ramesh Mani, and 
> Vishal Suvagia.
> 
> 
> Bugs: RANGER-4029
> https://issues.apache.org/jira/browse/RANGER-4029
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Added HBase compatible implementations balancer hooks for both HBase versions 
> below 2.5.0 and HBase version starting from HBase 2.5.0. So we can specify 
> which compabile version of HBase need to be used why building the hbase 
> ranger plugins.
> 
> 
> Diffs
> -
> 
>   hbase-agent/pom.xml c6acf9756 
>   
> hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
>  281f3ec75 
>   hbase-compat-2.4/pom.xml PRE-CREATION 
>   
> hbase-compat-2.4/src/main/java/org/apache/ranger/authorization/hbase/CompatMasterObserver.java
>  PRE-CREATION 
>   hbase-compat-2.5/pom.xml PRE-CREATION 
>   
> hbase-compat-2.5/src/main/java/org/apache/ranger/authorization/hbase/CompatMasterObserver.java
>  PRE-CREATION 
>   pom.xml 5d9fd9222 
>   ranger-hbase-plugin-shim/pom.xml add905f46 
>   
> ranger-hbase-plugin-shim/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
>  3e6e5d08c 
> 
> 
> Diff: https://reviews.apache.org/r/74299/diff/1/
> 
> 
> Testing
> ---
> 
> Build the ranger with HBase compat profile 2.4 and HBase compat profile 2.5 
> and tested those in the real cluster. All the functionality working as 
> expected.
> 
> 
> Thanks,
> 
> Rajeshbabu Chintaguntla
> 
>

Re: Review Request 74377: RANGER-4163: Upgrade spring framework to 5.3.26 and jettison to 1.5.4

[bhavik patel]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74377/#review225332
---


Ship it!




Ship It!

- bhavik patel


On March 31, 2023, 12:41 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74377/
> ---
> 
> (Updated March 31, 2023, 12:41 p.m.)
> 
> 
> Review request for ranger, bhavik patel, Abhay Kulkarni, Madhan Neethiraj, 
> Nikhil P, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4163
> https://issues.apache.org/jira/browse/RANGER-4163
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Here I am proposing to Upgrade Spring Framework to 5.3.26 and jettison to 
> 1.5.4
> 
> 
> Diffs
> -
> 
>   pom.xml 3a039565d 
> 
> 
> Diff: https://reviews.apache.org/r/74377/diff/1/
> 
> 
> Testing
> ---
> 
> Tested ranger admin installation, password change, CRUD operation on Ranger 
> service, policy, users and group.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

[jira] [Commented] (RANGER-4167) Kafka Test Suite Issues (RANGER-4144 on 3.0.0 branch)

[Selvamohan Neethiraj (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707741#comment-17707741
 ] 

Selvamohan Neethiraj commented on RANGER-4167:
--

Commit :   06fc1ef09220cf82096a990a072ff18dbb148151 - Done after review 
approved.

> Kafka Test Suite Issues (RANGER-4144 on 3.0.0 branch)
> -
>
> Key: RANGER-4167
> URL: https://issues.apache.org/jira/browse/RANGER-4167
> Project: Ranger
>  Issue Type: Bug
>  Components: plugins
>Affects Versions: 3.0.0
>Reporter: Selvamohan Neethiraj
>Assignee: Selvamohan Neethiraj
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: 0001-RANGER-4167-fixes-to-kafka-test-suite-issues.patch
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [ranger] dependabot[bot] opened a new pull request, #239: Bump junit from 4.13 to 4.13.1 in /plugin-kafka

[via GitHub]

dependabot[bot] opened a new pull request, #239:
URL: https://github.com/apache/ranger/pull/239

   Bumps [junit](https://github.com/junit-team/junit4) from 4.13 to 4.13.1.
   
   Release notes
   Sourced from https://github.com/junit-team/junit4/releases";>junit's 
releases.
   
   JUnit 4.13.1
   Please refer to the https://github.com/junit-team/junit/blob/HEAD/doc/ReleaseNotes4.13.1.md";>release
 notes for details.
   
   
   
   Changelog
   Sourced from https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.13.1.md";>junit's
 changelog.
   
   Summary of changes in version 4.13.1
   Rules
   Security fix: TemporaryFolder now limits access to 
temporary folders on Java 1.7 or later
   A local information disclosure vulnerability in 
TemporaryFolder has been fixed. See the published https://github.com/junit-team/junit4/security/advisories/GHSA-269g-pwp5-87pp";>security
 advisory for details.
   Test Runners
   [Pull request https://redirect.github.com/junit-team/junit4/issues/1669";>#1669:](https://redirect.github.com/junit-team/junit/pull/1669";>junit-team/junit#1669)
 Make FrameworkField constructor public
   Prior to this change, custom runners could make 
FrameworkMethod instances, but not FrameworkField 
instances. This small change allows for both now, because 
FrameworkField's constructor has been promoted from 
package-private to public.
   
   
   
   Commits
   
   https://github.com/junit-team/junit4/commit/1b683f4ec07bcfa40149f086d32240f805487e66";>1b683f4
 [maven-release-plugin] prepare release r4.13.1
   https://github.com/junit-team/junit4/commit/ce6ce3aadc070db2902698fe0d3dc6729cd631f2";>ce6ce3a
 Draft 4.13.1 release notes
   https://github.com/junit-team/junit4/commit/c29dd8239d6b353e699397eb090a1fd27411fa24";>c29dd82
 Change version to 4.13.1-SNAPSHOT
   https://github.com/junit-team/junit4/commit/1d174861f0b64f97ab0722bb324a760bfb02f567";>1d17486
 Add a link to assertThrows in exception testing
   https://github.com/junit-team/junit4/commit/543905df72ff10364b94dda27552efebf3dd04e9";>543905d
 Use separate line for annotation in Javadoc
   https://github.com/junit-team/junit4/commit/510e906b391e7e46a346e1c852416dc7be934944";>510e906
 Add sub headlines to class Javadoc
   https://github.com/junit-team/junit4/commit/610155b8c22138329f0723eec22521627dbc52ae";>610155b
 Merge pull request from GHSA-269g-pwp5-87pp
   https://github.com/junit-team/junit4/commit/b6cfd1e3d736cc2106242a8be799615b472c7fec";>b6cfd1e
 Explicitly wrap float parameter for consistency (https://redirect.github.com/junit-team/junit4/issues/1671";>#1671)
   https://github.com/junit-team/junit4/commit/a5d205c7956dbed302b3bb5ecde5ba4299f0b646";>a5d205c
 Fix GitHub link in FAQ (https://redirect.github.com/junit-team/junit4/issues/1672";>#1672)
   https://github.com/junit-team/junit4/commit/3a5c6b4d08f408c8ca6a8e0bae71a9bc5a8f97e8";>3a5c6b4
 Deprecated since jdk9 replacing constructor instance of Double and Float (https://redirect.github.com/junit-team/junit4/issues/1660";>#1660)
   Additional commits viewable in https://github.com/junit-team/junit4/compare/r4.13...r4.13.1";>compare 
view
   
   
   
   
   
   [![Dependabot compatibility 
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=junit:junit&package-manager=maven&previous-version=4.13&new-version=4.13.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
   
   Dependabot will resolve any conflicts with this PR as long as you don't 
alter it yourself. You can also trigger a rebase manually by commenting 
`@dependabot rebase`.
   
   [//]: # (dependabot-automerge-start)
   [//]: # (dependabot-automerge-end)
   
   ---
   
   
   Dependabot commands and options
   
   
   You can trigger Dependabot actions by commenting on this PR:
   - `@dependabot rebase` will rebase this PR
   - `@dependabot recreate` will recreate this PR, overwriting any edits that 
have been made to it
   - `@dependabot merge` will merge this PR after your CI passes on it
   - `@dependabot squash and merge` will squash and merge this PR after your CI 
passes on it
   - `@dependabot cancel merge` will cancel a previously requested merge and 
block automerging
   - `@dependabot reopen` will reopen this PR if it is closed
   - `@dependabot close` will close this PR and stop Dependabot recreating it. 
You can achieve the same result by closing it manually
   - `@dependabot ignore this major version` will close this PR and stop 
Dependabot creating any more for this major version (unless you reopen the PR 
or upgrade to it yourself)
   - `@dependabot ignore this minor version` will close this PR and stop 
Dependabot creating any more for this minor version (unless you reopen the PR 
or upgrade to it yourself)
   - `@dependabot ignore this dependency` will close this PR and stop 
Dependabot creating any more for this dependency (unless you reopen the PR or 
upgrade to it y

[jira] [Updated] (RANGER-4124) Value of -Dlogback.configurationFile in ranger-tagsync-services.sh is not configurable

[Madhan Neethiraj (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated RANGER-4124:
-
Fix Version/s: 2.4.1

> Value of -Dlogback.configurationFile in ranger-tagsync-services.sh  is not 
> configurable
> ---
>
> Key: RANGER-4124
> URL: https://issues.apache.org/jira/browse/RANGER-4124
> Project: Ranger
>  Issue Type: Improvement
>  Components: tagsync
>Reporter: Lei Yao
>Assignee: Lei Yao
>Priority: Minor
> Fix For: 3.0.0, 2.4.1
>
> Attachments: 
> 0001-RANGER-4124-Value-of-Dlogback.configurationFile-in-r.patch
>
>
> In file ranger-tagsync-services.sh，we can find -Dlogback.configurationFile 
> not configurable
> {code:java}
> nohup java -Dproc_rangertagsync ${JAVA_OPTS} 
> -Dlogdir="${RANGER_TAGSYNC_LOG_DIR}" 
> -Dlogback.configurationFile=file:/etc/ranger/tagsync/conf/logback.xml -cp 
> "${cp}" org.apache.ranger.tagsync.process.TagSynchronizer  > 
> ${RANGER_TAGSYNC_LOG_DIR}/tagsync.out 2>&1 &
> {code}
> Like  usersync， suggest add variable TAGSYNC_CONF_DIR  and set  
> -Dlogback.configurationFile=file:${TAGSYNC_CONF_DIR}/logback.xml
> After applying this patch, we can support customization setting for 
> logback.xml location in cluster management tools like Ambari.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (RANGER-4124) Value of -Dlogback.configurationFile in ranger-tagsync-services.sh is not configurable

[Lei Yao (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei Yao reassigned RANGER-4124:
---

Assignee: Lei Yao

> Value of -Dlogback.configurationFile in ranger-tagsync-services.sh  is not 
> configurable
> ---
>
> Key: RANGER-4124
> URL: https://issues.apache.org/jira/browse/RANGER-4124
> Project: Ranger
>  Issue Type: Improvement
>  Components: tagsync
>Reporter: Lei Yao
>Assignee: Lei Yao
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: 
> 0001-RANGER-4124-Value-of-Dlogback.configurationFile-in-r.patch
>
>
> In file ranger-tagsync-services.sh，we can find -Dlogback.configurationFile 
> not configurable
> {code:java}
> nohup java -Dproc_rangertagsync ${JAVA_OPTS} 
> -Dlogdir="${RANGER_TAGSYNC_LOG_DIR}" 
> -Dlogback.configurationFile=file:/etc/ranger/tagsync/conf/logback.xml -cp 
> "${cp}" org.apache.ranger.tagsync.process.TagSynchronizer  > 
> ${RANGER_TAGSYNC_LOG_DIR}/tagsync.out 2>&1 &
> {code}
> Like  usersync， suggest add variable TAGSYNC_CONF_DIR  and set  
> -Dlogback.configurationFile=file:${TAGSYNC_CONF_DIR}/logback.xml
> After applying this patch, we can support customization setting for 
> logback.xml location in cluster management tools like Ambari.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-4147) Ranger KMS consume 50% of CPU memory

[Madhan Neethiraj (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707686#comment-17707686
 ] 

Madhan Neethiraj commented on RANGER-4147:
--

[~bpatel]  - higher the number of simultaneous requests, the higher the CPU 
usage would be. However, higher memory requirement, of 40gb, doesn't seem 
right. To further troubleshoot this issue, can you please attach KMS server 
access log file?

> Ranger KMS consume 50% of CPU memory 
> -
>
> Key: RANGER-4147
> URL: https://issues.apache.org/jira/browse/RANGER-4147
> Project: Ranger
>  Issue Type: Bug
>  Components: kms
>Affects Versions: 2.3.0
> Environment: secured
>Reporter: Bhavik Patel
>Priority: Critical
>
> Ranger KMS consume 50% of CPU memory and many time it consume even 150%
> cc: [~dhavalshah9131] [~kirbyzhou] 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74378: RANGER-4167: fixes to kafka test suite issues

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74378/#review225331
---


Ship it!




Ship It!

- Madhan Neethiraj


On April 1, 2023, 10:15 p.m., Selvamohan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74378/
> ---
> 
> (Updated April 1, 2023, 10:15 p.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, 
> Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4167
> https://issues.apache.org/jira/browse/RANGER-4167
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> When running with -DskipTests=false, the kafka test suite failing (see below 
> for the errors)
> 
> 
> Diffs
> -
> 
>   plugin-kafka/pom.xml 75db9e9de 
>   
> plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java
>  1060c4d78 
>   pom.xml 3a039565d 
> 
> 
> Diff: https://reviews.apache.org/r/74378/diff/1/
> 
> 
> Testing
> ---
> 
> Command used to build: 
> $ mvn -Pall -pl plugin-kafka -am -DskipTests=false clean package install
> 
> Run build before the change:
> 
> [WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 
> 0.033 s - in 
> org.apache.ranger.authorization.kafka.authorizer.KafkaRangerAuthorizerSASLSSLTest
> [INFO] 
> [INFO] Results:
> [INFO] 
> [ERROR] Errors: 
> [ERROR]   KafkaRangerAuthorizerGSSTest.setup:149 » ZooKeeperClientTimeout 
> Timed out waiting for connection while in state: CONNECTING
> [ERROR]   KafkaRangerAuthorizerTest.setup:140 » ZooKeeperClientTimeout Timed 
> out waiting for connection while in state: CONNECTING
> [ERROR]   KafkaRangerTopicCreationTest.setup:129 » ZooKeeperClientTimeout 
> Timed out waiting for connection while in state: CONNECTING
> [INFO] 
> [ERROR] Tests run: 4, Failures: 0, Errors: 3, Skipped: 1
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> 
> Run build after the change:
> 
> [INFO] ranger . SUCCESS [  2.913 
> s]
> [INFO] Credential Support . SUCCESS [  7.440 
> s]
> [INFO] Audit Component  SUCCESS [  9.794 
> s]
> [INFO] ranger-plugin-classloader .. SUCCESS [  3.524 
> s]
> [INFO] Common library for Plugins . SUCCESS [ 26.835 
> s]
> [INFO] Credential Builder . SUCCESS [  5.497 
> s]
> [INFO] KAFKA Security Plugin .. SUCCESS [ 47.131 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> 
> 
> 
> Thanks,
> 
> Selvamohan Neethiraj
> 
>