[jira] [Resolved] (RANGER-4325) Need api for collective search of user/group/roles

[Madhan Neethiraj (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved RANGER-4325.
--
Fix Version/s: 3.0.0
   Resolution: Fixed

[~prashant_satam]  - thank you for the enhancement. The patch is merged in 
RANGER-3923 branch.
{noformat}
commit 25162b8422bae6c3cd56351481f47c8327546408 (HEAD -> RANGER-3923, 
origin/RANGER-3923)
Author: Prashant Satam 
Date:   Wed Aug 23 11:02:05 2023 +0530

RANGER-4325: REST API to lookup principals (user/group/role) by name

Signed-off-by: Madhan Neethiraj 
{noformat}

> Need api for collective search of user/group/roles
> --
>
> Key: RANGER-4325
> URL: https://issues.apache.org/jira/browse/RANGER-4325
> Project: Ranger
>  Issue Type: Sub-task
>  Components: Ranger
>Reporter: Anand Nadar
>Assignee: Prashant Satam
>Priority: Major
> Fix For: 3.0.0
>
>
> The api should search partially for user/groups/roles and return the results 
> as an object with user/group/role list



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74528: RANGER-4325: GDS: Need api for collective search of user/group/roles

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74528/#review225672
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 23, 2023, 5:35 a.m., Prashant Satam wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74528/
> ---
> 
> (Updated Aug. 23, 2023, 5:35 a.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Subhrat Chaudhary.
> 
> 
> Bugs: RANGER-4325
> https://issues.apache.org/jira/browse/RANGER-4325
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> We get users,groups,roles all we get in response from one API 
> (/xusers/users/groups/roles) we can also pass Query Params like 
> name,isVisible Also we can PartialSearch Name for users,groups,roles
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> b4e3f57b8 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> 6b82aead4 
>   security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 1f282948b 
>   security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java 283d84fe1 
>   
> security-admin/src/main/java/org/apache/ranger/entity/view/VXXPrincipal.java 
> PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java 
> 9a2253a3d 
>   
> security-admin/src/main/java/org/apache/ranger/security/context/RangerAPIList.java
>  4398764ae 
>   security-admin/src/main/resources/META-INF/jpa_named_queries.xml 59a20a25e 
> 
> 
> Diff: https://reviews.apache.org/r/74528/diff/7/
> 
> 
> Testing
> ---
> 
> We get users,groups,roles combined in response from API 
> (/xusers/users/groups/roles) we can pass query params like name(PartialSearch 
> Available),isVisible to filter the users,groups,roles we get in response
> 
> 
> Thanks,
> 
> Prashant Satam
> 
>

Re: Review Request 74528: RANGER-4325: GDS: Need api for collective search of user/group/roles

[Prashant Satam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74528/
---

(Updated Aug. 23, 2023, 5:35 a.m.)


Review request for ranger, Madhan Neethiraj and Subhrat Chaudhary.


Bugs: RANGER-4325
https://issues.apache.org/jira/browse/RANGER-4325


Repository: ranger


Description
---

We get users,groups,roles all we get in response from one API 
(/xusers/users/groups/roles) we can also pass Query Params like name,isVisible 
Also we can PartialSearch Name for users,groups,roles


Diffs (updated)
-

  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql b4e3f57b8 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
6b82aead4 
  security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 1f282948b 
  security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java 283d84fe1 
  security-admin/src/main/java/org/apache/ranger/entity/view/VXXPrincipal.java 
PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java 9a2253a3d 
  
security-admin/src/main/java/org/apache/ranger/security/context/RangerAPIList.java
 4398764ae 
  security-admin/src/main/resources/META-INF/jpa_named_queries.xml 59a20a25e 


Diff: https://reviews.apache.org/r/74528/diff/7/

Changes: https://reviews.apache.org/r/74528/diff/6-7/


Testing
---

We get users,groups,roles combined in response from API 
(/xusers/users/groups/roles) we can pass query params like name(PartialSearch 
Available),isVisible to filter the users,groups,roles we get in response


Thanks,

Prashant Satam

[jira] [Commented] (RANGER-4196) Tomcat metrics collection

[Bhavik Patel (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757796#comment-17757796
 ] 

Bhavik Patel commented on RANGER-4196:
--

Thanks [~vikkumar] .


{code:java}
"KMS": {
    "GET_CURRENT_KEY_COUNT": 0,
    "DELETE_KEY_ELAPSED_TIME": 0,
    "EEK_DECRYPT_ELAPSED_TIME": 0,
    "GET_KEYS_METADATA_ELAPSED_TIME": 0,
    "EEK_GENERATE_ELAPSED_TIME": 0,
    "GET_CURRENT_KEY_ELAPSED_TIME": 0,
    "EEK_REENCRYPT_ELAPSED_TIME": 0,
    "KEY_CREATE_COUNT": 0,
    "UNAUTHORIZED_CALLS_COUNT": 0,
    "KEY_CREATE_ELAPSED_TIME": 0,
    "GET_KEY_VERSION_COUNT": 0,
    "ROLL_NEW_VERSION_ELAPSED_TIME": 0,
    "REENCRYPT_EEK_BATCH_COUNT": 0,
    "REENCRYPT_EEK_BATCH_ELAPSED_TIME": 0,
    "GET_KEYS_METADATA_COUNT": 0,
    "GET_KEY_VERSIONS_COUNT": 0,
    "GET_KEY_VERSIONS_ELAPSED_TIME": 0,
    "GET_KEYS_COUNT": 0,
    "EEK_GENERATE_COUNT": 0,
    "INVALIDATE_CACHE_COUNT": 0,
    "GET_METADATA_COUNT": 0,
    "REENCRYPT_EEK_BATCH_KEYS_COUNT": 0,
    "EEK_REENCRYPT_COUNT": 0,
    "UNAUTHENTICATED_CALLS_COUNT": 3,
    "GET_KEY_VERSION_ELAPSED_TIME": 0,
    "INVALIDATE_CACHE_ELAPSED_TIME": 0,
    "ROLL_NEW_VERSION_COUNT": 0,
    "EEK_DECRYPT_COUNT": 0,
    "GET_KEYS_METADATA_KEYNAMES_COUNT": 0,
    "DELETE_KEY_COUNT": 0,
    "GET_KEYS_ELAPSED_TIME": 0,
    "GET_METADATA_ELAPSED_TIME": 0,
    "TOTAL_CALL_COUNT": 6
  }, {code}
Regarding the above output, can you perform some operations than get the 
metrics and then we can actually map operations with TOTAL_CALL_COUNT

> Tomcat metrics collection 
> --
>
> Key: RANGER-4196
> URL: https://issues.apache.org/jira/browse/RANGER-4196
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Vikas Kumar
>Assignee: Vikas Kumar
>Priority: Major
>
> Like "JVM mterics" and other application metrics, Tomcat related metrics 
> would be very useful for following use cases:
> In case client gets "SoketTimeout" or Connection refused errors, having such 
> metrics at server end will really help triaging the issue.
> It should contain following metrics:
>  # maxAllowedConnection
>  # currentActiveConnectionCount
>  # ConnectionTimeout
>  # acceptCount
>  # maxContainerThreadsCount
>  # activeContainerThreadsCount
>  # minSpareThreadsCount // corePoolSize
> And other useful metrics.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-4119) [UI] Syntax check button missing in policy level condition

[Madhan Neethiraj (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757512#comment-17757512
 ] 

Madhan Neethiraj commented on RANGER-4119:
--

[~Dhaval.Rajpara]  - thank you for the update in react.js UI.

> [UI] Syntax check button missing in policy level condition
> --
>
> Key: RANGER-4119
> URL: https://issues.apache.org/jira/browse/RANGER-4119
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Affects Versions: 2.4.0
>Reporter: Madhan Neethiraj
>Assignee: Dhaval Rajpara
>Priority: Major
>  Labels: ranger-react
> Attachments: image-2023-03-03-00-04-10-367.png, 
> image-2023-03-03-00-04-45-845.png
>
>
> Ranger policy UI provides {{Syntax check}} button that allows policy authors 
> to check if condition expression entered is valid or not. This button is 
> available for condition in policy-item level, but its not available for 
> condition at policy level - as seen in following images:
>  
> Policy-item level:
> !image-2023-03-03-00-04-45-845.png|width=661,height=453!
>  
> Policy level:
> !image-2023-03-03-00-04-10-367.png|width=661,height=488!
>  
> CC: [~ni3galave], [~Dhaval.Rajpara] 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74557: RANGER-4196- Tomcat runtime metrics collection

[Ramachandran Krishnan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74557/#review225671
---




ranger-metrics/src/main/java/org/apache/ranger/metrics/source/RangerMetricsContainerSource.java
Lines 78 (patched)


Is it possible to write a UT for covering this


- Ramachandran Krishnan


On Aug. 22, 2023, 1:05 p.m., Vikas Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74557/
> ---
> 
> (Updated Aug. 22, 2023, 1:05 p.m.)
> 
> 
> Review request for ranger, Dhaval Shah, Abhay Kulkarni, Madhan Neethiraj, and 
> Sailaja Polavarapu.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-4196- Tomcat runtime metrics collection .
> 
> As part of RANGER-4196, developed the feature to collect runtime container 
> metrics that can help in debugging any performance issue. Although I have 
> tested it  for KMS only, but change has been made in a way that it will be 
> avilable for all ranger modules if registered with "ranger-metrics" module.
> 
> 
> Diffs
> -
> 
>   
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
>  cae9075a7 
>   
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServerMetricsCollector.java
>  PRE-CREATION 
>   ranger-metrics/pom.xml 44602c3b8 
>   
> ranger-metrics/src/main/java/org/apache/ranger/metrics/RangerMetricsSystemWrapper.java
>  cd806574d 
>   
> ranger-metrics/src/main/java/org/apache/ranger/metrics/source/RangerMetricsContainerSource.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74557/diff/1/
> 
> 
> Testing
> ---
> 
> build is passed.
> Patched the new jar on existing cluster and tried to fetch the kms metrics. 
> Following is the output:
> 
> command: curl -ivk --negotiate -u : -H "Content-Type: application/json" -X 
> GET http://:9292/kms/metrics/json
> 
> Output:
> 
> {
>   "KMS": {
> "GET_CURRENT_KEY_COUNT": 0,
> "DELETE_KEY_ELAPSED_TIME": 0,
> "EEK_DECRYPT_ELAPSED_TIME": 0,
> "GET_KEYS_METADATA_ELAPSED_TIME": 0,
> "EEK_GENERATE_ELAPSED_TIME": 0,
> "GET_CURRENT_KEY_ELAPSED_TIME": 0,
> "EEK_REENCRYPT_ELAPSED_TIME": 0,
> "KEY_CREATE_COUNT": 0,
> "UNAUTHORIZED_CALLS_COUNT": 0,
> "KEY_CREATE_ELAPSED_TIME": 0,
> "GET_KEY_VERSION_COUNT": 0,
> "ROLL_NEW_VERSION_ELAPSED_TIME": 0,
> "REENCRYPT_EEK_BATCH_COUNT": 0,
> "REENCRYPT_EEK_BATCH_ELAPSED_TIME": 0,
> "GET_KEYS_METADATA_COUNT": 0,
> "GET_KEY_VERSIONS_COUNT": 0,
> "GET_KEY_VERSIONS_ELAPSED_TIME": 0,
> "GET_KEYS_COUNT": 0,
> "EEK_GENERATE_COUNT": 0,
> "INVALIDATE_CACHE_COUNT": 0,
> "GET_METADATA_COUNT": 0,
> "REENCRYPT_EEK_BATCH_KEYS_COUNT": 0,
> "EEK_REENCRYPT_COUNT": 0,
> "UNAUTHENTICATED_CALLS_COUNT": 3,
> "GET_KEY_VERSION_ELAPSED_TIME": 0,
> "INVALIDATE_CACHE_ELAPSED_TIME": 0,
> "ROLL_NEW_VERSION_COUNT": 0,
> "EEK_DECRYPT_COUNT": 0,
> "GET_KEYS_METADATA_KEYNAMES_COUNT": 0,
> "DELETE_KEY_COUNT": 0,
> "GET_KEYS_ELAPSED_TIME": 0,
> "GET_METADATA_ELAPSED_TIME": 0,
> "TOTAL_CALL_COUNT": 6
>   },
>   "RangerWebContainer": {
> "ActiveConnectionsCount": 2,
> "ConnectionTimeout": 6,
> "MaxWorkerThreadsCount": 200,
> "TotalWorkerThreadsCount": 10,
> "KeepAliveTimeout": 6,
> "ActiveWorkerThreadsCount": 0,
> "ConnectionAcceptCount": 100,
> "MinSpareWorkerThreadsCount": 10,
> "MaxConnectionsCount": 8192
>   },
>   "RangerJvm": {
> "GcTimeTotal": 595,
> "SystemLoadAvg": 10.21,
> "ThreadsBusy": 5,
> "GcCountTotal": 17,
> "MemoryMax": 967311360,
> "MemoryCurrent": 273806280,
> "ThreadsWaiting": 17,
> "ProcessorsAvailable": 4,
> "GcTimeMax": 595,
> "ThreadsBlocked": 0,
> "ThreadsRemaining": 16
>   }
> }
> 
> Knowing the Tomcat connector type is also useful, so logged following on 
> container startup.
> 
> 2023-08-22 12:43:45,080 INFO  org.apache.ranger.server.tomcat.EmbeddedServer: 
> [vktomcat-1.vktomcat.-startStop-1]: Selected Tomcat protocolHandler: 
> "http-nio-9292"
> 
> 
> Thanks,
> 
> Vikas Kumar
> 
>

Re: Review Request 74552: RANGER-4353: Introduce config within Ranger to control retention period of x_trx_log data

[Ramachandran Krishnan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74552/#review225670
---


Ship it!




Ship It!

- Ramachandran Krishnan


On Aug. 21, 2023, 1:46 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74552/
> ---
> 
> (Updated Aug. 21, 2023, 1:46 p.m.)
> 
> 
> Review request for ranger, bhavik patel, Abhay Kulkarni, Madhan Neethiraj, 
> Nikhil P, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4353
> https://issues.apache.org/jira/browse/RANGER-4353
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** Currently ranger transaction entries are being stored 
> in x_trx_log table which may have lot of entries in few days. User need to 
> manually remove the entries from x_trx_log table time to time in order to 
> maintain disk space or handle disk space issues in a production env.
> 
> 
> ** Proposed Solution: **
> 
> 
> Option-1: Delete the entries during every start of ranger-admin service:
> 
> 
> This patch exposes two ranger configs 
> 1) "ranger.admin.init.purge.transaction_records" => should be set to 'true'. 
> default is false.
> 2) "ranger.admin.init.purge.transaction_records.retention.days" => which 
> accepts positive numerical values in days. 
> 
> 
> According to above configs During the start of ranger-admin x_trx_log table 
> entries older than the mentioned days shall be removed. 
> 
> 
> When "ranger.admin.init.purge.transaction_records" is set to 'true' and 
> "ranger.admin.init.purge.transaction_records.retention.days" value set to a 
> positive number this feature shall be affective.
> 
> 
> Option-2:  : User can call below mentioned REST api to delete the records. 
> User must use a credential which has admin role in the ranger to call this 
> REST API.
> 
> 
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
> application/json" -X DELETE 
> 'http://localhost:6080/service/public/v2/api/server/purge/records?type=trx_records=5'
> 
> 
> if retentionDays parameter is not provided then default value 180 shall be 
> considered.
> 
> 
> Note: The proposed implementation shall not delete entries every day as there 
> is no daemon process shall be running at the background, hence deletion of 
> entries shall be attempted only during the start of ranger.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> ed1ea0376 
>   security-admin/src/main/java/org/apache/ranger/db/XXAuthSessionDao.java 
> f69b8d2bb 
>   security-admin/src/main/java/org/apache/ranger/db/XXTrxLogDao.java 
> a83e91f5b 
>   security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 
> d2d76733e 
>   security-admin/src/main/resources/META-INF/jpa_named_queries.xml 1e8e4e2c5 
>   security-admin/src/main/resources/conf.dist/ranger-admin-site.xml d6bf174e9 
> 
> 
> Diff: https://reviews.apache.org/r/74552/diff/2/
> 
> 
> Testing
> ---
> 
> **Approach-1:**  ranger-admin start approach
> 
> Added "ranger.admin.init.purge.login_records" and 
> "ranger.admin.init.purge.login_records.retention.days" in 
> ranger-admin-site.xml with value 30.
> Added "ranger.admin.init.purge.transaction_records" and 
> "ranger.admin.init.purge.transaction_records.retention.days" in 
> ranger-admin-site.xml with value 31.
> Restarted the ranger-admin.
> 
> Logs:
> 
> 2023-08-22 09:56:19,595 INFO  org.apache.ranger.db.XXAuthSessionDao: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Deleting x_auth_sess records 
> that are older than 30 days, that is, older than Sun Jul 23 09:56:19 UTC 2023
> 2023-08-22 09:56:19,893 INFO  org.apache.ranger.db.XXAuthSessionDao: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Deleted 3743 x_auth_sess records
> 2023-08-22 09:56:19,893 INFO  org.apache.ranger.db.XXAuthSessionDao: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Updating x_trx_log.sess_id with 
> null which are older than 30 days, that is, older than Sun Jul 23 09:56:19 
> UTC 2023
> 2023-08-22 09:56:19,903 INFO  org.apache.ranger.db.XXAuthSessionDao: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Updated 9 x_trx_log records
> 2023-08-22 09:56:19,903 INFO  org.apache.ranger.biz.ServiceDBStore: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Deleted 3743 records from 
> x_auth_sess that are older than 30 days
> 2023-08-22 09:56:19,920 INFO  org.apache.ranger.db.XXTrxLogDao: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Deleting x_trx_log records that 
> are older than 31 days, that is, older than Sat Jul 22 09:56:19 UTC 2023
> 2023-08-22 09:56:19,924 INFO  org.apache.ranger.db.XXTrxLogDao: 
> 

Re: Review Request 74552: RANGER-4353: Introduce config within Ranger to control retention period of x_trx_log data

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74552/#review225669
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 21, 2023, 1:46 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74552/
> ---
> 
> (Updated Aug. 21, 2023, 1:46 p.m.)
> 
> 
> Review request for ranger, bhavik patel, Abhay Kulkarni, Madhan Neethiraj, 
> Nikhil P, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4353
> https://issues.apache.org/jira/browse/RANGER-4353
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** Currently ranger transaction entries are being stored 
> in x_trx_log table which may have lot of entries in few days. User need to 
> manually remove the entries from x_trx_log table time to time in order to 
> maintain disk space or handle disk space issues in a production env.
> 
> 
> ** Proposed Solution: **
> 
> 
> Option-1: Delete the entries during every start of ranger-admin service:
> 
> 
> This patch exposes two ranger configs 
> 1) "ranger.admin.init.purge.transaction_records" => should be set to 'true'. 
> default is false.
> 2) "ranger.admin.init.purge.transaction_records.retention.days" => which 
> accepts positive numerical values in days. 
> 
> 
> According to above configs During the start of ranger-admin x_trx_log table 
> entries older than the mentioned days shall be removed. 
> 
> 
> When "ranger.admin.init.purge.transaction_records" is set to 'true' and 
> "ranger.admin.init.purge.transaction_records.retention.days" value set to a 
> positive number this feature shall be affective.
> 
> 
> Option-2:  : User can call below mentioned REST api to delete the records. 
> User must use a credential which has admin role in the ranger to call this 
> REST API.
> 
> 
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
> application/json" -X DELETE 
> 'http://localhost:6080/service/public/v2/api/server/purge/records?type=trx_records=5'
> 
> 
> if retentionDays parameter is not provided then default value 180 shall be 
> considered.
> 
> 
> Note: The proposed implementation shall not delete entries every day as there 
> is no daemon process shall be running at the background, hence deletion of 
> entries shall be attempted only during the start of ranger.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> ed1ea0376 
>   security-admin/src/main/java/org/apache/ranger/db/XXAuthSessionDao.java 
> f69b8d2bb 
>   security-admin/src/main/java/org/apache/ranger/db/XXTrxLogDao.java 
> a83e91f5b 
>   security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 
> d2d76733e 
>   security-admin/src/main/resources/META-INF/jpa_named_queries.xml 1e8e4e2c5 
>   security-admin/src/main/resources/conf.dist/ranger-admin-site.xml d6bf174e9 
> 
> 
> Diff: https://reviews.apache.org/r/74552/diff/2/
> 
> 
> Testing
> ---
> 
> **Approach-1:**  ranger-admin start approach
> 
> Added "ranger.admin.init.purge.login_records" and 
> "ranger.admin.init.purge.login_records.retention.days" in 
> ranger-admin-site.xml with value 30.
> Added "ranger.admin.init.purge.transaction_records" and 
> "ranger.admin.init.purge.transaction_records.retention.days" in 
> ranger-admin-site.xml with value 31.
> Restarted the ranger-admin.
> 
> Logs:
> 
> 2023-08-22 09:56:19,595 INFO  org.apache.ranger.db.XXAuthSessionDao: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Deleting x_auth_sess records 
> that are older than 30 days, that is, older than Sun Jul 23 09:56:19 UTC 2023
> 2023-08-22 09:56:19,893 INFO  org.apache.ranger.db.XXAuthSessionDao: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Deleted 3743 x_auth_sess records
> 2023-08-22 09:56:19,893 INFO  org.apache.ranger.db.XXAuthSessionDao: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Updating x_trx_log.sess_id with 
> null which are older than 30 days, that is, older than Sun Jul 23 09:56:19 
> UTC 2023
> 2023-08-22 09:56:19,903 INFO  org.apache.ranger.db.XXAuthSessionDao: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Updated 9 x_trx_log records
> 2023-08-22 09:56:19,903 INFO  org.apache.ranger.biz.ServiceDBStore: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Deleted 3743 records from 
> x_auth_sess that are older than 30 days
> 2023-08-22 09:56:19,920 INFO  org.apache.ranger.db.XXTrxLogDao: 
> [myhost-1.myhost.root.hwx.site-startStop-1]: Deleting x_trx_log records that 
> are older than 31 days, that is, older than Sat Jul 22 09:56:19 UTC 2023
> 2023-08-22 09:56:19,924 INFO  org.apache.ranger.db.XXTrxLogDao: 
> 

[jira] [Comment Edited] (RANGER-4196) Tomcat metrics collection

[Vikas Kumar (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757463#comment-17757463
 ] 

Vikas Kumar edited comment on RANGER-4196 at 8/22/23 1:22 PM:
--

New metrics added:
{code:java}
"RangerWebContainer":
{     "ActiveConnectionsCount": 2,     "ConnectionTimeout": 6,     
"MaxWorkerThreadsCount": 200,     "TotalWorkerThreadsCount": 10,     
"KeepAliveTimeout": 6,     "ActiveWorkerThreadsCount": 0,     
"ConnectionAcceptCount": 100,     "MinSpareWorkerThreadsCount": 10,     
"MaxConnectionsCount": 8192   }
  {code}
{*}Description{*}: A new segment "RangerWebContainer" added. And following 
metrics were added:
 # MaxConnectionsCount : {color:#505f79}Max configured tomcat connection count. 
{color}
 # ConnectionAcceptCount : {color:#505f79}The maximum queue length for incoming 
connection requests when all possible request processing threads are in use. 
Any requests received when the queue is full will be refused. The default value 
is 100.{color}
 # {color:#505f79}*ActiveConnectionsCount* : active connections count{color}
 # {color:#505f79}*ConnectionTimeout* : The number of milliseconds this 
*Connector* will wait, after accepting a connection, for the request URI line 
to be presented. Default is 60 sec.{color}
 # {color:#505f79}*KeepAliveTimeout* : The number of milliseconds this 
*Connector* will wait for another HTTP request before closing the connection. 
The default value is to use the value that has been set for the 
*connectionTimeout* attribute.{color}
 # {color:#505f79}*MaxWorkerThreadsCount* : The max worker(container) Threads 
allowed/configured. Default is 200 .{color}
 # {color:#505f79}*MinSpareWorkerThreadsCount:*  core pool size of the worker 
executor.{color}
 # {color:#505f79}T{*}otalWorkerThreadsCount:{*}  total number of worker/worker 
threads in the process, irrespective of their state. They might me sitting idle 
or processing any request.{color}
 # {color:#505f79}*ActiveWorkerThreadsCount* : active worker/worker threads 
count in the process. These number of workers are active and processing a 
request.{color}

{color:#505f79}I will also update the spreadsheet with more details.{color}

{color:#505f79}Requesting community to please review and provide your feedback. 
Thanks.{color}

{color:#505f79}CC: [~bpatel]  [~madhan] {color}

 


was (Author: JIRAUSER295683):
New metrics added:
"RangerWebContainer": {
    "ActiveConnectionsCount": 2,
    "ConnectionTimeout": 6,
    "MaxWorkerThreadsCount": 200,
    "TotalWorkerThreadsCount": 10,
    "KeepAliveTimeout": 6,
    "ActiveWorkerThreadsCount": 0,
    "ConnectionAcceptCount": 100,
    "MinSpareWorkerThreadsCount": 10,
    "MaxConnectionsCount": 8192
  }
 

{*}Description{*}: A new segment "RangerWebContainer" added. And following 
metrics were added:
 # MaxConnectionsCount : {color:#505f79}Max configured tomcat connection count. 
{color}
 # ConnectionAcceptCount : {color:#505f79}The maximum queue length for incoming 
connection requests when all possible request processing threads are in use. 
Any requests received when the queue is full will be refused. The default value 
is 100.{color}
 # {color:#505f79}*ActiveConnectionsCount* : active connections count{color}
 # {color:#505f79}*ConnectionTimeout* : The number of milliseconds this 
*Connector* will wait, after accepting a connection, for the request URI line 
to be presented. Default is 60 sec.{color}
 # {color:#505f79}*KeepAliveTimeout* : The number of milliseconds this 
*Connector* will wait for another HTTP request before closing the connection. 
The default value is to use the value that has been set for the 
*connectionTimeout* attribute.{color}
 # {color:#505f79}*MaxWorkerThreadsCount* : The max worker(container) Threads 
allowed/configured. Default is 200 .{color}
 # {color:#505f79}*MinSpareWorkerThreadsCount:*  core pool size of the worker 
executor.{color}
 # {color:#505f79}T{*}otalWorkerThreadsCount:{*}  total number of worker/worker 
threads in the process, irrespective of their state. They might me sitting idle 
or processing any request.{color}
 # {color:#505f79}*ActiveWorkerThreadsCount* : active worker/worker threads 
count in the process. These number of workers are active and processing a 
request.{color}

{color:#505f79}I will also update the spreadsheet with more details.{color}

{color:#505f79}Requesting community to please review and provide your feedback. 
Thanks.{color}

{color:#505f79}CC: [~bpatel]  [~madhan] {color}

 

> Tomcat metrics collection 
> --
>
> Key: RANGER-4196
> URL: https://issues.apache.org/jira/browse/RANGER-4196
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Vikas Kumar
>Assignee: Vikas Kumar
>Priority: Major
>
> Like "JVM mterics" and other 

[jira] [Commented] (RANGER-4196) Tomcat metrics collection

[Vikas Kumar (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757463#comment-17757463
 ] 

Vikas Kumar commented on RANGER-4196:
-

New metrics added:
"RangerWebContainer": {
    "ActiveConnectionsCount": 2,
    "ConnectionTimeout": 6,
    "MaxWorkerThreadsCount": 200,
    "TotalWorkerThreadsCount": 10,
    "KeepAliveTimeout": 6,
    "ActiveWorkerThreadsCount": 0,
    "ConnectionAcceptCount": 100,
    "MinSpareWorkerThreadsCount": 10,
    "MaxConnectionsCount": 8192
  }
 

{*}Description{*}: A new segment "RangerWebContainer" added. And following 
metrics were added:
 # MaxConnectionsCount : {color:#505f79}Max configured tomcat connection count. 
{color}
 # ConnectionAcceptCount : {color:#505f79}The maximum queue length for incoming 
connection requests when all possible request processing threads are in use. 
Any requests received when the queue is full will be refused. The default value 
is 100.{color}
 # {color:#505f79}*ActiveConnectionsCount* : active connections count{color}
 # {color:#505f79}*ConnectionTimeout* : The number of milliseconds this 
*Connector* will wait, after accepting a connection, for the request URI line 
to be presented. Default is 60 sec.{color}
 # {color:#505f79}*KeepAliveTimeout* : The number of milliseconds this 
*Connector* will wait for another HTTP request before closing the connection. 
The default value is to use the value that has been set for the 
*connectionTimeout* attribute.{color}
 # {color:#505f79}*MaxWorkerThreadsCount* : The max worker(container) Threads 
allowed/configured. Default is 200 .{color}
 # {color:#505f79}*MinSpareWorkerThreadsCount:*  core pool size of the worker 
executor.{color}
 # {color:#505f79}T{*}otalWorkerThreadsCount:{*}  total number of worker/worker 
threads in the process, irrespective of their state. They might me sitting idle 
or processing any request.{color}
 # {color:#505f79}*ActiveWorkerThreadsCount* : active worker/worker threads 
count in the process. These number of workers are active and processing a 
request.{color}

{color:#505f79}I will also update the spreadsheet with more details.{color}

{color:#505f79}Requesting community to please review and provide your feedback. 
Thanks.{color}

{color:#505f79}CC: [~bpatel]  [~madhan] {color}

 

> Tomcat metrics collection 
> --
>
> Key: RANGER-4196
> URL: https://issues.apache.org/jira/browse/RANGER-4196
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Vikas Kumar
>Assignee: Vikas Kumar
>Priority: Major
>
> Like "JVM mterics" and other application metrics, Tomcat related metrics 
> would be very useful for following use cases:
> In case client gets "SoketTimeout" or Connection refused errors, having such 
> metrics at server end will really help triaging the issue.
> It should contain following metrics:
>  # maxAllowedConnection
>  # currentActiveConnectionCount
>  # ConnectionTimeout
>  # acceptCount
>  # maxContainerThreadsCount
>  # activeContainerThreadsCount
>  # minSpareThreadsCount // corePoolSize
> And other useful metrics.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (RANGER-4196) Tomcat metrics collection

[Vikas Kumar (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757450#comment-17757450
 ] 

Vikas Kumar edited comment on RANGER-4196 at 8/22/23 1:07 PM:
--

Hi, 

Review request raised: [https://reviews.apache.org/r/74557/]

Testing: 
build is passed.
Patched the new jar on existing cluster and tried to fetch the kms metrics. 
Following is the output:

command: curl -ivk --negotiate -u : -H "Content-Type: application/json" -X GET 
[http://:9292/kms/metrics/json|http://%3Ckmsserver%3E:9292/kms/metrics/json]

Output:
{code:java}
{
  "KMS": {
    "GET_CURRENT_KEY_COUNT": 0,
    "DELETE_KEY_ELAPSED_TIME": 0,
    "EEK_DECRYPT_ELAPSED_TIME": 0,
    "GET_KEYS_METADATA_ELAPSED_TIME": 0,
    "EEK_GENERATE_ELAPSED_TIME": 0,
    "GET_CURRENT_KEY_ELAPSED_TIME": 0,
    "EEK_REENCRYPT_ELAPSED_TIME": 0,
    "KEY_CREATE_COUNT": 0,
    "UNAUTHORIZED_CALLS_COUNT": 0,
    "KEY_CREATE_ELAPSED_TIME": 0,
    "GET_KEY_VERSION_COUNT": 0,
    "ROLL_NEW_VERSION_ELAPSED_TIME": 0,
    "REENCRYPT_EEK_BATCH_COUNT": 0,
    "REENCRYPT_EEK_BATCH_ELAPSED_TIME": 0,
    "GET_KEYS_METADATA_COUNT": 0,
    "GET_KEY_VERSIONS_COUNT": 0,
    "GET_KEY_VERSIONS_ELAPSED_TIME": 0,
    "GET_KEYS_COUNT": 0,
    "EEK_GENERATE_COUNT": 0,
    "INVALIDATE_CACHE_COUNT": 0,
    "GET_METADATA_COUNT": 0,
    "REENCRYPT_EEK_BATCH_KEYS_COUNT": 0,
    "EEK_REENCRYPT_COUNT": 0,
    "UNAUTHENTICATED_CALLS_COUNT": 3,
    "GET_KEY_VERSION_ELAPSED_TIME": 0,
    "INVALIDATE_CACHE_ELAPSED_TIME": 0,
    "ROLL_NEW_VERSION_COUNT": 0,
    "EEK_DECRYPT_COUNT": 0,
    "GET_KEYS_METADATA_KEYNAMES_COUNT": 0,
    "DELETE_KEY_COUNT": 0,
    "GET_KEYS_ELAPSED_TIME": 0,
    "GET_METADATA_ELAPSED_TIME": 0,
    "TOTAL_CALL_COUNT": 6
  },
  "RangerWebContainer": {
    "ActiveConnectionsCount": 2,
    "ConnectionTimeout": 6,
    "MaxWorkerThreadsCount": 200,
    "TotalWorkerThreadsCount": 10,
    "KeepAliveTimeout": 6,
    "ActiveWorkerThreadsCount": 0,
    "ConnectionAcceptCount": 100,
    "MinSpareWorkerThreadsCount": 10,
    "MaxConnectionsCount": 8192
  },
  "RangerJvm": {
    "GcTimeTotal": 595,
    "SystemLoadAvg": 10.21,
    "ThreadsBusy": 5,
    "GcCountTotal": 17,
    "MemoryMax": 967311360,
    "MemoryCurrent": 273806280,
    "ThreadsWaiting": 17,
    "ProcessorsAvailable": 4,
    "GcTimeMax": 595,
    "ThreadsBlocked": 0,
    "ThreadsRemaining": 16
  }
} {code}
Knowing the Tomcat connector type is also useful, so logged following on 
container startup.

2023-08-22 12:43:45,080 INFO org.apache.ranger.server.tomcat.EmbeddedServer: 
[vktomcat-1.vktomcat.\{*}_*_\{*}*-startStop-1]: Selected Tomcat 
protocolHandler: "http-nio-9292"


was (Author: JIRAUSER295683):
Hi, 

Review request raised: [https://reviews.apache.org/r/74557/]

Testing: 
build is passed.
Patched the new jar on existing cluster and tried to fetch the kms metrics. 
Following is the output:

command: curl -ivk --negotiate -u : -H "Content-Type: application/json" -X GET 
[http://:9292/kms/metrics/json|http://%3Ckmsserver%3E:9292/kms/metrics/json]

Output:

{
  "KMS": \{
"GET_CURRENT_KEY_COUNT": 0,
"DELETE_KEY_ELAPSED_TIME": 0,
"EEK_DECRYPT_ELAPSED_TIME": 0,
"GET_KEYS_METADATA_ELAPSED_TIME": 0,
"EEK_GENERATE_ELAPSED_TIME": 0,
"GET_CURRENT_KEY_ELAPSED_TIME": 0,
"EEK_REENCRYPT_ELAPSED_TIME": 0,
"KEY_CREATE_COUNT": 0,
"UNAUTHORIZED_CALLS_COUNT": 0,
"KEY_CREATE_ELAPSED_TIME": 0,
"GET_KEY_VERSION_COUNT": 0,
"ROLL_NEW_VERSION_ELAPSED_TIME": 0,
"REENCRYPT_EEK_BATCH_COUNT": 0,
"REENCRYPT_EEK_BATCH_ELAPSED_TIME": 0,
"GET_KEYS_METADATA_COUNT": 0,
"GET_KEY_VERSIONS_COUNT": 0,
"GET_KEY_VERSIONS_ELAPSED_TIME": 0,
"GET_KEYS_COUNT": 0,
"EEK_GENERATE_COUNT": 0,
"INVALIDATE_CACHE_COUNT": 0,
"GET_METADATA_COUNT": 0,
"REENCRYPT_EEK_BATCH_KEYS_COUNT": 0,
"EEK_REENCRYPT_COUNT": 0,
"UNAUTHENTICATED_CALLS_COUNT": 3,
"GET_KEY_VERSION_ELAPSED_TIME": 0,
"INVALIDATE_CACHE_ELAPSED_TIME": 0,
"ROLL_NEW_VERSION_COUNT": 0,
"EEK_DECRYPT_COUNT": 0,
"GET_KEYS_METADATA_KEYNAMES_COUNT": 0,
"DELETE_KEY_COUNT": 0,
"GET_KEYS_ELAPSED_TIME": 0,
"GET_METADATA_ELAPSED_TIME": 0,
"TOTAL_CALL_COUNT": 6
  },
  "RangerWebContainer": \{
"ActiveConnectionsCount": 2,
"ConnectionTimeout": 6,
"MaxWorkerThreadsCount": 200,
"TotalWorkerThreadsCount": 10,
"KeepAliveTimeout": 6,
"ActiveWorkerThreadsCount": 0,
"ConnectionAcceptCount": 100,
"MinSpareWorkerThreadsCount": 10,
"MaxConnectionsCount": 8192
  },
  "RangerJvm": \{
"GcTimeTotal": 595,
"SystemLoadAvg": 10.21,
"ThreadsBusy": 5,
"GcCountTotal": 17,
"MemoryMax": 967311360,
"MemoryCurrent": 273806280,
"ThreadsWaiting": 17,
"ProcessorsAvailable": 4,
"GcTimeMax": 595,
"ThreadsBlocked": 0,
"ThreadsRemaining": 16
  }
}

Knowing the Tomcat connector type is also useful, so 

[jira] [Commented] (RANGER-4196) Tomcat metrics collection

[Vikas Kumar (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757450#comment-17757450
 ] 

Vikas Kumar commented on RANGER-4196:
-

Hi, 

Review request raised: [https://reviews.apache.org/r/74557/]

Testing: 
build is passed.
Patched the new jar on existing cluster and tried to fetch the kms metrics. 
Following is the output:

command: curl -ivk --negotiate -u : -H "Content-Type: application/json" -X GET 
[http://:9292/kms/metrics/json|http://%3Ckmsserver%3E:9292/kms/metrics/json]

Output:

{
  "KMS": \{
"GET_CURRENT_KEY_COUNT": 0,
"DELETE_KEY_ELAPSED_TIME": 0,
"EEK_DECRYPT_ELAPSED_TIME": 0,
"GET_KEYS_METADATA_ELAPSED_TIME": 0,
"EEK_GENERATE_ELAPSED_TIME": 0,
"GET_CURRENT_KEY_ELAPSED_TIME": 0,
"EEK_REENCRYPT_ELAPSED_TIME": 0,
"KEY_CREATE_COUNT": 0,
"UNAUTHORIZED_CALLS_COUNT": 0,
"KEY_CREATE_ELAPSED_TIME": 0,
"GET_KEY_VERSION_COUNT": 0,
"ROLL_NEW_VERSION_ELAPSED_TIME": 0,
"REENCRYPT_EEK_BATCH_COUNT": 0,
"REENCRYPT_EEK_BATCH_ELAPSED_TIME": 0,
"GET_KEYS_METADATA_COUNT": 0,
"GET_KEY_VERSIONS_COUNT": 0,
"GET_KEY_VERSIONS_ELAPSED_TIME": 0,
"GET_KEYS_COUNT": 0,
"EEK_GENERATE_COUNT": 0,
"INVALIDATE_CACHE_COUNT": 0,
"GET_METADATA_COUNT": 0,
"REENCRYPT_EEK_BATCH_KEYS_COUNT": 0,
"EEK_REENCRYPT_COUNT": 0,
"UNAUTHENTICATED_CALLS_COUNT": 3,
"GET_KEY_VERSION_ELAPSED_TIME": 0,
"INVALIDATE_CACHE_ELAPSED_TIME": 0,
"ROLL_NEW_VERSION_COUNT": 0,
"EEK_DECRYPT_COUNT": 0,
"GET_KEYS_METADATA_KEYNAMES_COUNT": 0,
"DELETE_KEY_COUNT": 0,
"GET_KEYS_ELAPSED_TIME": 0,
"GET_METADATA_ELAPSED_TIME": 0,
"TOTAL_CALL_COUNT": 6
  },
  "RangerWebContainer": \{
"ActiveConnectionsCount": 2,
"ConnectionTimeout": 6,
"MaxWorkerThreadsCount": 200,
"TotalWorkerThreadsCount": 10,
"KeepAliveTimeout": 6,
"ActiveWorkerThreadsCount": 0,
"ConnectionAcceptCount": 100,
"MinSpareWorkerThreadsCount": 10,
"MaxConnectionsCount": 8192
  },
  "RangerJvm": \{
"GcTimeTotal": 595,
"SystemLoadAvg": 10.21,
"ThreadsBusy": 5,
"GcCountTotal": 17,
"MemoryMax": 967311360,
"MemoryCurrent": 273806280,
"ThreadsWaiting": 17,
"ProcessorsAvailable": 4,
"GcTimeMax": 595,
"ThreadsBlocked": 0,
"ThreadsRemaining": 16
  }
}

Knowing the Tomcat connector type is also useful, so logged following on 
container startup.

2023-08-22 12:43:45,080 INFO  org.apache.ranger.server.tomcat.EmbeddedServer: 
[vktomcat-1.vktomcat.{*}_*_{*}*-startStop-1]: Selected Tomcat protocolHandler: 
"http-nio-9292"

> Tomcat metrics collection 
> --
>
> Key: RANGER-4196
> URL: https://issues.apache.org/jira/browse/RANGER-4196
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Vikas Kumar
>Assignee: Vikas Kumar
>Priority: Major
>
> Like "JVM mterics" and other application metrics, Tomcat related metrics 
> would be very useful for following use cases:
> In case client gets "SoketTimeout" or Connection refused errors, having such 
> metrics at server end will really help triaging the issue.
> It should contain following metrics:
>  # maxAllowedConnection
>  # currentActiveConnectionCount
>  # ConnectionTimeout
>  # acceptCount
>  # maxContainerThreadsCount
>  # activeContainerThreadsCount
>  # minSpareThreadsCount // corePoolSize
> And other useful metrics.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Review Request 74557: RANGER-4196- Tomcat runtime metrics collection

[Vikas Kumar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74557/
---

Review request for ranger, Dhaval Shah, Abhay Kulkarni, Madhan Neethiraj, and 
Sailaja Polavarapu.


Repository: ranger


Description
---

RANGER-4196- Tomcat runtime metrics collection .

As part of RANGER-4196, developed the feature to collect runtime container 
metrics that can help in debugging any performance issue. Although I have 
tested it  for KMS only, but change has been made in a way that it will be 
avilable for all ranger modules if registered with "ranger-metrics" module.


Diffs
-

  
embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
 cae9075a7 
  
embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServerMetricsCollector.java
 PRE-CREATION 
  ranger-metrics/pom.xml 44602c3b8 
  
ranger-metrics/src/main/java/org/apache/ranger/metrics/RangerMetricsSystemWrapper.java
 cd806574d 
  
ranger-metrics/src/main/java/org/apache/ranger/metrics/source/RangerMetricsContainerSource.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/74557/diff/1/


Testing
---

build is passed.
Patched the new jar on existing cluster and tried to fetch the kms metrics. 
Following is the output:

command: curl -ivk --negotiate -u : -H "Content-Type: application/json" -X GET 
http://:9292/kms/metrics/json

Output:

{
  "KMS": {
"GET_CURRENT_KEY_COUNT": 0,
"DELETE_KEY_ELAPSED_TIME": 0,
"EEK_DECRYPT_ELAPSED_TIME": 0,
"GET_KEYS_METADATA_ELAPSED_TIME": 0,
"EEK_GENERATE_ELAPSED_TIME": 0,
"GET_CURRENT_KEY_ELAPSED_TIME": 0,
"EEK_REENCRYPT_ELAPSED_TIME": 0,
"KEY_CREATE_COUNT": 0,
"UNAUTHORIZED_CALLS_COUNT": 0,
"KEY_CREATE_ELAPSED_TIME": 0,
"GET_KEY_VERSION_COUNT": 0,
"ROLL_NEW_VERSION_ELAPSED_TIME": 0,
"REENCRYPT_EEK_BATCH_COUNT": 0,
"REENCRYPT_EEK_BATCH_ELAPSED_TIME": 0,
"GET_KEYS_METADATA_COUNT": 0,
"GET_KEY_VERSIONS_COUNT": 0,
"GET_KEY_VERSIONS_ELAPSED_TIME": 0,
"GET_KEYS_COUNT": 0,
"EEK_GENERATE_COUNT": 0,
"INVALIDATE_CACHE_COUNT": 0,
"GET_METADATA_COUNT": 0,
"REENCRYPT_EEK_BATCH_KEYS_COUNT": 0,
"EEK_REENCRYPT_COUNT": 0,
"UNAUTHENTICATED_CALLS_COUNT": 3,
"GET_KEY_VERSION_ELAPSED_TIME": 0,
"INVALIDATE_CACHE_ELAPSED_TIME": 0,
"ROLL_NEW_VERSION_COUNT": 0,
"EEK_DECRYPT_COUNT": 0,
"GET_KEYS_METADATA_KEYNAMES_COUNT": 0,
"DELETE_KEY_COUNT": 0,
"GET_KEYS_ELAPSED_TIME": 0,
"GET_METADATA_ELAPSED_TIME": 0,
"TOTAL_CALL_COUNT": 6
  },
  "RangerWebContainer": {
"ActiveConnectionsCount": 2,
"ConnectionTimeout": 6,
"MaxWorkerThreadsCount": 200,
"TotalWorkerThreadsCount": 10,
"KeepAliveTimeout": 6,
"ActiveWorkerThreadsCount": 0,
"ConnectionAcceptCount": 100,
"MinSpareWorkerThreadsCount": 10,
"MaxConnectionsCount": 8192
  },
  "RangerJvm": {
"GcTimeTotal": 595,
"SystemLoadAvg": 10.21,
"ThreadsBusy": 5,
"GcCountTotal": 17,
"MemoryMax": 967311360,
"MemoryCurrent": 273806280,
"ThreadsWaiting": 17,
"ProcessorsAvailable": 4,
"GcTimeMax": 595,
"ThreadsBlocked": 0,
"ThreadsRemaining": 16
  }
}

Knowing the Tomcat connector type is also useful, so logged following on 
container startup.

2023-08-22 12:43:45,080 INFO  org.apache.ranger.server.tomcat.EmbeddedServer: 
[vktomcat-1.vktomcat.-startStop-1]: Selected Tomcat protocolHandler: 
"http-nio-9292"


Thanks,

Vikas Kumar

[jira] [Updated] (RANGER-4353) Introduce config within Ranger to control retention period of x_trx_log data

[Pradeep Agrawal (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pradeep Agrawal updated RANGER-4353:

Attachment: RANGER-4353-2.patch

> Introduce config within Ranger to control retention period of x_trx_log data
> 
>
> Key: RANGER-4353
> URL: https://issues.apache.org/jira/browse/RANGER-4353
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Reporter: Madhan Neethiraj
>Assignee: Pradeep Agrawal
>Priority: Major
> Attachments: RANGER-4353-2.patch
>
>
> RANGER-4255 introduced REST API to purge x_auth_sess table records that were 
> created earlier than 180 days (by default). This purging should be extended 
> to x_trx_log table records as well, as it can take up significant space. 
> x_trx_log table track details of changes to individual attributes of 
> policies/services/security-zones/users/groups/roles.
> CC: [~pradeep], [~abhay], [~vel], [~ankita], [~suchnit] 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4353) Introduce config within Ranger to control retention period of x_trx_log data

[Pradeep Agrawal (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pradeep Agrawal updated RANGER-4353:

Attachment: (was: 
0001-RANGER-4353-Introduce-option-in-Ranger-to-control-re.patch)

> Introduce config within Ranger to control retention period of x_trx_log data
> 
>
> Key: RANGER-4353
> URL: https://issues.apache.org/jira/browse/RANGER-4353
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Reporter: Madhan Neethiraj
>Assignee: Pradeep Agrawal
>Priority: Major
> Attachments: RANGER-4353-2.patch
>
>
> RANGER-4255 introduced REST API to purge x_auth_sess table records that were 
> created earlier than 180 days (by default). This purging should be extended 
> to x_trx_log table records as well, as it can take up significant space. 
> x_trx_log table track details of changes to individual attributes of 
> policies/services/security-zones/users/groups/roles.
> CC: [~pradeep], [~abhay], [~vel], [~ankita], [~suchnit] 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74514: RANGER-4119 : [UI] Syntax check button missing in policy level condition

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74514/#review225668
---


Ship it!




Ship It!

- Mugdha Varadkar


On July 25, 2023, 11:42 a.m., Dhaval Rajpara wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74514/
> ---
> 
> (Updated July 25, 2023, 11:42 a.m.)
> 
> 
> Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Harshal Chavan, 
> Kishor Gollapalliwar, Madhan Neethiraj, Mehul Parikh, Mugdha Varadkar, Nitin 
> Galave, Pradeep Agrawal, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4119
> https://issues.apache.org/jira/browse/RANGER-4119
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger policy UI provides Syntax check button that allows policy authors to 
> check if condition expression entered is valid or not. This button is 
> available for condition in policy-item level, but its not available for 
> condition at policy level - as seen in following images:
> 
> 
> Diffs
> -
> 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-abfs.json 
> 7dcf38895 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json 
> 2f511eff5 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-knox.json 
> 410b9ef58 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-ozone.json 
> 4b899736b 
>   
> agents-common/src/main/resources/service-defs/ranger-servicedef-schema-registry.json
>  987a50fc5 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-solr.json 
> de3399845 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-tag.json 
> 7cb523075 
>   security-admin/src/main/webapp/react-webapp/src/components/Editable.jsx 
> ea22b32ad 
>   security-admin/src/main/webapp/react-webapp/src/utils/XAUtils.js 30fc4225b 
>   
> security-admin/src/main/webapp/react-webapp/src/views/AuditEvent/AdminLogs/PolicyViewDetails.jsx
>  1f89900eb 
>   
> security-admin/src/main/webapp/react-webapp/src/views/PolicyListing/AddUpdatePolicyForm.jsx
>  13a8eaf4f 
>   
> security-admin/src/main/webapp/react-webapp/src/views/PolicyListing/PolicyConditionsComp.jsx
>  9498d3fe3 
>   
> security-admin/src/main/webapp/react-webapp/src/views/PolicyListing/PolicyPermissionItem.jsx
>  380c2ec30 
>   
> security-admin/src/main/webapp/react-webapp/src/views/ServiceManager/ServiceAuditFilter.jsx
>  f75ddd4c3 
>   
> security-admin/src/main/webapp/react-webapp/src/views/ServiceManager/ServiceForm.jsx
>  83b2b386b 
> 
> 
> Diff: https://reviews.apache.org/r/74514/diff/2/
> 
> 
> Testing
> ---
> 
> 1)Build and Verified Ranger Admin setup with these changes.
> 2) Validated below scenarios on new UI:
>1. Tested Resource Based/Tag Based/ KMS Policy CRUD and check that policy 
> Conditions populate properly and give proper validation for text area.
>2. Resource level policy conditions provided run type validation.
>3. Policy Item level policy condition has syntax check button.
> 
> 
> Thanks,
> 
> Dhaval Rajpara
> 
>

[jira] [Created] (RANGER-4374) Getting page not found when wrong password is send in 'Old Password'

[Mugdha Varadkar (Jira)]

Mugdha Varadkar created RANGER-4374:
---

 Summary: Getting page not found when wrong password is send in 
'Old Password'
 Key: RANGER-4374
 URL: https://issues.apache.org/jira/browse/RANGER-4374
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Reporter: Mugdha Varadkar
Assignee: Mugdha Varadkar


Steps to reproduce :

1.Create a user 'testuser1' and password 'Testuser1'
2.Login with 'testuser1'
3.Go to profile and then change password
4.Give some wrong password in 'old password'
5.And give some new password
6.Click on save



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4368) Audit filter in Tag base service display wrong value for resources

[Mugdha Varadkar (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mugdha Varadkar updated RANGER-4368:

Labels: ranger-react  (was: )

> Audit filter in Tag base service display wrong value for resources
> --
>
> Key: RANGER-4368
> URL: https://issues.apache.org/jira/browse/RANGER-4368
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Dhaval Rajpara
>Assignee: Dhaval Rajpara
>Priority: Major
>  Labels: ranger-react
>
> Step
> 1) Click on Add service for tag base policy.
> 2) Click on add resources in the Audit filter.
> 3) Enter the value and save the resources.
> 4) That time Enter a value for that resource not displayed properly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-4119) [UI] Syntax check button missing in policy level condition

[Dhaval Rajpara (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757301#comment-17757301
 ] 

Dhaval Rajpara commented on RANGER-4119:


Hi [~madhan],

Regarding this, we added fixes only in react.js UI.
In the new react.js UI, we provided run time validation. we removed the Syntax 
check button.
The error is displayed when the user enters a value in the expression column.


Thanks,
Dhaval

> [UI] Syntax check button missing in policy level condition
> --
>
> Key: RANGER-4119
> URL: https://issues.apache.org/jira/browse/RANGER-4119
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Affects Versions: 2.4.0
>Reporter: Madhan Neethiraj
>Assignee: Dhaval Rajpara
>Priority: Major
>  Labels: ranger-react
> Attachments: image-2023-03-03-00-04-10-367.png, 
> image-2023-03-03-00-04-45-845.png
>
>
> Ranger policy UI provides {{Syntax check}} button that allows policy authors 
> to check if condition expression entered is valid or not. This button is 
> available for condition in policy-item level, but its not available for 
> condition at policy level - as seen in following images:
>  
> Policy-item level:
> !image-2023-03-03-00-04-45-845.png|width=661,height=453!
>  
> Policy level:
> !image-2023-03-03-00-04-10-367.png|width=661,height=488!
>  
> CC: [~ni3galave], [~Dhaval.Rajpara] 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74556: RANGER-4373: Deleting a role which is already present in policy is giving incorrect message.

[Ramachandran Krishnan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74556/#review225667
---




security-admin/src/main/java/org/apache/ranger/rest/RoleREST.java
Line 289 (original), 290 (patched)


Can you write a Unit case to cover this condition ?


- Ramachandran Krishnan


On Aug. 22, 2023, 7:14 a.m., sanket shelar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74556/
> ---
> 
> (Updated Aug. 22, 2023, 7:14 a.m.)
> 
> 
> Review request for ranger, dinesh  akhand, Kishor Gollapalliwar, Abhay 
> Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4373
> https://issues.apache.org/jira/browse/RANGER-4373
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> In case if a role is already present in policy and we try to delete the role 
> then we are getting message as "data not found" instead of "Role  can not be 
> deleted as it is referenced in one or more policies"
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/rest/RoleREST.java 4bfaa862c 
> 
> 
> Diff: https://reviews.apache.org/r/74556/diff/1/
> 
> 
> Testing
> ---
> 
> Tested for role delete scenarios.
> 
> 
> Thanks,
> 
> sanket shelar
> 
>

Review Request 74556: RANGER-4373: Deleting a role which is already present in policy is giving incorrect message.

[sanket shelar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74556/
---

Review request for ranger, dinesh  akhand, Kishor Gollapalliwar, Abhay 
Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, 
and Velmurugan Periasamy.


Bugs: RANGER-4373
https://issues.apache.org/jira/browse/RANGER-4373


Repository: ranger


Description
---

In case if a role is already present in policy and we try to delete the role 
then we are getting message as "data not found" instead of "Role  can not be 
deleted as it is referenced in one or more policies"


Diffs
-

  security-admin/src/main/java/org/apache/ranger/rest/RoleREST.java 4bfaa862c 


Diff: https://reviews.apache.org/r/74556/diff/1/


Testing
---

Tested for role delete scenarios.


Thanks,

sanket shelar

[jira] [Created] (RANGER-4373) Deleting a role which is already present in policy is giving incorrect message.

[Sanket Shelar (Jira)]

Sanket Shelar created RANGER-4373:
-

 Summary: Deleting a role which is already present in policy is 
giving incorrect message.
 Key: RANGER-4373
 URL: https://issues.apache.org/jira/browse/RANGER-4373
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Reporter: Sanket Shelar
Assignee: Sanket Shelar


In case if a role is already present in policy and we try to delete the role 
then we are getting message as "data not found" instead of "Role  can not be 
deleted as it is referenced in one or more policies"



--
This message was sent by Atlassian Jira
(v8.20.10#820010)