-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75238/
-----------------------------------------------------------
(Updated Nov. 19, 2024, 9:35 a.m.)
Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay
Kulkarni, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, sanket
shelar, Sailaja Polavarapu, and Velmurugan Periasamy.
Bugs: RANGER-4692
https://issues.apache.org/jira/browse/RANGER-4692
Repository: ranger
Description
-------
Currently there are different type like (Policy, Role, Tag, UserStore and GDS)
on plugin status.
when we have plugins(large enough to accommodate in single page), It will be
helpful to identify any out-of-sync plugins based on download times, sorting
them by policyDownloadTime, tagDownloadTime, roleDownloadTime,
userstoreDownloadTime and gdsDownloadTime.
Diffs
-----
agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPluginInfo.java
8d46b99f7
security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 5fb0d95db
security-admin/db/mysql/patches/076-add-columns-in-x_plugin_info.sql
PRE-CREATION
security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql
593cffe56
security-admin/db/oracle/patches/076-add-columns-in-x_plugin_info.sql
PRE-CREATION
security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
16263a2d7
security-admin/db/postgres/patches/076-add-columns-in-x_plugin_info.sql
PRE-CREATION
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
fa9832314
security-admin/db/sqlanywhere/patches/076-add-columns-in-x_plugin_info.sql
PRE-CREATION
security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql
a0e47004b
security-admin/db/sqlserver/patches/076-add-columns-in-x_plugin_info.sql
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java 620d1e619
security-admin/src/main/java/org/apache/ranger/entity/XXPluginInfo.java
a15d8d056
security-admin/src/main/java/org/apache/ranger/entity/view/VXXPluginInfo.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/patch/PatchForPluginStatusEventSorting_J10064.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/service/RangerPluginInfoService.java
9dc26ceb7
Diff: https://reviews.apache.org/r/75238/diff/5/
Testing (updated)
-------
Tested and verified that the plugins are correctly sorted by event type
(Policy, Role, Tag, UserStore, GDS) using the following APIs:
/service/plugins/info?sortType=desc&sortBy=policyDownloadTime
/service/plugins/info?sortType=desc&sortBy=policyActivationTime
/service/plugins/info?sortType=desc&sortBy=lastPolicyUpdateTime
/service/plugins/info?sortType=desc&sortBy=tagDownloadTime
/service/plugins/info?sortType=desc&sortBy=tagActivationTime
/service/plugins/info?sortType=desc&sortBy=lasttagUpdateTime
/service/plugins/info?sortType=desc&sortBy=gdsDownloadTime
/service/plugins/info?sortType=desc&sortBy=gdsActivationTime
/service/plugins/info?sortType=desc&sortBy=lastgdsUpdateTime
/service/plugins/info?sortType=desc&sortBy=roleDownloadTime
/service/plugins/info?sortType=desc&sortBy=roleActivationTime
/service/plugins/info?sortType=desc&sortBy=lastroleUpdateTime
/service/plugins/info?sortType=desc&sortBy=userstoreDownloadTime
/service/plugins/info?sortType=desc&sortBy=userstoreActivationTime
/service/plugins/info?sortType=desc&sortBy=clusterName
/service/plugins/info?sortType=desc&sortBy=serviceType
Also tested filtering by:
/service/plugins/info?serviceType={serviceTypeName}
/service/plugins/info?clusterName={clusterName}
Tested Java patch for upgrade scenario: completed processing 1,00,009 records
in 21 seconds.
Here’s are contents from ranger_db_patch.log:
2024-11-19 12:19:25,941 [main] INFO Found 100009 plugin info records to
process.
2024-11-19 12:19:26,537 [Thread-10] INFO PROGRESS: 1000 plugin status records
processed.
2024-11-19 12:19:26,890 [Thread-12] INFO PROGRESS: 2000 plugin status records
processed.
2024-11-19 12:19:27,222 [Thread-11] INFO PROGRESS: 3000 plugin status records
processed.
...
2024-11-19 12:19:46,647 [Thread-11] INFO PROGRESS: 99000 plugin status records
processed.
2024-11-19 12:19:46,808 [Thread-10] INFO PROGRESS: 100000 plugin status
records processed.
2024-11-19 12:19:46,813 [main] INFO Loading completed!!!. Time
taken=21secs, 660ms
2024-11-19 12:19:46,813 [Loader Monitor] INFO Monitor Thread exiting!!!
2024-11-19 12:19:46,902 [main] INFO Load complete. Exiting!!!
Performance Results:
API Endpoint: /service/plugins/plugins/info
Pre-Patch:
Response Time for pageSize=10,000: 1 minute 59.28 seconds
Post-Patch:
pageSize=10,000: 590 ms.
pageSize=50,000: 1420 ms.
pageSize=1,00,000: 2.73 s
Thanks,
Rakesh Gupta
