Re: Review Request 74974: RANGER-4789: Admin audits for security-zone are blank for new and old value, when compression is enabled
> On May 6, 2024, 10:54 p.m., Madhan Neethiraj wrote: > > > For the change only in security-zone resource name (resource count of the > > > zone is same), admin audit is not generated. > > Subhrat - instead of skipping admin audit in this scenario, I suggest audit > > to indicate that resources have been updated in services - with text like > > '{ "dev_hdfs": "resources updated", "dev_hbase": "resources updated" } > > > > if (oldValue == null || oldValue.equalsIgnoreCase(value)) { // existing > > line #357 > > Map resourceUpdateSummary = > > getResourceUpdateSummary(securityZoneDB.getServices(), > > vSecurityZone.getServices()); > > > > if (MapUtils.isNotEmpty(resourceUpdateSummary)) { > > oldValue = ""; > > value= new Gson().toJson(resourceUpdateSummary, Map.class); > > } else { > > continue; > > } > > } else { > > continue; // existing line #358 > > } // existing line #359 Json of Map does not work, since UI is expecting json of Map>. Please review the changes. One drawback with approach is - if there are mutiple services in the zone and resource name for only one service is updated, audit for all the service will be updated as -- service_name : resources updated. Please refer the image attached. Proper solution for this would be to implement getResourceUpdateSummary(securityZoneDB.getServices(), vSecurityZone.getServices()), iterating thorugh each service, comparing resource json with older version, updating for the ones for which resource have changed and return json of Map>. I tried working on this approach, but changes were getting too complex and extensive for this case. Please suggest. - Subhrat --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74974/#review226415 --- On May 9, 2024, 10:09 p.m., Subhrat Chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74974/ > --- > > (Updated May 9, 2024, 10:09 p.m.) > > > Review request for ranger, Anand Nadar, Asit Vadhavkar, Madhan Neethiraj, > Monika Kachhadiya, and Siddhesh Phatak. > > > Bugs: RANGER-4789 > https://issues.apache.org/jira/browse/RANGER-4789 > > > Repository: ranger > > > Description > --- > > In security-zone when resource name is updated, admin audit is generated for > same, with details about old and new value. > > When the json data compression is enabled in the security-zone with the > property: > > ranger.admin.store.security.zone.compress.json_data > > the old and new value in the generated admin audit is blank, when only the > resource name is changed. The reason for this is, if compression is enabled, > only the resource count is added in the new and old values. Hence if the > resource count does not change, change details in the admin audit is blank. > > In the code flow to update security-zone, when no change is noticed in the > new and old values, a dummy admin audit is being added with null for old and > new values. In this fix, removing the that code block. > > > Diffs > - > > > security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneServiceService.java > a4b7616e1 > > > Diff: https://reviews.apache.org/r/74974/diff/2/ > > > Testing > --- > > Validations done: > 1. For the change only in security-zone resource name (resource count of the > zone is same), admin audit is not generated. > 2. For above case x_service_version_info.policy_version is incremented (same > as existing behavior). > 3. If a resource is added or removed from the security-zone, admin audit is > generated for same. > 4. All the existing Junits are passing > > > File Attachments > > > audit.png > > https://reviews.apache.org/media/uploaded/files/2024/05/09/e0b4debc-b20e-41a2-a635-1bf766d87efd__audit.png > > > Thanks, > > Subhrat Chaudhary > >
Re: Review Request 74974: RANGER-4789: Admin audits for security-zone are blank for new and old value, when compression is enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74974/ --- (Updated May 9, 2024, 10:09 p.m.) Review request for ranger, Anand Nadar, Asit Vadhavkar, Madhan Neethiraj, Monika Kachhadiya, and Siddhesh Phatak. Bugs: RANGER-4789 https://issues.apache.org/jira/browse/RANGER-4789 Repository: ranger Description --- In security-zone when resource name is updated, admin audit is generated for same, with details about old and new value. When the json data compression is enabled in the security-zone with the property: ranger.admin.store.security.zone.compress.json_data the old and new value in the generated admin audit is blank, when only the resource name is changed. The reason for this is, if compression is enabled, only the resource count is added in the new and old values. Hence if the resource count does not change, change details in the admin audit is blank. In the code flow to update security-zone, when no change is noticed in the new and old values, a dummy admin audit is being added with null for old and new values. In this fix, removing the that code block. Diffs (updated) - security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneServiceService.java a4b7616e1 Diff: https://reviews.apache.org/r/74974/diff/2/ Changes: https://reviews.apache.org/r/74974/diff/1-2/ Testing --- Validations done: 1. For the change only in security-zone resource name (resource count of the zone is same), admin audit is not generated. 2. For above case x_service_version_info.policy_version is incremented (same as existing behavior). 3. If a resource is added or removed from the security-zone, admin audit is generated for same. 4. All the existing Junits are passing File Attachments (updated) audit.png https://reviews.apache.org/media/uploaded/files/2024/05/09/e0b4debc-b20e-41a2-a635-1bf766d87efd__audit.png Thanks, Subhrat Chaudhary
Re: Review Request 74926: RANGER-4076: Support Java 17 for build and runtime
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74926/ --- (Updated May 9, 2024, 3:03 p.m.) Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, sanket shelar, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-4076 https://issues.apache.org/jira/browse/RANGER-4076 Repository: ranger Description --- Currently only Java 8 and 11 are supported. Java 17 is a major LTS version of Java and adding support would modernize our Java version support. This patch enables manual and Docker-based build of Apache Ranger. It ensures compatibility with Java 8, Java 11, and Java 17 for both build and runtime environments. Diffs - agents-common/pom.xml 12e093f78 agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerScriptConditionEvaluator.java 6eb192270 agents-common/src/main/java/org/apache/ranger/plugin/util/GraalScriptEngineCreator.java 512d8d3ca agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEngineCreator.java 4a0081579 agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java db620df92 agents-common/src/main/java/org/apache/ranger/plugin/util/ScriptEngineUtil.java 8d76c1d81 agents-common/src/test/java/org/apache/ranger/plugin/conditionevaluator/RangerRequestScriptEvaluatorTest.java 0059bef88 dev-support/ranger-docker/.env 91613e5ec dev-support/ranger-docker/Dockerfile.ranger-base 72a850482 dev-support/ranger-docker/Dockerfile.ranger-build 9a192f152 dev-support/ranger-docker/docker-compose.ranger-build.yml c8760aab5 dev-support/ranger-docker/scripts/ranger-hadoop-setup.sh 10f04acd9 dev-support/ranger-docker/scripts/ranger-hbase-setup.sh 95a1bdf21 distro/src/main/assembly/admin-web.xml 245d9ca09 distro/src/main/assembly/hbase-agent.xml ffb1de0c5 distro/src/main/assembly/hdfs-agent.xml 349badfd2 distro/src/main/assembly/hive-agent.xml 5bae92ac5 distro/src/main/assembly/knox-agent.xml 3fa867150 distro/src/main/assembly/plugin-atlas.xml c389282fc distro/src/main/assembly/plugin-kafka.xml 4ffbdd611 distro/src/main/assembly/plugin-kms.xml 7c6a16330 distro/src/main/assembly/plugin-kylin.xml 9aefa49d4 distro/src/main/assembly/plugin-ozone.xml 294e5b9ef distro/src/main/assembly/plugin-presto.xml d50edf2d9 distro/src/main/assembly/plugin-solr.xml bcb24c0fc distro/src/main/assembly/plugin-sqoop.xml adc2a9c26 distro/src/main/assembly/plugin-yarn.xml 5db884710 distro/src/main/assembly/storm-agent.xml a334e247c docs/src/site/resources/index.js bb876f28d hdfs-agent/pom.xml dece8f46f kms/pom.xml bfac6424a kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java b6fc32950 knox-agent/pom.xml 4387efb5c plugin-nestedstructure/pom.xml 0e208f99c plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java 77767767c plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java 9cb161b8d pom.xml efd152040 security-admin/pom.xml fc59287d0 Diff: https://reviews.apache.org/r/74926/diff/4/ Testing (updated) --- This patch has been successfully tested and verified with Java versions 8, 11, and 17. The verification process included both manual and Docker setup. Now we are able to build and run on same jdk. ### JDK 8 maven build successfully mvn clean compile package install ### JDK 11 maven build successfully mvn clean compile package install ### JDK 17 maven build successfully mvn clean compile package install -Pranger-all-modules-jdk17 ### Validated policy enforcement for below scenario : ## Scenario 1: Successfully validated policy enforcement(policy-condition) for HDFS, HBase and Hive services using Ranger with JDK 8 for both build and runtime environments. ## Scenario 2: Successfully validated policy enforcement(policy-condition) for HDFS, HBase services using Ranger with JDK 17 for both build and runtime environments. Thanks, Rakesh Gupta
Re: Review Request 74926: RANGER-4076: Support Java 17 for build and runtime
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74926/ --- (Updated May 9, 2024, 3:02 p.m.) Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, sanket shelar, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-4076 https://issues.apache.org/jira/browse/RANGER-4076 Repository: ranger Description --- Currently only Java 8 and 11 are supported. Java 17 is a major LTS version of Java and adding support would modernize our Java version support. This patch enables manual and Docker-based build of Apache Ranger. It ensures compatibility with Java 8, Java 11, and Java 17 for both build and runtime environments. Diffs (updated) - agents-common/pom.xml 12e093f78 agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerScriptConditionEvaluator.java 6eb192270 agents-common/src/main/java/org/apache/ranger/plugin/util/GraalScriptEngineCreator.java 512d8d3ca agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEngineCreator.java 4a0081579 agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java db620df92 agents-common/src/main/java/org/apache/ranger/plugin/util/ScriptEngineUtil.java 8d76c1d81 agents-common/src/test/java/org/apache/ranger/plugin/conditionevaluator/RangerRequestScriptEvaluatorTest.java 0059bef88 dev-support/ranger-docker/.env 91613e5ec dev-support/ranger-docker/Dockerfile.ranger-base 72a850482 dev-support/ranger-docker/Dockerfile.ranger-build 9a192f152 dev-support/ranger-docker/docker-compose.ranger-build.yml c8760aab5 dev-support/ranger-docker/scripts/ranger-hadoop-setup.sh 10f04acd9 dev-support/ranger-docker/scripts/ranger-hbase-setup.sh 95a1bdf21 distro/src/main/assembly/admin-web.xml 245d9ca09 distro/src/main/assembly/hbase-agent.xml ffb1de0c5 distro/src/main/assembly/hdfs-agent.xml 349badfd2 distro/src/main/assembly/hive-agent.xml 5bae92ac5 distro/src/main/assembly/knox-agent.xml 3fa867150 distro/src/main/assembly/plugin-atlas.xml c389282fc distro/src/main/assembly/plugin-kafka.xml 4ffbdd611 distro/src/main/assembly/plugin-kms.xml 7c6a16330 distro/src/main/assembly/plugin-kylin.xml 9aefa49d4 distro/src/main/assembly/plugin-ozone.xml 294e5b9ef distro/src/main/assembly/plugin-presto.xml d50edf2d9 distro/src/main/assembly/plugin-solr.xml bcb24c0fc distro/src/main/assembly/plugin-sqoop.xml adc2a9c26 distro/src/main/assembly/plugin-yarn.xml 5db884710 distro/src/main/assembly/storm-agent.xml a334e247c docs/src/site/resources/index.js bb876f28d hdfs-agent/pom.xml dece8f46f kms/pom.xml bfac6424a kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java b6fc32950 knox-agent/pom.xml 4387efb5c plugin-nestedstructure/pom.xml 0e208f99c plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java 77767767c plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java 9cb161b8d pom.xml efd152040 security-admin/pom.xml fc59287d0 Diff: https://reviews.apache.org/r/74926/diff/4/ Changes: https://reviews.apache.org/r/74926/diff/3-4/ Testing (updated) --- This patch has been successfully tested and verified with Java versions 8, 11, and 17. The verification process included both manual and Docker setup. Now we are able to build and run on same jdk. ### JDK 8 maven build successfully mvn clean compile package install ### JDK 11 maven build successfully mvn clean compile package install ### JDK 17 maven build successfully mvn clean compile package install -Pranger-all-modules-jdk17 Validated policy enforcement for below scenario : ## Scenario 1: Successfully validated policy enforcement(policy-condition) for HDFS, HBase and Hive services using Ranger with JDK 8 for both build and runtime environments. ## Scenario 2: Successfully validated policy enforcement(policy-condition) for HDFS, HBase services using Ranger with JDK 17 for both build and runtime environments. Thanks, Rakesh Gupta
Re: Review Request 74926: RANGER-4076: Support Java 17 for build and runtime
> On April 23, 2024, 3:29 p.m., bhavik patel wrote: > > Have you validated the policy enforcement for any plugin? Validated policy enforcement for below plugin : Scenario 1: Successfully validated policy enforcement(policy-condition) for HDFS, HBase, and Hive services using Ranger with JDK 8 for both build and runtime environments. Scenario 2: Successfully validated policy enforcement(policy-condition) for HDFS, HBase services using Ranger with JDK 17 for both build and runtime environments. - Rakesh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74926/#review226395 --- On April 23, 2024, 12:32 p.m., Rakesh Gupta wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74926/ > --- > > (Updated April 23, 2024, 12:32 p.m.) > > > Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, > sanket shelar, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-4076 > https://issues.apache.org/jira/browse/RANGER-4076 > > > Repository: ranger > > > Description > --- > > Currently only Java 8 and 11 are supported. Java 17 is a major LTS version > of Java and adding support would modernize our Java version support. > > This patch enables manual and Docker-based build of Apache Ranger. It ensures > compatibility with Java 8, Java 11, and Java 17 for both build and runtime > environments. > > > Diffs > - > > agents-common/pom.xml 12e093f78 > > agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerScriptConditionEvaluator.java > 6eb192270 > > agents-common/src/main/java/org/apache/ranger/plugin/util/GraalScriptEngineCreator.java > 512d8d3ca > > agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEngineCreator.java > 4a0081579 > > agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java > db620df92 > > agents-common/src/main/java/org/apache/ranger/plugin/util/ScriptEngineUtil.java > 8d76c1d81 > > agents-common/src/test/java/org/apache/ranger/plugin/conditionevaluator/RangerRequestScriptEvaluatorTest.java > 0059bef88 > dev-support/ranger-docker/.env 91613e5ec > dev-support/ranger-docker/Dockerfile.ranger-build 9a192f152 > dev-support/ranger-docker/docker-compose.ranger-build.yml c8760aab5 > distro/src/main/assembly/admin-web.xml 245d9ca09 > distro/src/main/assembly/hbase-agent.xml ffb1de0c5 > distro/src/main/assembly/hdfs-agent.xml 349badfd2 > distro/src/main/assembly/hive-agent.xml 5bae92ac5 > distro/src/main/assembly/knox-agent.xml 3fa867150 > distro/src/main/assembly/plugin-atlas.xml c389282fc > distro/src/main/assembly/plugin-kafka.xml 4ffbdd611 > distro/src/main/assembly/plugin-kms.xml 7c6a16330 > distro/src/main/assembly/plugin-kylin.xml 9aefa49d4 > distro/src/main/assembly/plugin-ozone.xml 294e5b9ef > distro/src/main/assembly/plugin-presto.xml d50edf2d9 > distro/src/main/assembly/plugin-solr.xml bcb24c0fc > distro/src/main/assembly/plugin-sqoop.xml adc2a9c26 > distro/src/main/assembly/plugin-yarn.xml 5db884710 > distro/src/main/assembly/storm-agent.xml a334e247c > docs/src/site/resources/index.js bb876f28d > hdfs-agent/pom.xml dece8f46f > kms/pom.xml 0b37ce52d > kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java > b6fc32950 > knox-agent/pom.xml 4387efb5c > plugin-nestedstructure/pom.xml 0e208f99c > > plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java > 77767767c > > plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java > 9cb161b8d > pom.xml efd152040 > security-admin/pom.xml fc59287d0 > > > Diff: https://reviews.apache.org/r/74926/diff/3/ > > > Testing > --- > > This patch has been successfully tested and verified with Java versions 8, > 11, and 17. The verification process included both manual and Docker setup. > Now we are able to build and run on same jdk. > > ### JDK 8 maven build successfully > mvn clean compile package install > > ### JDK 11 maven build successfully > mvn clean compile package install > > ### JDK 17 maven build successfully > mvn clean compile package install -Pranger-all-modules-jdk17 > > > Thanks, > > Rakesh Gupta > >