----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73756/#review223859 -----------------------------------------------------------
agents-audit/src/main/java/org/apache/ranger/audit/destination/AmazonCloudWatchAuditDestination.java Line 159 (original), 176 (patched) <https://reviews.apache.org/r/73756/#comment312896> DescribeLogStreams API supports pagination hence you may not get the stream by just calling it once. I saw you changed the log stream name to get rid of the UUID. There were two reasons behind it. 1/ We have a use case that one engine/service would generate audit events not only on master nodes but also on core nodes hence we prefer an unique stream for each node. 2/ By appending the UUID, we no longer need to pre-check if the log stream exists when ranger plugin gets restarted on the same machine. security-admin/scripts/install.properties Line 86 (original), 86 (patched) <https://reviews.apache.org/r/73756/#comment312897> nit: cloudwatch -> cloudwatch logs security-admin/scripts/install.properties Lines 117-119 (patched) <https://reviews.apache.org/r/73756/#comment312898> Prefer to not have any default settings given there's no default setup at plugin side. security-admin/src/main/java/org/apache/ranger/amazon/cloudwatch/CloudWatchMgr.java Lines 58 (patched) <https://reviews.apache.org/r/73756/#comment312901> Why do we want to create log stream here? Isn't that Ranger Admin Server only needs to query audit events? security-admin/src/main/java/org/apache/ranger/amazon/cloudwatch/CloudWatchUtil.java Lines 99 (patched) <https://reviews.apache.org/r/73756/#comment312899> nit: add a comment on this. security-admin/src/main/java/org/apache/ranger/amazon/cloudwatch/CloudWatchUtil.java Lines 137-141 (patched) <https://reviews.apache.org/r/73756/#comment312900> same comment: The DescribeLogStreams API supports pagination which needs to be handled. - Yao Zhou On Dec. 10, 2021, 8:08 p.m., Pradeep Agrawal wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73756/ > ----------------------------------------------------------- > > (Updated Dec. 10, 2021, 8:08 p.m.) > > > Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, > and Velmurugan Periasamy. > > > Bugs: RANGER-3540 > https://issues.apache.org/jira/browse/RANGER-3540 > > > Repository: ranger > > > Description > ------- > > **Problem Statement: ** This is related to RANGER-2967 which includes changes > only at the plugin end. Access audit logs should be accessible and appear at > Ranger admin UI end as well. > > **Proposed Solution: ** Proposed patch make use of AWS API's to read access > audit logs from cloudwatch loggroup. > > **Known issue:** Cloudwatch APIs does not provide sorting of recording in > descending order of timestamp, hence read operation will be slow. Hence its > recommended to use the filter to minimise the resultset which shall reduce > the response time and access audit page will load faster. > Due to this issue as of now maximum 10k records will be loaded at a time to > handle out of memory issue. > > > Diffs > ----- > > agents-audit/pom.xml 33fa256bb > > agents-audit/src/main/java/org/apache/ranger/audit/destination/AmazonCloudWatchAuditDestination.java > b236a2653 > agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java > f58b813f8 > hbase-agent/conf/ranger-hbase-audit-changes.cfg a6c7ffd41 > hbase-agent/scripts/install.properties 87a24819e > hdfs-agent/conf/ranger-hdfs-audit-changes.cfg 92d2a4b08 > hdfs-agent/scripts/install.properties 323b878cf > hive-agent/conf/ranger-hive-audit-changes.cfg 52c715ef5 > hive-agent/scripts/install.properties 3720b66c8 > kms/scripts/install.properties 6b6b66270 > knox-agent/conf/ranger-knox-audit-changes.cfg 52c715ef5 > knox-agent/scripts/install.properties 470400499 > plugin-atlas/conf/ranger-atlas-audit-changes.cfg 2d8251b5f > plugin-atlas/scripts/install.properties 3b777bd6a > plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg 52c715ef5 > plugin-elasticsearch/scripts/install.properties 4111afe3f > plugin-kafka/conf/ranger-kafka-audit-changes.cfg bc5a0890d > plugin-kafka/scripts/install.properties 1e325e0ec > plugin-kms/conf/ranger-kms-audit-changes.cfg e5e9ae489 > plugin-kylin/conf/ranger-kylin-audit-changes.cfg 52c715ef5 > plugin-kylin/scripts/install.properties 013433837 > plugin-ozone/conf/ranger-ozone-audit-changes.cfg 0eace6d29 > plugin-ozone/scripts/install.properties 1891d565f > plugin-presto/conf/ranger-presto-audit-changes.cfg bc5a0890d > plugin-presto/scripts/install.properties ce162a2bd > plugin-solr/conf/ranger-solr-audit-changes.cfg ffa0a7696 > plugin-solr/scripts/install.properties d1852e695 > plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg 52c715ef5 > plugin-sqoop/scripts/install.properties 81b4526a6 > plugin-yarn/conf/ranger-yarn-audit-changes.cfg 52c715ef5 > plugin-yarn/scripts/install.properties e73ab8b14 > pom.xml 6d3cafcf4 > security-admin/pom.xml 3e7a64f2a > security-admin/scripts/install.properties 5a8b00c13 > security-admin/scripts/ranger-admin-site-template.xml 72ff66eaf > security-admin/scripts/setup.sh c3f51a03a > security-admin/scripts/upgrade_admin.py 10fa485bd > security-admin/src/main/java/org/apache/ranger/AccessAuditsService.java > e902e65d0 > > security-admin/src/main/java/org/apache/ranger/amazon/cloudwatch/CloudWatchAccessAuditsService.java > PRE-CREATION > > security-admin/src/main/java/org/apache/ranger/amazon/cloudwatch/CloudWatchMgr.java > PRE-CREATION > > security-admin/src/main/java/org/apache/ranger/amazon/cloudwatch/CloudWatchUtil.java > PRE-CREATION > security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java d3ce25158 > security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java > 75ebae6f5 > security-admin/src/main/java/org/apache/ranger/biz/XAuditMgr.java 4e5410e8b > > security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchAccessAuditsService.java > 0b2e7df7f > > security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchUtil.java > 9bee640a5 > > security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java > 0aea46d1b > security-admin/src/main/java/org/apache/ranger/solr/SolrUtil.java 239698f3f > security-admin/src/main/resources/conf.dist/ranger-admin-site.xml d32a324ec > > security-admin/src/test/java/org/apache/ranger/audit/destination/AmazonCloudWatchAuditDestinationTest.java > dde8bb568 > storm-agent/conf/ranger-storm-audit-changes.cfg 52c715ef5 > storm-agent/scripts/install.properties d219abf59 > > > Diff: https://reviews.apache.org/r/73756/diff/1/ > > > Testing > ------- > > Tested by creating IAM user in AWS and provided required configuration in the > install.properties. > > **Note:** AWS region name, access key and secret key should be provided in > the environment. > > > Thanks, > > Pradeep Agrawal > >
