Re: Review Request 74136: RANGER-3685:hive 'show' sql produces excessive audit log

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74136/#review224773
---




hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
Lines 1132 (patched)


RANGER-3000 introduced audit-filtering feature, which enables to skip 
generation of audit logs for conditions specified in service-configuration. 
Default service-configuration for Hive was updated in RANGER-3214 (shown below) 
to skip auditing of METADATA OPERATIONS, like SHOW DATABASES. Note the 
following text in Hive service-def configuration: 

{'actions':['METADATA OPERATION'], 'isAudited': false}
 

{
  "itemId":6,
  "name": "ranger.plugin.audit.filters",
  "type": "string",
  "mandatory":false,
  "label":"Ranger Default Audit Filters",
  "defaultValue": "[ {'accessResult': 'DENIED', 'isAudited': true}, 
{'actions':['METADATA OPERATION'], 'isAudited': false}, 
{'users':['hive','hue'],'actions':['SHOW_ROLES'],'isAudited':false} ]"
}
 

Can you please verify the configuration of your Hive service (like 
dev_hive) if appropriate value is specified for "ranger.plugin.audit.filters"?


- Madhan Neethiraj


On Sept. 24, 2022, 2:37 p.m., Xuze Yang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74136/
> ---
> 
> (Updated Sept. 24, 2022, 2:37 p.m.)
> 
> 
> Review request for ranger, Kirby Zhou, Abhay Kulkarni, and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-3685
> https://issues.apache.org/jira/browse/RANGER-3685
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> hive show sql produces too many audit logs, which will flood valuable audit 
> logs and take up a lot of storage resources. This patch aims to provide a 
> configuration item to support simplified hive audit logging
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/authorization/hadoop/constants/RangerHadoopConstants.java
>  6675125e1 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
>  8f6801be1 
> 
> 
> Diff: https://reviews.apache.org/r/74136/diff/1/
> 
> 
> Testing
> ---
> 
> 1. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is not 
> set or set to false. The code behaves the same as before,i.e record audit 
> logs for each databases or tables
> 2. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is set 
> to true. It will only record an audit log about USE operation
> 
> 
> Thanks,
> 
> Xuze Yang
> 
>

Re: Review Request 74136: RANGER-3685:hive 'show' sql produces excessive audit log

[Kirby Zhou]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74136/#review224738
---


Ship it!




Ship It!

- Kirby Zhou


On 九月 24, 2022, 2:37 p.m., Xuze Yang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74136/
> ---
> 
> (Updated 九月 24, 2022, 2:37 p.m.)
> 
> 
> Review request for ranger, Kirby Zhou, Abhay Kulkarni, and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-3685
> https://issues.apache.org/jira/browse/RANGER-3685
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> hive show sql produces too many audit logs, which will flood valuable audit 
> logs and take up a lot of storage resources. This patch aims to provide a 
> configuration item to support simplified hive audit logging
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/authorization/hadoop/constants/RangerHadoopConstants.java
>  6675125e1 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
>  8f6801be1 
> 
> 
> Diff: https://reviews.apache.org/r/74136/diff/1/
> 
> 
> Testing
> ---
> 
> 1. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is not 
> set or set to false. The code behaves the same as before,i.e record audit 
> logs for each databases or tables
> 2. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is set 
> to true. It will only record an audit log about USE operation
> 
> 
> Thanks,
> 
> Xuze Yang
> 
>

Re: Review Request 74136: RANGER-3685:hive 'show' sql produces excessive audit log

[Xuze Yang]

> On 九月 28, 2022, 9:15 a.m., Kirby Zhou wrote:
> > Why modify filterListCmdObjects() only?

filterListCmdObjects() will filter out any objects that should not be shown to 
the user, from the list of tables or databases coming from a 'show tables' or 
'show databases' command. This patch gives an option to disable record audit 
logs for every tables or databases when execute 'show' sql. checkPrivileges() 
will record one audit log about 'show' sql(accetype is USE), and this audit log 
is valuable, we should keep it.  
I would like to know what else needs to be changed, can you give more hints? 
Thanks


- Xuze


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74136/#review224707
---


On 九月 24, 2022, 2:37 p.m., Xuze Yang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74136/
> ---
> 
> (Updated 九月 24, 2022, 2:37 p.m.)
> 
> 
> Review request for ranger, Kirby Zhou, Abhay Kulkarni, and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-3685
> https://issues.apache.org/jira/browse/RANGER-3685
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> hive show sql produces too many audit logs, which will flood valuable audit 
> logs and take up a lot of storage resources. This patch aims to provide a 
> configuration item to support simplified hive audit logging
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/authorization/hadoop/constants/RangerHadoopConstants.java
>  6675125e1 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
>  8f6801be1 
> 
> 
> Diff: https://reviews.apache.org/r/74136/diff/1/
> 
> 
> Testing
> ---
> 
> 1. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is not 
> set or set to false. The code behaves the same as before,i.e record audit 
> logs for each databases or tables
> 2. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is set 
> to true. It will only record an audit log about USE operation
> 
> 
> Thanks,
> 
> Xuze Yang
> 
>

Re: Review Request 74136: RANGER-3685:hive 'show' sql produces excessive audit log

[Kirby Zhou]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74136/#review224707
---



Why modify filterListCmdObjects() only?

- Kirby Zhou


On 九月 24, 2022, 2:37 p.m., Xuze Yang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74136/
> ---
> 
> (Updated 九月 24, 2022, 2:37 p.m.)
> 
> 
> Review request for ranger, Kirby Zhou, Abhay Kulkarni, and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-3685
> https://issues.apache.org/jira/browse/RANGER-3685
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> hive show sql produces too many audit logs, which will flood valuable audit 
> logs and take up a lot of storage resources. This patch aims to provide a 
> configuration item to support simplified hive audit logging
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/authorization/hadoop/constants/RangerHadoopConstants.java
>  6675125e1 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
>  8f6801be1 
> 
> 
> Diff: https://reviews.apache.org/r/74136/diff/1/
> 
> 
> Testing
> ---
> 
> 1. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is not 
> set or set to false. The code behaves the same as before,i.e record audit 
> logs for each databases or tables
> 2. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is set 
> to true. It will only record an audit log about USE operation
> 
> 
> Thanks,
> 
> Xuze Yang
> 
>

Review Request 74136: RANGER-3685:hive 'show' sql produces excessive audit log

[Xuze Yang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74136/
---

Review request for ranger, Kirby Zhou, Abhay Kulkarni, and Madhan Neethiraj.


Bugs: RANGER-3685
https://issues.apache.org/jira/browse/RANGER-3685


Repository: ranger


Description
---

hive show sql produces too many audit logs, which will flood valuable audit 
logs and take up a lot of storage resources. This patch aims to provide a 
configuration item to support simplified hive audit logging


Diffs
-

  
agents-common/src/main/java/org/apache/ranger/authorization/hadoop/constants/RangerHadoopConstants.java
 6675125e1 
  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
 8f6801be1 


Diff: https://reviews.apache.org/r/74136/diff/1/


Testing
---

1. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is not 
set or set to false. The code behaves the same as before,i.e record audit logs 
for each databases or tables
2. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is set to 
true. It will only record an audit log about USE operation


Thanks,

Xuze Yang