[GitHub] [ranger] luohaifang commented on pull request #40: Bump nimbus-jose-jwt from 4.41.2 to 7.9 in /security-admin
luohaifang commented on PR #40: URL: https://github.com/apache/ranger/pull/40#issuecomment-1257118683 > I run the fllowing cmd to compile the branch 2.0.0 "mvn clean compile package install assembly:assembly -Dmaven.test.skip=true -DskipJSTests" ，the module security-admin comes error security-admin\target\jsmain\node_modules.bin\r.js > > [ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.6:npm (npm run r.js) on project security-admin-web: Failed to run task: 'npm run r.js -- -o ../security-admin-web-2.0.0/minify.build.js' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1] > > 17 verbose argv "D:\bigdata\ranger-release-ranger-2.0.0\security-admin\target\node\node.exe" "D:\bigdata\ranger-release-ranger-2.0.0\security-admin\target\node\node_modules\npm\bin\npm-cli.js" "run" "r.js" "--" "-o" "../security-admin-web-2.0.0/minify.build.js" 18 verbose node v8.12.0 19 verbose npm v6.4.1 20 error code ELIFECYCLE 21 error errno 1 22 error security-admin@2.0.0 r.js: `r.js "-o" "../security-admin-web-2.0.0/minify.build.js"` 22 error Exit status 1 23 error Failed at the security-admin@2.0.0 r.js script. 23 error This is probably not a problem with npm. There is likely additional logging output above. 24 verbose exit [ 1, true ] > I run the fllowing cmd to compile the branch 2.0.0 "mvn clean compile package install assembly:assembly -Dmaven.test.skip=true -DskipJSTests" ，the module security-admin comes error security-admin\target\jsmain\node_modules.bin\r.js > > [ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.6:npm (npm run r.js) on project security-admin-web: Failed to run task: 'npm run r.js -- -o ../security-admin-web-2.0.0/minify.build.js' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1] > > 17 verbose argv "D:\bigdata\ranger-release-ranger-2.0.0\security-admin\target\node\node.exe" "D:\bigdata\ranger-release-ranger-2.0.0\security-admin\target\node\node_modules\npm\bin\npm-cli.js" "run" "r.js" "--" "-o" "../security-admin-web-2.0.0/minify.build.js" 18 verbose node v8.12.0 19 verbose npm v6.4.1 20 error code ELIFECYCLE 21 error errno 1 22 error security-admin@2.0.0 r.js: `r.js "-o" "../security-admin-web-2.0.0/minify.build.js"` 22 error Exit status 1 23 error Failed at the security-admin@2.0.0 r.js script. 23 error This is probably not a problem with npm. There is likely additional logging output above. 24 verbose exit [ 1, true ] hello，Have you solved the problem? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (RANGER-3685) hive 'show' sql produces excessive audit log
[ https://issues.apache.org/jira/browse/RANGER-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuze Yang reassigned RANGER-3685: - Assignee: Xuze Yang > hive 'show' sql produces excessive audit log > > > Key: RANGER-3685 > URL: https://issues.apache.org/jira/browse/RANGER-3685 > Project: Ranger > Issue Type: Improvement > Components: audit >Affects Versions: 2.1.0 >Reporter: Xuze Yang >Assignee: Xuze Yang >Priority: Major > Attachments: > RANGER-3685-hive-show-sql-produces-excessive-audit-l.patch > > > Since ranger2.1.0. For "show databases", user needs any permission on > Database to get authorized. RangerHiveAuthorizer.filterListCmdObjects() is > implemented to filter out the database which user don't have access to. > This is a good implementation, but a problem comes with it：the method will > record an audit log for each database(each table for "show tables"). In our > production environment, There are 80,000 tables under a database of hive. A > show tables operation will generate 80001(The extra one is the verification > of USE permissions) audit logs. Unfortunately, our customers will frequently > call the show tables operation. > This brings up two problems: > # Valuable audit logs are flooded > # Take up a lot of storage resources > For problem.2, such a scenario has occurred in our environment: our audit log > destination is down. All audit logs are spooled in disk files, several days > later, the size of the disk file exceeded 800G, causing other components to > fail to provide services normally. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3685) hive 'show' sql produces excessive audit log
[ https://issues.apache.org/jira/browse/RANGER-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17609008#comment-17609008 ] Xuze Yang commented on RANGER-3685: --- review request available at: https://reviews.apache.org/r/74136/ > hive 'show' sql produces excessive audit log > > > Key: RANGER-3685 > URL: https://issues.apache.org/jira/browse/RANGER-3685 > Project: Ranger > Issue Type: Improvement > Components: audit >Affects Versions: 2.1.0 >Reporter: Xuze Yang >Priority: Major > Attachments: > RANGER-3685-hive-show-sql-produces-excessive-audit-l.patch > > > Since ranger2.1.0. For "show databases", user needs any permission on > Database to get authorized. RangerHiveAuthorizer.filterListCmdObjects() is > implemented to filter out the database which user don't have access to. > This is a good implementation, but a problem comes with it：the method will > record an audit log for each database(each table for "show tables"). In our > production environment, There are 80,000 tables under a database of hive. A > show tables operation will generate 80001(The extra one is the verification > of USE permissions) audit logs. Unfortunately, our customers will frequently > call the show tables operation. > This brings up two problems: > # Valuable audit logs are flooded > # Take up a lot of storage resources > For problem.2, such a scenario has occurred in our environment: our audit log > destination is down. All audit logs are spooled in disk files, several days > later, the size of the disk file exceeded 800G, causing other components to > fail to provide services normally. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3685) hive 'show' sql produces excessive audit log
[ https://issues.apache.org/jira/browse/RANGER-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuze Yang updated RANGER-3685: -- Attachment: RANGER-3685-hive-show-sql-produces-excessive-audit-l.patch > hive 'show' sql produces excessive audit log > > > Key: RANGER-3685 > URL: https://issues.apache.org/jira/browse/RANGER-3685 > Project: Ranger > Issue Type: Improvement > Components: audit >Affects Versions: 2.1.0 >Reporter: Xuze Yang >Priority: Major > Attachments: > RANGER-3685-hive-show-sql-produces-excessive-audit-l.patch > > > Since ranger2.1.0. For "show databases", user needs any permission on > Database to get authorized. RangerHiveAuthorizer.filterListCmdObjects() is > implemented to filter out the database which user don't have access to. > This is a good implementation, but a problem comes with it：the method will > record an audit log for each database(each table for "show tables"). In our > production environment, There are 80,000 tables under a database of hive. A > show tables operation will generate 80001(The extra one is the verification > of USE permissions) audit logs. Unfortunately, our customers will frequently > call the show tables operation. > This brings up two problems: > # Valuable audit logs are flooded > # Take up a lot of storage resources > For problem.2, such a scenario has occurred in our environment: our audit log > destination is down. All audit logs are spooled in disk files, several days > later, the size of the disk file exceeded 800G, causing other components to > fail to provide services normally. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3685) hive 'show' sql produces excessive audit log
[ https://issues.apache.org/jira/browse/RANGER-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuze Yang updated RANGER-3685: -- Attachment: (was: submit patch.pdf) > hive 'show' sql produces excessive audit log > > > Key: RANGER-3685 > URL: https://issues.apache.org/jira/browse/RANGER-3685 > Project: Ranger > Issue Type: Improvement > Components: audit >Affects Versions: 2.1.0 >Reporter: Xuze Yang >Priority: Major > Attachments: > RANGER-3685-hive-show-sql-produces-excessive-audit-l.patch > > > Since ranger2.1.0. For "show databases", user needs any permission on > Database to get authorized. RangerHiveAuthorizer.filterListCmdObjects() is > implemented to filter out the database which user don't have access to. > This is a good implementation, but a problem comes with it：the method will > record an audit log for each database(each table for "show tables"). In our > production environment, There are 80,000 tables under a database of hive. A > show tables operation will generate 80001(The extra one is the verification > of USE permissions) audit logs. Unfortunately, our customers will frequently > call the show tables operation. > This brings up two problems: > # Valuable audit logs are flooded > # Take up a lot of storage resources > For problem.2, such a scenario has occurred in our environment: our audit log > destination is down. All audit logs are spooled in disk files, several days > later, the size of the disk file exceeded 800G, causing other components to > fail to provide services normally. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3685) hive 'show' sql produces excessive audit log
[ https://issues.apache.org/jira/browse/RANGER-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuze Yang updated RANGER-3685: -- Attachment: (was: 0001-1.-hive-show-sql.patch) > hive 'show' sql produces excessive audit log > > > Key: RANGER-3685 > URL: https://issues.apache.org/jira/browse/RANGER-3685 > Project: Ranger > Issue Type: Improvement > Components: audit >Affects Versions: 2.1.0 >Reporter: Xuze Yang >Priority: Major > Attachments: > RANGER-3685-hive-show-sql-produces-excessive-audit-l.patch > > > Since ranger2.1.0. For "show databases", user needs any permission on > Database to get authorized. RangerHiveAuthorizer.filterListCmdObjects() is > implemented to filter out the database which user don't have access to. > This is a good implementation, but a problem comes with it：the method will > record an audit log for each database(each table for "show tables"). In our > production environment, There are 80,000 tables under a database of hive. A > show tables operation will generate 80001(The extra one is the verification > of USE permissions) audit logs. Unfortunately, our customers will frequently > call the show tables operation. > This brings up two problems: > # Valuable audit logs are flooded > # Take up a lot of storage resources > For problem.2, such a scenario has occurred in our environment: our audit log > destination is down. All audit logs are spooled in disk files, several days > later, the size of the disk file exceeded 800G, causing other components to > fail to provide services normally. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3685) hive 'show' sql produces excessive audit log
[ https://issues.apache.org/jira/browse/RANGER-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17609007#comment-17609007 ] Xuze Yang commented on RANGER-3685: --- I uploaded a patch in which a configuration item "xasecure.hive.simplify.audit.of.hive.show.sql" was added to decide whether to simplify the audit log of hive show sql. When not specified, the default is false, that is, the hive audit log is not simplified, which is compatible with the previous behavior. Please find attached patch and review PR as per your convenience. Thanks. CC: [~madhan] [~kirbyzhou] [~kulkabhay] > hive 'show' sql produces excessive audit log > > > Key: RANGER-3685 > URL: https://issues.apache.org/jira/browse/RANGER-3685 > Project: Ranger > Issue Type: Improvement > Components: audit >Affects Versions: 2.1.0 >Reporter: Xuze Yang >Priority: Major > Attachments: 0001-1.-hive-show-sql.patch, submit patch.pdf > > > Since ranger2.1.0. For "show databases", user needs any permission on > Database to get authorized. RangerHiveAuthorizer.filterListCmdObjects() is > implemented to filter out the database which user don't have access to. > This is a good implementation, but a problem comes with it：the method will > record an audit log for each database(each table for "show tables"). In our > production environment, There are 80,000 tables under a database of hive. A > show tables operation will generate 80001(The extra one is the verification > of USE permissions) audit logs. Unfortunately, our customers will frequently > call the show tables operation. > This brings up two problems: > # Valuable audit logs are flooded > # Take up a lot of storage resources > For problem.2, such a scenario has occurred in our environment: our audit log > destination is down. All audit logs are spooled in disk files, several days > later, the size of the disk file exceeded 800G, causing other components to > fail to provide services normally. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Review Request 74136: RANGER-3685:hive 'show' sql produces excessive audit log
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74136/ --- Review request for ranger, Kirby Zhou, Abhay Kulkarni, and Madhan Neethiraj. Bugs: RANGER-3685 https://issues.apache.org/jira/browse/RANGER-3685 Repository: ranger Description --- hive show sql produces too many audit logs, which will flood valuable audit logs and take up a lot of storage resources. This patch aims to provide a configuration item to support simplified hive audit logging Diffs - agents-common/src/main/java/org/apache/ranger/authorization/hadoop/constants/RangerHadoopConstants.java 6675125e1 hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java 8f6801be1 Diff: https://reviews.apache.org/r/74136/diff/1/ Testing --- 1. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is not set or set to false. The code behaves the same as before,i.e record audit logs for each databases or tables 2. when configuration "xasecure.hive.simplify.audit.of.hive.show.sql" is set to true. It will only record an audit log about USE operation Thanks, Xuze Yang
[jira] [Updated] (RANGER-3685) hive 'show' sql produces excessive audit log
[ https://issues.apache.org/jira/browse/RANGER-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuze Yang updated RANGER-3685: -- Attachment: submit patch.pdf > hive 'show' sql produces excessive audit log > > > Key: RANGER-3685 > URL: https://issues.apache.org/jira/browse/RANGER-3685 > Project: Ranger > Issue Type: Improvement > Components: audit >Affects Versions: 2.1.0 >Reporter: Xuze Yang >Priority: Major > Attachments: 0001-1.-hive-show-sql.patch, submit patch.pdf > > > Since ranger2.1.0. For "show databases", user needs any permission on > Database to get authorized. RangerHiveAuthorizer.filterListCmdObjects() is > implemented to filter out the database which user don't have access to. > This is a good implementation, but a problem comes with it：the method will > record an audit log for each database(each table for "show tables"). In our > production environment, There are 80,000 tables under a database of hive. A > show tables operation will generate 80001(The extra one is the verification > of USE permissions) audit logs. Unfortunately, our customers will frequently > call the show tables operation. > This brings up two problems: > # Valuable audit logs are flooded > # Take up a lot of storage resources > For problem.2, such a scenario has occurred in our environment: our audit log > destination is down. All audit logs are spooled in disk files, several days > later, the size of the disk file exceeded 800G, causing other components to > fail to provide services normally. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3685) hive 'show' sql produces excessive audit log
[ https://issues.apache.org/jira/browse/RANGER-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuze Yang updated RANGER-3685: -- Attachment: 0001-1.-hive-show-sql.patch > hive 'show' sql produces excessive audit log > > > Key: RANGER-3685 > URL: https://issues.apache.org/jira/browse/RANGER-3685 > Project: Ranger > Issue Type: Improvement > Components: audit >Affects Versions: 2.1.0 >Reporter: Xuze Yang >Priority: Major > Attachments: 0001-1.-hive-show-sql.patch > > > Since ranger2.1.0. For "show databases", user needs any permission on > Database to get authorized. RangerHiveAuthorizer.filterListCmdObjects() is > implemented to filter out the database which user don't have access to. > This is a good implementation, but a problem comes with it：the method will > record an audit log for each database(each table for "show tables"). In our > production environment, There are 80,000 tables under a database of hive. A > show tables operation will generate 80001(The extra one is the verification > of USE permissions) audit logs. Unfortunately, our customers will frequently > call the show tables operation. > This brings up two problems: > # Valuable audit logs are flooded > # Take up a lot of storage resources > For problem.2, such a scenario has occurred in our environment: our audit log > destination is down. All audit logs are spooled in disk files, several days > later, the size of the disk file exceeded 800G, causing other components to > fail to provide services normally. -- This message was sent by Atlassian Jira (v8.20.10#820010)
