Re: Review Request 57837: Remember filters on all tabs of Ranger Audits page
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57837/ --- (Updated April 19, 2017, 6:45 a.m.) Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-1471 https://issues.apache.org/jira/browse/RANGER-1471 Repository: ranger Description --- Currently, when we apply filter for anything in Ranger Audit page for any of the tabs. It resets the filter on change of tab or if we move to any other page in Ranger. Planning to add feature of remembering latest filters on all Tabs of Audits page. That will help users to stay focused on what they are looking for in audits tab and users will not have to apply for filters again and again to check audit events of a particular service. Diffs (updated) - security-admin/src/main/java/org/apache/ranger/common/SearchUtil.java fe253ef security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java f0d2401 security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java 0776021 security-admin/src/main/java/org/apache/ranger/solr/SolrUtil.java 049985c security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 8f7d5d9 security-admin/src/main/webapp/scripts/utils/XAUtils.js 480c515 security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js 8a0abb8 security-admin/src/main/webapp/scripts/views/reports/LoginSessionDetail.js 6f1069d security-admin/src/main/webapp/styles/xa.css 7a5ec2e security-admin/src/main/webapp/templates/common/ServiceManagerLayout_tmpl.html ea2f198 security-admin/src/main/webapp/templates/reports/AuditLayout_tmpl.html 028fdbf security-admin/src/main/webapp/templates/reports/LoginSessionDetail_tmpl.html ddd6e3d security-admin/src/test/java/org/apache/ranger/rest/TestXUserREST.java c544832 Diff: https://reviews.apache.org/r/57837/diff/3/ Changes: https://reviews.apache.org/r/57837/diff/2-3/ Testing --- 1. Tested multiple search is working correctly in "Audit" tab. 2. Tested search remains the same when we navigate from one tab to other. 3. Tested search is working correctly for different user role. Thanks, bhavik patel
[jira] [Updated] (RANGER-1471) Remember filters on all tabs of Ranger Audits page
[ https://issues.apache.org/jira/browse/RANGER-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bhavik patel updated RANGER-1471: - Attachment: RANGER-1471-2.patch > Remember filters on all tabs of Ranger Audits page > -- > > Key: RANGER-1471 > URL: https://issues.apache.org/jira/browse/RANGER-1471 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 0.7.1 >Reporter: bhavik patel >Assignee: bhavik patel > Fix For: 0.7.1 > > Attachments: RANGER-1471-1.patch, RANGER-1471-2.patch, > RANGER-1471.patch > > > Currently, when we apply filter for anything in Ranger Audit page for any of > the tabs. It resets the filter on change of tab or if we move to any other > page in Ranger. > Planning to add feature of remembering latest filters on all Tabs of Audits > page. That will help users to stay focused on what they are looking for in > audits tab and users will not have to apply for filters again and again to > check audit events of a particular service. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Review Request 58229: RANGER-1481 : Capture cluster name in ranger audit info
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58229/#review172319 --- Ship it! Ship It! - Ankita Sinha On April 6, 2017, 7:16 a.m., bhavik patel wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58229/ > --- > > (Updated April 6, 2017, 7:16 a.m.) > > > Review request for ranger, Ankita Sinha, Gautam Borad, Abhay Kulkarni, Madhan > Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, > and Velmurugan Periasamy. > > > Bugs: RANGER-1481 > https://issues.apache.org/jira/browse/RANGER-1481 > > > Repository: ranger > > > Description > --- > > In order to support Ranger authorization from multiple clusters, it will be > useful to capture details of Ambari cluster name, Ranger needs to make > provision to capture that info to be shown in Audit Access logs. > This will be helpful when centralized Ranger is used to authorize hadoop > components across multiple clusters setup by Ambari. > > > Diffs > - > > > agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java > e689e5d > > agents-audit/src/main/java/org/apache/ranger/audit/model/AuthzAuditEvent.java > b547c43 > > agents-audit/src/main/java/org/apache/ranger/audit/provider/solr/SolrAuditProvider.java > 22aebb5 > > agents-common/src/main/java/org/apache/ranger/admin/client/RangerAdminRESTClient.java > dec649d > > agents-common/src/main/java/org/apache/ranger/plugin/audit/RangerDefaultAuditHandler.java > cee46a3 > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequest.java > 0668d57 > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequestImpl.java > 15e872a > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequestReadOnly.java > a18e8bc > > agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java > acf8d15 > > agents-common/src/main/java/org/apache/ranger/plugin/util/GrantRevokeRequest.java > 33f1dd4 > > agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTUtils.java > 609f717 > > hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/AuthorizationSession.java > dedbe1e > > hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java > 8ee3580 > > hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java > 460c692 > > hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAccessRequest.java > b9f1cde > > hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java > 2baa97b > > knox-agent/src/main/java/org/apache/ranger/admin/client/RangerAdminJersey2RESTClient.java > fb92616 > > knox-agent/src/main/java/org/apache/ranger/authorization/knox/KnoxRangerPlugin.java > 61604b0 > > knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java > 55ebf58 > > plugin-atlas/src/main/java/org/apache/ranger/authorization/atlas/authorizer/RangerAtlasAuthorizer.java > 2038645 > > plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java > 472b734 > > plugin-kms/src/main/java/org/apache/ranger/authorization/kms/authorizer/RangerKmsAuthorizer.java > 9bebafa > > plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java > 2338ba1 > security-admin/db/mysql/patches/026-add-column-in-x_policy_export_audit.sql > PRE-CREATION > > security-admin/db/oracle/patches/026-add-column-in-x_policy_export_audit.sql > PRE-CREATION > > security-admin/db/postgres/patches/026-add-column-in-x_policy_export_audit.sql > PRE-CREATION > > security-admin/db/sqlanywhere/patches/026-add-column-in-x_policy_export_audit.sql > PRE-CREATION > > security-admin/db/sqlserver/patches/026-add-column-in-x_policy_export_audit.sql > PRE-CREATION > > security-admin/src/main/java/org/apache/ranger/entity/XXPolicyExportAudit.java > 4544614 > security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java f0d2401 > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > b9f1832 > > security-admin/src/main/java/org/apache/ranger/service/XAccessAuditService.java > e8fff6a > > security-admin/src/main/java/org/apache/ranger/service/XPolicyExportAuditService.java > 870e45d > > security-admin/src/main/java/org/apache/ranger/service/XPolicyExportAuditServiceBase.java > e83d70a > > security-admin/src/main/java/org/apache/ranger/solr/SolrAcces
Re: Review Request 58476: RANGER-1520:Some codes do not follow the python language development rules in usersync. They are messy.
> On April 18, 2017, 8:58 a.m., Colm O hEigeartaigh wrote: > > You could change "proprty" to "property" (3 times) Ok. Modified and updated the patch. Thanks. - Qiang --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58476/#review172175 --- On April 19, 2017, 5:07 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58476/ > --- > > (Updated April 19, 2017, 5:07 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, and Velmurugan Periasamy. > > > Bugs: RANGER-1520 > https://issues.apache.org/jira/browse/RANGER-1520 > > > Repository: ranger > > > Description > --- > > Python is the language that uses indent control code.The indentation of the > rows should be consistent. Some codes do not follow the rule in usersync. > They are messy. > > > Diffs > - > > unixauthservice/scripts/setup.py d7872ea > > > Diff: https://reviews.apache.org/r/58476/diff/2/ > > > Testing > --- > > > Thanks, > > Qiang Zhang > >
Re: Review Request 58476: RANGER-1520:Some codes do not follow the python language development rules in usersync. They are messy.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58476/ --- (Updated April 19, 2017, 5:07 a.m.) Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy. Bugs: RANGER-1520 https://issues.apache.org/jira/browse/RANGER-1520 Repository: ranger Description --- Python is the language that uses indent control code.The indentation of the rows should be consistent. Some codes do not follow the rule in usersync. They are messy. Diffs (updated) - unixauthservice/scripts/setup.py d7872ea Diff: https://reviews.apache.org/r/58476/diff/2/ Changes: https://reviews.apache.org/r/58476/diff/1-2/ Testing --- Thanks, Qiang Zhang
[jira] [Resolved] (RANGER-1518) Do some code improvement for the error message in PolicyMgrUserGroupBuilder.java
[
https://issues.apache.org/jira/browse/RANGER-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang Zhang resolved RANGER-1518.
-
Resolution: Fixed
> Do some code improvement for the error message in
> PolicyMgrUserGroupBuilder.java
>
>
> Key: RANGER-1518
> URL: https://issues.apache.org/jira/browse/RANGER-1518
> Project: Ranger
> Issue Type: Improvement
> Components: admin
>Affects Versions: 1.0.0
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>Priority: Trivial
> Fix For: 1.0.0
>
> Attachments: 0001-Ranger-1518-Do-some-code-inprovement.patch
>
>
> {code}
> if (addUserGroupInfo(ugInfo) == null) {
> String msg = "Failed to
> add add user group info";
> LOG.error(msg);
> throw new
> Exception(msg);
> }
> {code}
> Duplicate 'add' for this error message.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (RANGER-1519) Error occurred after execute enable-hive-plugin.sh
[ https://issues.apache.org/jira/browse/RANGER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Zhang resolved RANGER-1519. - Resolution: Fixed Fix Version/s: 1.0.0 > Error occurred after execute enable-hive-plugin.sh > -- > > Key: RANGER-1519 > URL: https://issues.apache.org/jira/browse/RANGER-1519 > Project: Ranger > Issue Type: Bug > Components: plugins >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1519-Error-occurred-after-execute-enable-hive.patch > > > [root@zdh41 ranger-1.0.0-SNAPSHOT-hive-plugin]# ./enable-hive-plugin.sh > Custom user and group is available, using custom user and group. > ERROR: Unable to find the conf directory of component [hive]; dir > [/home/xiehh/rangerplugin/hive/conf] not found. > Exiting installation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (RANGER-1508) The browser returns garbled when we login ranger security admin in non-English environment. We need support the internationalization function to solve this problem.
[ https://issues.apache.org/jira/browse/RANGER-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Zhang resolved RANGER-1508. - Resolution: Fixed Fix Version/s: 1.0.0 > The browser returns garbled when we login ranger security admin in > non-English environment. We need support the internationalization function to > solve this problem. > > > Key: RANGER-1508 > URL: https://issues.apache.org/jira/browse/RANGER-1508 > Project: Ranger > Issue Type: New Feature > Components: admin >Affects Versions: 1.0.0 >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Critical > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1508-support-internationalization.patch > > > Use the browser to log in Ranger admin in Chinese operation system. The > browser returns garbled when an error occurs. The error reason is that the > Ranger admin used hard code to compare error information between Chinese > character and English character. So we need support the internationalization > function to solve this problem. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (RANGER-1517) In RangerBasePlugin.java LOG.debug spelling error
[
https://issues.apache.org/jira/browse/RANGER-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang Zhang resolved RANGER-1517.
-
Resolution: Fixed
Fix Version/s: 1.0.0
> In RangerBasePlugin.java LOG.debug spelling error
> --
>
> Key: RANGER-1517
> URL: https://issues.apache.org/jira/browse/RANGER-1517
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 1.0.0
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments:
> 0001-RANGER-1517-In-RangerBasePlugin.java-LOG.debug-spell.patch
>
>
> In RangerBasePlugin.java LOG.debug spelling error for
> LOG.debug("Scheduled PolicyEngineRefresher to reorder policies nbased on
> number of evaluations in and every " + policyReorderIntervalMs + "
> milliseconds");
> change from "nbased " to "based "
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Re: Review Request 58332: The browser returns garbled when we login ranger security admin in non-English environment. We need support the internationalization function to solve this problem
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58332/#review172295 --- Ship it! It looks good. The function is right through tested and verified. - Qiang Zhang On 四月 17, 2017, 5:30 a.m., pengjianhua wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58332/ > --- > > (Updated 四月 17, 2017, 5:30 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-1508 > https://issues.apache.org/jira/browse/RANGER-1508 > > > Repository: ranger > > > Description > --- > > Use the browser to log in Ranger admin in Chinese operation system. The > browser returns garbled when an error occurs. The error reason is that the > Ranger admin used hard code to compare error information between Chinese > character and English character. So we need support the internationalization > function to solve this problem. > > > Diffs > - > > > security-admin/src/main/java/org/apache/ranger/security/web/authentication/RangerAuthFailureHandler.java > 580b6bc > security-admin/src/main/java/org/apache/ranger/util/CLIUtil.java 0fd0e70 > security-admin/src/main/resources/internationalization/messages.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_cs_CZ.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_de.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_es_ES.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_fr.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_it.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_ko_KR.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_lt.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_pl.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_pt_BR.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_pt_PT.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_uk_UA.properties > PRE-CREATION > > security-admin/src/main/resources/internationalization/messages_zh_CN.properties > PRE-CREATION > security-admin/src/main/webapp/META-INF/applicationContext.xml 95d462b > > > Diff: https://reviews.apache.org/r/58332/diff/2/ > > > Testing > --- > > tested it > > > Thanks, > > pengjianhua > >
[jira] [Commented] (RANGER-1521) Ranger database script for mysql requires admin privileges for replicated database
[
https://issues.apache.org/jira/browse/RANGER-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973176#comment-15973176
]
Yan commented on RANGER-1521:
-
[~eyang] Thanks for info. Surely automated installation/deployment is
preferred. Is this the only place that can achieve the same goal?
My take is the same: we'd better to understand the usage first, before land on
a solution. Risk evaluation should also follow careful analysis and
understanding of the usage.
> Ranger database script for mysql requires admin privileges for replicated
> database
> --
>
> Key: RANGER-1521
> URL: https://issues.apache.org/jira/browse/RANGER-1521
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 0.6.2
>Reporter: Eric Yang
> Attachments: RANGER-1521.patch
>
>
> When running install or upgrade on replicated MySQL or MariaDB, ranger
> installation might fail with:
> {code}
> Error executing: CREATE FUNCTION `getTempPolicyCount`(assetId bigint, resId
> bigint) RETURNS int(11) BEGIN DECLARE tempPolicyCount int default 1; DECLARE
> dbResourceId bigint; DECLARE exitLoop int DEFAULT FALSE; DECLARE policyList
> CURSOR FOR SELECT id from x_resource where asset_id = assetId; DECLARE
> CONTINUE HANDLER FOR NOT FOUND SET exitLoop = true; OPEN policyList;
> readPolicy : LOOP FETCH policyList into dbResourceId; IF exitLoop THEN set
> tempPolicyCount = tempPolicyCount + 1; LEAVE readPolicy; END IF; IF (resId =
> dbResourceId) THEN LEAVE readPolicy; END IF; set tempPolicyCount =
> tempPolicyCount + 1; END LOOP; CLOSE policyList; RETURN tempPolicyCount; END
> java.sql.SQLException: This function has none of DETERMINISTIC, NO SQL, or
> READS SQL DATA in its declaration and binary logging is enabled (you *might*
> want to use the less safe log_bin_trust_function_creators variable)
> SQLException : SQL state: HY000 java.sql.SQLException: This function has none
> of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary
> logging is enabled (you *might* want to use the less safe
> log_bin_trust_function_creators variable) ErrorCode: 1418 2017-04-05
> 22:59:00,345 [JISQL]
> /usr/jdk64/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/bin/java -cp
> /usr/iop/current/ranger-admin/ews/lib/mysql-connector-java.jar:/usr/iop/current/ranger-admin/jisql/lib/*
> org.apache.util.sql.Jisql -driver mysqlconj -cstring
> jdbc:mysql://tmh21-3-dal10-bi-mn000.fyre.ibm.com/ranger -u 'ranger' -p
> '' -noheader -trim -c \; -query "delete from x_db_version_h where
> version='007' and active='N' and
> updated_by='tmh21-3-dal10-bi-mn004.fyre.ibm.com';" 2017-04-05 22:59:00,781
> [E] 007-updateBlankPolicyName.sql import failed!
> {code}
> Two files under ranger-admin/db/mysql/patches:
> 007-updateBlankPolicyName.sql and 008-removeTrailingSlash.sql are using
> rand() functions to generate transaction ID, which makes them
> non-deterministic functions and are causing failures on replicated MySQL
> database.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (RANGER-1521) Ranger database script for mysql requires admin privileges for replicated database
[
https://issues.apache.org/jira/browse/RANGER-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973146#comment-15973146
]
Eric Yang commented on RANGER-1521:
---
[~yzhou2001] This is particularly painful for cloud customers where hundred
customers request mysql admin privileges to roll out Ranger at scale. This is
a low risk change that can make operation procedure easier and reducing
replication database corruption risk.
MariaDB 10.2.4 and newer are default to use MIXED format for binlog. This
change allows user to apply the DDL without worry database corruption that is
associated with usage of rand() in the MIXED format environment. See release
note from: https://mariadb.com/kb/en/mariadb/mariadb-1024-release-notes/
> Ranger database script for mysql requires admin privileges for replicated
> database
> --
>
> Key: RANGER-1521
> URL: https://issues.apache.org/jira/browse/RANGER-1521
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 0.6.2
>Reporter: Eric Yang
> Attachments: RANGER-1521.patch
>
>
> When running install or upgrade on replicated MySQL or MariaDB, ranger
> installation might fail with:
> {code}
> Error executing: CREATE FUNCTION `getTempPolicyCount`(assetId bigint, resId
> bigint) RETURNS int(11) BEGIN DECLARE tempPolicyCount int default 1; DECLARE
> dbResourceId bigint; DECLARE exitLoop int DEFAULT FALSE; DECLARE policyList
> CURSOR FOR SELECT id from x_resource where asset_id = assetId; DECLARE
> CONTINUE HANDLER FOR NOT FOUND SET exitLoop = true; OPEN policyList;
> readPolicy : LOOP FETCH policyList into dbResourceId; IF exitLoop THEN set
> tempPolicyCount = tempPolicyCount + 1; LEAVE readPolicy; END IF; IF (resId =
> dbResourceId) THEN LEAVE readPolicy; END IF; set tempPolicyCount =
> tempPolicyCount + 1; END LOOP; CLOSE policyList; RETURN tempPolicyCount; END
> java.sql.SQLException: This function has none of DETERMINISTIC, NO SQL, or
> READS SQL DATA in its declaration and binary logging is enabled (you *might*
> want to use the less safe log_bin_trust_function_creators variable)
> SQLException : SQL state: HY000 java.sql.SQLException: This function has none
> of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary
> logging is enabled (you *might* want to use the less safe
> log_bin_trust_function_creators variable) ErrorCode: 1418 2017-04-05
> 22:59:00,345 [JISQL]
> /usr/jdk64/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/bin/java -cp
> /usr/iop/current/ranger-admin/ews/lib/mysql-connector-java.jar:/usr/iop/current/ranger-admin/jisql/lib/*
> org.apache.util.sql.Jisql -driver mysqlconj -cstring
> jdbc:mysql://tmh21-3-dal10-bi-mn000.fyre.ibm.com/ranger -u 'ranger' -p
> '' -noheader -trim -c \; -query "delete from x_db_version_h where
> version='007' and active='N' and
> updated_by='tmh21-3-dal10-bi-mn004.fyre.ibm.com';" 2017-04-05 22:59:00,781
> [E] 007-updateBlankPolicyName.sql import failed!
> {code}
> Two files under ranger-admin/db/mysql/patches:
> 007-updateBlankPolicyName.sql and 008-removeTrailingSlash.sql are using
> rand() functions to generate transaction ID, which makes them
> non-deterministic functions and are causing failures on replicated MySQL
> database.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (RANGER-1521) Ranger database script for mysql requires admin privileges for replicated database
[
https://issues.apache.org/jira/browse/RANGER-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973102#comment-15973102
]
Yan commented on RANGER-1521:
-
[~eyang] Not sure can follow your admin privilege arguments. Note that 1) the
binlog_format by default is "statement" prior to 5.7.7 and "row" afterwards.
And changing that value would require admin privilege anyways; 2) Ranger
install/upgrade ops are intrinsically "intrusive" to the DB. If no admin role
is possible, it'd be a bit involved or even tricky to set up the privileges
properly; 3) Ranger install/update ops are not routine jobs to be performed
daily. So it should be feasible/acceptable for the DB Admin and Ranger Admin to
work out a plan to perform the job by, say, temporarily granting the Admin
role. Maybe you have some concrete use cases to share?
On other aspects:
Right, rand() does not guarantee uniqueness and NOW() could be made to be more
unique. My opinion is that in general rand() is much more likely to be unique
than NOW(). Anyways, in terms of entropy, NOW() can't compare with rand().
After all it depends upon the usage of the "trans_id".
On the data integrity side, the point is not deterministic vs.
nondeterministic nor safe vs. unsafe functions. The point is the "binary
logging" in the replication. In the MySQL doc, there is a dazzling array of
conditions, options and exceptions on the topic. Before we find an optimal
solution in that domain, I guess it is more appropriate to first discern the
usage of the "transaction id". For instances, if it is of practically no use,
we should consider its removal; if it's used for identification purpose, then
uniqueness will be an important property; if it's used for verification
purposes, then entropy will be important. Any insights from the community ?
Before we are clear on the usage, IMHO, a proper approach is to find a
workaround.
> Ranger database script for mysql requires admin privileges for replicated
> database
> --
>
> Key: RANGER-1521
> URL: https://issues.apache.org/jira/browse/RANGER-1521
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 0.6.2
>Reporter: Eric Yang
> Attachments: RANGER-1521.patch
>
>
> When running install or upgrade on replicated MySQL or MariaDB, ranger
> installation might fail with:
> {code}
> Error executing: CREATE FUNCTION `getTempPolicyCount`(assetId bigint, resId
> bigint) RETURNS int(11) BEGIN DECLARE tempPolicyCount int default 1; DECLARE
> dbResourceId bigint; DECLARE exitLoop int DEFAULT FALSE; DECLARE policyList
> CURSOR FOR SELECT id from x_resource where asset_id = assetId; DECLARE
> CONTINUE HANDLER FOR NOT FOUND SET exitLoop = true; OPEN policyList;
> readPolicy : LOOP FETCH policyList into dbResourceId; IF exitLoop THEN set
> tempPolicyCount = tempPolicyCount + 1; LEAVE readPolicy; END IF; IF (resId =
> dbResourceId) THEN LEAVE readPolicy; END IF; set tempPolicyCount =
> tempPolicyCount + 1; END LOOP; CLOSE policyList; RETURN tempPolicyCount; END
> java.sql.SQLException: This function has none of DETERMINISTIC, NO SQL, or
> READS SQL DATA in its declaration and binary logging is enabled (you *might*
> want to use the less safe log_bin_trust_function_creators variable)
> SQLException : SQL state: HY000 java.sql.SQLException: This function has none
> of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary
> logging is enabled (you *might* want to use the less safe
> log_bin_trust_function_creators variable) ErrorCode: 1418 2017-04-05
> 22:59:00,345 [JISQL]
> /usr/jdk64/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/bin/java -cp
> /usr/iop/current/ranger-admin/ews/lib/mysql-connector-java.jar:/usr/iop/current/ranger-admin/jisql/lib/*
> org.apache.util.sql.Jisql -driver mysqlconj -cstring
> jdbc:mysql://tmh21-3-dal10-bi-mn000.fyre.ibm.com/ranger -u 'ranger' -p
> '' -noheader -trim -c \; -query "delete from x_db_version_h where
> version='007' and active='N' and
> updated_by='tmh21-3-dal10-bi-mn004.fyre.ibm.com';" 2017-04-05 22:59:00,781
> [E] 007-updateBlankPolicyName.sql import failed!
> {code}
> Two files under ranger-admin/db/mysql/patches:
> 007-updateBlankPolicyName.sql and 008-removeTrailingSlash.sql are using
> rand() functions to generate transaction ID, which makes them
> non-deterministic functions and are causing failures on replicated MySQL
> database.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (RANGER-1499) Upgrade Tomcat version
[ https://issues.apache.org/jira/browse/RANGER-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973011#comment-15973011 ] Colm O hEigeartaigh commented on RANGER-1499: - Hi [~vperiasamy], no reason specifically to update to 7.0.77. It's a trade-off between shipping with known bugs versus unknown bugs of a new version. I would always lean to picking up the latest (minor) release + testing it properly, but I'll leave the decision up to you either way here. > Upgrade Tomcat version > -- > > Key: RANGER-1499 > URL: https://issues.apache.org/jira/browse/RANGER-1499 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Velmurugan Periasamy > Fix For: 0.6.4, 1.0.0, 0.7.1 > > > Tomcat version used by Ranger & Ranger KMS is 7.0.68. > Need to upgrade to 7.0.73 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1521) Ranger database script for mysql requires admin privileges for replicated database
[
https://issues.apache.org/jira/browse/RANGER-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972804#comment-15972804
]
Eric Yang commented on RANGER-1521:
---
[~yzhou2001]: 1) the purpose of this change is to reduce the requirement of
Ranger DDL to use non-admin privileges for modifying the database. In some
company, it is more work to obtain mysql admin privileges for external mysql
database. The workaround does not work in such settings.
2) rand does not guarantee uniqueness. The transaction id is only useful to
see the changes that are being made on the same batch. By changing it to
Now(), it does not make it less unique since rand() could produce duplicated
number. We can also make the change to produce micro-seconds number to make
transaction number more unique. The transaction id in the DDL are only used
for recording the changes to x_resource table. There is no use of transaction
id for further processing beyond the patches. Therefore, existing semantics
are preserved to the closest possible details.
[~pradeep.agrawal] The SQL statements are non-deterministic by default. If it
always produce the same result, then it is deterministic. Insert statement
does not imply non-deterministic. Insert and Update can further annotate the
data being MODIFIES SQL DATA. However, the changes of statements are for
making the DDL work without admin privileges, and make it future proof for
binlog_format=MIXED environment. These changes are improvement to reduce the
need of asking mysql admin privileges because rand() is used.
> Ranger database script for mysql requires admin privileges for replicated
> database
> --
>
> Key: RANGER-1521
> URL: https://issues.apache.org/jira/browse/RANGER-1521
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 0.6.2
>Reporter: Eric Yang
> Attachments: RANGER-1521.patch
>
>
> When running install or upgrade on replicated MySQL or MariaDB, ranger
> installation might fail with:
> {code}
> Error executing: CREATE FUNCTION `getTempPolicyCount`(assetId bigint, resId
> bigint) RETURNS int(11) BEGIN DECLARE tempPolicyCount int default 1; DECLARE
> dbResourceId bigint; DECLARE exitLoop int DEFAULT FALSE; DECLARE policyList
> CURSOR FOR SELECT id from x_resource where asset_id = assetId; DECLARE
> CONTINUE HANDLER FOR NOT FOUND SET exitLoop = true; OPEN policyList;
> readPolicy : LOOP FETCH policyList into dbResourceId; IF exitLoop THEN set
> tempPolicyCount = tempPolicyCount + 1; LEAVE readPolicy; END IF; IF (resId =
> dbResourceId) THEN LEAVE readPolicy; END IF; set tempPolicyCount =
> tempPolicyCount + 1; END LOOP; CLOSE policyList; RETURN tempPolicyCount; END
> java.sql.SQLException: This function has none of DETERMINISTIC, NO SQL, or
> READS SQL DATA in its declaration and binary logging is enabled (you *might*
> want to use the less safe log_bin_trust_function_creators variable)
> SQLException : SQL state: HY000 java.sql.SQLException: This function has none
> of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary
> logging is enabled (you *might* want to use the less safe
> log_bin_trust_function_creators variable) ErrorCode: 1418 2017-04-05
> 22:59:00,345 [JISQL]
> /usr/jdk64/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/bin/java -cp
> /usr/iop/current/ranger-admin/ews/lib/mysql-connector-java.jar:/usr/iop/current/ranger-admin/jisql/lib/*
> org.apache.util.sql.Jisql -driver mysqlconj -cstring
> jdbc:mysql://tmh21-3-dal10-bi-mn000.fyre.ibm.com/ranger -u 'ranger' -p
> '' -noheader -trim -c \; -query "delete from x_db_version_h where
> version='007' and active='N' and
> updated_by='tmh21-3-dal10-bi-mn004.fyre.ibm.com';" 2017-04-05 22:59:00,781
> [E] 007-updateBlankPolicyName.sql import failed!
> {code}
> Two files under ranger-admin/db/mysql/patches:
> 007-updateBlankPolicyName.sql and 008-removeTrailingSlash.sql are using
> rand() functions to generate transaction ID, which makes them
> non-deterministic functions and are causing failures on replicated MySQL
> database.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (RANGER-1499) Upgrade Tomcat version
[ https://issues.apache.org/jira/browse/RANGER-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972805#comment-15972805 ] Velmurugan Periasamy commented on RANGER-1499: -- [~coheigea] - My only concern is 7.0.77 is released only couple of weeks back. But if you see specific reasons to upgrade to that latest version and there is no impact on functionality, then I am fine with that. > Upgrade Tomcat version > -- > > Key: RANGER-1499 > URL: https://issues.apache.org/jira/browse/RANGER-1499 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Velmurugan Periasamy > Fix For: 0.6.4, 1.0.0, 0.7.1 > > > Tomcat version used by Ranger & Ranger KMS is 7.0.68. > Need to upgrade to 7.0.73 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Review Request 58494: Ranger hive service definition to use hive metastore directly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58494/#review172200 --- The indentation of the two thrift dependencies is a bit off. The groupId for the first dependency is at column 17, whereas the second is at column 15. - Colm O hEigeartaigh On April 18, 2017, 6:24 a.m., Ankita Sinha wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58494/ > --- > > (Updated April 18, 2017, 6:24 a.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-1483 > https://issues.apache.org/jira/browse/RANGER-1483 > > > Repository: ranger > > > Description > --- > > Added some libraries for Ranger to use Hive Metastore for Test connection and > Resource Lookup > > > Diffs > - > > pom.xml 4fb62e2 > security-admin/pom.xml caf3576 > > > Diff: https://reviews.apache.org/r/58494/diff/1/ > > > Testing > --- > > Tested on simple environment > > > Thanks, > > Ankita Sinha > >
Re: Review Request 58495: Ranger hive service definition to use hive metastore directly
> On April 18, 2017, 9:26 a.m., Colm O hEigeartaigh wrote: > > The diff is just the pom diff...it looks like the rest of the patch is > > missing? > > Ankita Sinha wrote: > The patch is complete, it just contains the changes related to pom. The > existing code needs those jar at runtime. > > Colm O hEigeartaigh wrote: > Did you mean to submit the rest of the patch in > https://reviews.apache.org/r/58494/ ? That just contains the pom changes as > well. > > Ankita Sinha wrote: > Committed to Master branch : > https://github.com/apache/ranger/commit/6cfb01883fb97bd98e5e5b7baacb3cdd85a15b68 > Committed to 0.7 branch : > https://github.com/apache/ranger/commit/6c0b06252ad670bdbd5ffba93a3b51015308e9ec > > So the current patch is on top of the above commits to work with > different setting of HiveMetastore. So in some cases the Jar was missing and > this and https://reviews.apache.org/r/58494/ patch handles that for > ranger-0.7 and master branch respectively. OK got it, thanks for the explanation! - Colm --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58495/#review172181 --- On April 18, 2017, 6:24 a.m., Ankita Sinha wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58495/ > --- > > (Updated April 18, 2017, 6:24 a.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-1483 > https://issues.apache.org/jira/browse/RANGER-1483 > > > Repository: ranger > > > Description > --- > > Added some libraries for Ranger to use Hive Metastore for Test connection and > Resource Lookup > > > Diffs > - > > pom.xml de2eeb2 > security-admin/pom.xml fd387a8 > > > Diff: https://reviews.apache.org/r/58495/diff/1/ > > > Testing > --- > > Tested Hive Test Connection and Resource Lookup with different settings on > simple and secure cluster. > > > Thanks, > > Ankita Sinha > >
[jira] [Commented] (RANGER-1521) Ranger database script for mysql requires admin privileges for replicated database
[
https://issues.apache.org/jira/browse/RANGER-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972680#comment-15972680
]
Pradeep Agrawal commented on RANGER-1521:
-
According to my understanding from
[link|https://dev.mysql.com/doc/refman/5.7/en/create-procedure.html] it can't
be a deterministic function as there are insert SQL statement in these
functions.
> Ranger database script for mysql requires admin privileges for replicated
> database
> --
>
> Key: RANGER-1521
> URL: https://issues.apache.org/jira/browse/RANGER-1521
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 0.6.2
>Reporter: Eric Yang
> Attachments: RANGER-1521.patch
>
>
> When running install or upgrade on replicated MySQL or MariaDB, ranger
> installation might fail with:
> {code}
> Error executing: CREATE FUNCTION `getTempPolicyCount`(assetId bigint, resId
> bigint) RETURNS int(11) BEGIN DECLARE tempPolicyCount int default 1; DECLARE
> dbResourceId bigint; DECLARE exitLoop int DEFAULT FALSE; DECLARE policyList
> CURSOR FOR SELECT id from x_resource where asset_id = assetId; DECLARE
> CONTINUE HANDLER FOR NOT FOUND SET exitLoop = true; OPEN policyList;
> readPolicy : LOOP FETCH policyList into dbResourceId; IF exitLoop THEN set
> tempPolicyCount = tempPolicyCount + 1; LEAVE readPolicy; END IF; IF (resId =
> dbResourceId) THEN LEAVE readPolicy; END IF; set tempPolicyCount =
> tempPolicyCount + 1; END LOOP; CLOSE policyList; RETURN tempPolicyCount; END
> java.sql.SQLException: This function has none of DETERMINISTIC, NO SQL, or
> READS SQL DATA in its declaration and binary logging is enabled (you *might*
> want to use the less safe log_bin_trust_function_creators variable)
> SQLException : SQL state: HY000 java.sql.SQLException: This function has none
> of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary
> logging is enabled (you *might* want to use the less safe
> log_bin_trust_function_creators variable) ErrorCode: 1418 2017-04-05
> 22:59:00,345 [JISQL]
> /usr/jdk64/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/bin/java -cp
> /usr/iop/current/ranger-admin/ews/lib/mysql-connector-java.jar:/usr/iop/current/ranger-admin/jisql/lib/*
> org.apache.util.sql.Jisql -driver mysqlconj -cstring
> jdbc:mysql://tmh21-3-dal10-bi-mn000.fyre.ibm.com/ranger -u 'ranger' -p
> '' -noheader -trim -c \; -query "delete from x_db_version_h where
> version='007' and active='N' and
> updated_by='tmh21-3-dal10-bi-mn004.fyre.ibm.com';" 2017-04-05 22:59:00,781
> [E] 007-updateBlankPolicyName.sql import failed!
> {code}
> Two files under ranger-admin/db/mysql/patches:
> 007-updateBlankPolicyName.sql and 008-removeTrailingSlash.sql are using
> rand() functions to generate transaction ID, which makes them
> non-deterministic functions and are causing failures on replicated MySQL
> database.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Re: Review Request 58495: Ranger hive service definition to use hive metastore directly
> On April 18, 2017, 9:26 a.m., Colm O hEigeartaigh wrote: > > The diff is just the pom diff...it looks like the rest of the patch is > > missing? > > Ankita Sinha wrote: > The patch is complete, it just contains the changes related to pom. The > existing code needs those jar at runtime. > > Colm O hEigeartaigh wrote: > Did you mean to submit the rest of the patch in > https://reviews.apache.org/r/58494/ ? That just contains the pom changes as > well. Committed to Master branch : https://github.com/apache/ranger/commit/6cfb01883fb97bd98e5e5b7baacb3cdd85a15b68 Committed to 0.7 branch : https://github.com/apache/ranger/commit/6c0b06252ad670bdbd5ffba93a3b51015308e9ec So the current patch is on top of the above commits to work with different setting of HiveMetastore. So in some cases the Jar was missing and this and https://reviews.apache.org/r/58494/ patch handles that for ranger-0.7 and master branch respectively. - Ankita --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58495/#review172181 --- On April 18, 2017, 6:24 a.m., Ankita Sinha wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58495/ > --- > > (Updated April 18, 2017, 6:24 a.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-1483 > https://issues.apache.org/jira/browse/RANGER-1483 > > > Repository: ranger > > > Description > --- > > Added some libraries for Ranger to use Hive Metastore for Test connection and > Resource Lookup > > > Diffs > - > > pom.xml de2eeb2 > security-admin/pom.xml fd387a8 > > > Diff: https://reviews.apache.org/r/58495/diff/1/ > > > Testing > --- > > Tested on simple environment > > > Thanks, > > Ankita Sinha > >
Re: Review Request 58495: Ranger hive service definition to use hive metastore directly
> On April 18, 2017, 9:26 a.m., Colm O hEigeartaigh wrote: > > The diff is just the pom diff...it looks like the rest of the patch is > > missing? > > Ankita Sinha wrote: > The patch is complete, it just contains the changes related to pom. The > existing code needs those jar at runtime. Did you mean to submit the rest of the patch in https://reviews.apache.org/r/58494/ ? That just contains the pom changes as well. - Colm --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58495/#review172181 --- On April 18, 2017, 6:24 a.m., Ankita Sinha wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58495/ > --- > > (Updated April 18, 2017, 6:24 a.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-1483 > https://issues.apache.org/jira/browse/RANGER-1483 > > > Repository: ranger > > > Description > --- > > Added some libraries for Ranger to use Hive Metastore for Test connection and > Resource Lookup > > > Diffs > - > > pom.xml de2eeb2 > security-admin/pom.xml fd387a8 > > > Diff: https://reviews.apache.org/r/58495/diff/1/ > > > Testing > --- > > Tested on simple environment > > > Thanks, > > Ankita Sinha > >
Re: Review Request 58475: RANGER-1519:Error occurred after execute enable-hive-plugin.sh
> On 四月 18, 2017, 9:03 a.m., Colm O hEigeartaigh wrote: > > I think it would be better to change the comment "Hive configuration > > directory" to "Hive installation directory". Yes,I have updated the patch. - Qiang --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58475/#review172178 --- On 四月 18, 2017, 11:15 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58475/ > --- > > (Updated 四月 18, 2017, 11:15 a.m.) > > > Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Ramesh Mani, > and Velmurugan Periasamy. > > > Bugs: RANGER-1519 > https://issues.apache.org/jira/browse/RANGER-1519 > > > Repository: ranger > > > Description > --- > > [root@zdh41 ranger-1.0.0-SNAPSHOT-hive-plugin]# ./enable-hive-plugin.sh > Custom user and group is available, using custom user and group. > ERROR: Unable to find the conf directory of component [hive]; dir > [/home/xiehh/rangerplugin/hive/conf] not found. > Exiting installation. > > > Diffs > - > > hive-agent/scripts/install.properties 2c109af > > > Diff: https://reviews.apache.org/r/58475/diff/2/ > > > Testing > --- > > > Thanks, > > Qiang Zhang > >
Re: Review Request 58475: RANGER-1519:Error occurred after execute enable-hive-plugin.sh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58475/#review172185 --- Ship it! Ship It! - Colm O hEigeartaigh On April 18, 2017, 11:15 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58475/ > --- > > (Updated April 18, 2017, 11:15 a.m.) > > > Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Ramesh Mani, > and Velmurugan Periasamy. > > > Bugs: RANGER-1519 > https://issues.apache.org/jira/browse/RANGER-1519 > > > Repository: ranger > > > Description > --- > > [root@zdh41 ranger-1.0.0-SNAPSHOT-hive-plugin]# ./enable-hive-plugin.sh > Custom user and group is available, using custom user and group. > ERROR: Unable to find the conf directory of component [hive]; dir > [/home/xiehh/rangerplugin/hive/conf] not found. > Exiting installation. > > > Diffs > - > > hive-agent/scripts/install.properties 2c109af > > > Diff: https://reviews.apache.org/r/58475/diff/2/ > > > Testing > --- > > > Thanks, > > Qiang Zhang > >
Re: Review Request 58475: RANGER-1519:Error occurred after execute enable-hive-plugin.sh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58475/ --- (Updated 四月 18, 2017, 11:15 a.m.) Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-1519 https://issues.apache.org/jira/browse/RANGER-1519 Repository: ranger Description --- [root@zdh41 ranger-1.0.0-SNAPSHOT-hive-plugin]# ./enable-hive-plugin.sh Custom user and group is available, using custom user and group. ERROR: Unable to find the conf directory of component [hive]; dir [/home/xiehh/rangerplugin/hive/conf] not found. Exiting installation. Diffs (updated) - hive-agent/scripts/install.properties 2c109af Diff: https://reviews.apache.org/r/58475/diff/2/ Changes: https://reviews.apache.org/r/58475/diff/1-2/ Testing --- Thanks, Qiang Zhang
[jira] [Updated] (RANGER-1522) Update consolidated db schema script for SQLServer DB flavor to reduce execution time
[ https://issues.apache.org/jira/browse/RANGER-1522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-1522: Attachment: RANGER-1522-master-branch.patch RANGER-1522-ranger-0.7-branch.patch > Update consolidated db schema script for SQLServer DB flavor to reduce > execution time > - > > Key: RANGER-1522 > URL: https://issues.apache.org/jira/browse/RANGER-1522 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 1.0.0, 0.7.1 >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal > Fix For: 1.0.0, 0.7.1 > > Attachments: RANGER-1522-master-branch.patch, > RANGER-1522-ranger-0.7-branch.patch > > > Modify the schema file to exclude the GO statements or use the minimum number > of GO Statement. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Review Request 58495: Ranger hive service definition to use hive metastore directly
> On April 18, 2017, 9:26 a.m., Colm O hEigeartaigh wrote: > > The diff is just the pom diff...it looks like the rest of the patch is > > missing? The patch is complete, it just contains the changes related to pom. The existing code needs those jar at runtime. - Ankita --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58495/#review172181 --- On April 18, 2017, 6:24 a.m., Ankita Sinha wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58495/ > --- > > (Updated April 18, 2017, 6:24 a.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-1483 > https://issues.apache.org/jira/browse/RANGER-1483 > > > Repository: ranger > > > Description > --- > > Added some libraries for Ranger to use Hive Metastore for Test connection and > Resource Lookup > > > Diffs > - > > pom.xml de2eeb2 > security-admin/pom.xml fd387a8 > > > Diff: https://reviews.apache.org/r/58495/diff/1/ > > > Testing > --- > > Tested on simple environment > > > Thanks, > > Ankita Sinha > >
Re: Review Request 57443: Useless configuration in unixauthservice pom.xml lead to project compiler error in eclipse
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57443/#review172183
---
Shouldn't we include log4j.properties in src/main/resources as well? Then we
can just delete the build section from the pom, as the log4j.properties get
automatically copied to classes.
- Colm O hEigeartaigh
On March 9, 2017, 3:13 a.m., Qiang Zhang wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57443/
> ---
>
> (Updated March 9, 2017, 3:13 a.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan
> Neethiraj, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1438
> https://issues.apache.org/jira/browse/RANGER-1438
>
>
> Repository: ranger
>
>
> Description
> ---
>
> Useless configuration in unixauthservice pom.xml lead to project compiler
> error in eclipse
>
> Steps
> In windows environment, enter into ranger project root directory
> 1.Execute : mvn clean compile package install assembly:assembly -DskipTests
> -X
> 2.Execute :mvn eclipse:eclipse -X
> When the maven project convert into eclipse project occurred error:
> [[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-eclipse-plugin:2.10:eclipse (default-cli) on
> project unixauthservice: Cant canonicalize system path:
> {0}: -> [Help 1]
> Caused by: org.apache.maven.plugin.MojoExecutionException: Cant canonicalize
> system path: {0}
> ]
> 3.Remove unless code in unixauthservice pom.xml
>
>
>
> $
> {project.build.outputDirectory}
>
> conf.dist
>
> log4j.properties
>
>
>
>
> 4.Rerun : mvn clean compile package install assembly:assembly -DskipTests -X
> 5.Rerun :mvn eclipse:eclipse -X (result:sucessed)
> 6.Check whether affect the Usersync Components work
> 1).Install the ranger-1.0.0-SNAPSHOT-usersync module in a new machine
> 2).and Reinstall the ranger-1.0.0-SNAPSHOT-usersync module in original machine
> 3)result:both work well ( Log upload in the attachment)
>
>
> Diffs
> -
>
> unixauthservice/pom.xml cc8fb1c
>
>
> Diff: https://reviews.apache.org/r/57443/diff/1/
>
>
> Testing
> ---
>
> tested it
>
>
> Thanks,
>
> Qiang Zhang
>
>
Review Request 58499: RANGER-1524 - Add tag based authorization tests for Apache Storm
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58499/ --- Review request for ranger. Bugs: RANGER-1524 https://issues.apache.org/jira/browse/RANGER-1524 Repository: ranger Description --- Similar to RANGER-1421, this task is to add a test for tag based authorization to the Storm component. Diffs - storm-agent/src/test/java/org/apache/ranger/authorization/storm/RangerAdminClientImpl.java ade034d storm-agent/src/test/java/org/apache/ranger/authorization/storm/StormRangerAuthorizerTest.java fc648b3 storm-agent/src/test/java/org/apache/ranger/authorization/storm/WordSpout.java 809c998 storm-agent/src/test/resources/ranger-storm-security.xml adff2b9 storm-agent/src/test/resources/storm-policies-tag.json PRE-CREATION storm-agent/src/test/resources/storm-policies.json 5c04b5d Diff: https://reviews.apache.org/r/58499/diff/1/ Testing --- Thanks, Colm O hEigeartaigh
[jira] [Updated] (RANGER-1524) Add tag based authorization tests for Apache Storm
[ https://issues.apache.org/jira/browse/RANGER-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1524: Attachment: 0001-RANGER-1524-Add-tag-based-authorization-tests-for-Ap.patch > Add tag based authorization tests for Apache Storm > -- > > Key: RANGER-1524 > URL: https://issues.apache.org/jira/browse/RANGER-1524 > Project: Ranger > Issue Type: Test > Components: plugins >Reporter: Colm O hEigeartaigh >Assignee: Colm O hEigeartaigh > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1524-Add-tag-based-authorization-tests-for-Ap.patch > > > Similar to RANGER-1421, this task is to add a test for tag based > authorization to the Storm component. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (RANGER-1524) Add tag based authorization tests for Apache Storm
Colm O hEigeartaigh created RANGER-1524: --- Summary: Add tag based authorization tests for Apache Storm Key: RANGER-1524 URL: https://issues.apache.org/jira/browse/RANGER-1524 Project: Ranger Issue Type: Test Components: plugins Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Fix For: 1.0.0 Similar to RANGER-1421, this task is to add a test for tag based authorization to the Storm component. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Review Request 58498: RANGER-1523 - Add AtlasResourceMapper implementation for Apache Storm
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58498/ --- Review request for ranger. Bugs: RANGER-1523 https://issues.apache.org/jira/browse/RANGER-1523 Repository: ranger Description --- This task is to add a AtlasResourceMapper implementation for Apache Storm. This will allow us to sync tags associated with Storm topologies into Ranger via the REST aproach + to create tag based authorization policies as a result. Diffs - tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasResourceMapperUtil.java f9f0eaf tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasStormResourceMapper.java PRE-CREATION Diff: https://reviews.apache.org/r/58498/diff/1/ Testing --- Tested that we can sync Storm tags in to Ranger from Atlas + create tag based policies as a result for Storm. Thanks, Colm O hEigeartaigh
[jira] [Updated] (RANGER-1523) Add AtlasResourceMapper implementation for Apache Storm
[ https://issues.apache.org/jira/browse/RANGER-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1523: Attachment: 0001-RANGER-1523-Add-AtlasResourceMapper-implementation-f.patch > Add AtlasResourceMapper implementation for Apache Storm > --- > > Key: RANGER-1523 > URL: https://issues.apache.org/jira/browse/RANGER-1523 > Project: Ranger > Issue Type: New Feature > Components: tagsync >Reporter: Colm O hEigeartaigh >Assignee: Colm O hEigeartaigh > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1523-Add-AtlasResourceMapper-implementation-f.patch > > > This task is to add a AtlasResourceMapper implementation for Apache Storm. > This will allow us to sync tags associated with Storm topologies into Ranger > via the REST aproach + to create tag based authorization policies as a result. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (RANGER-1523) Add AtlasResourceMapper implementation for Apache Storm
Colm O hEigeartaigh created RANGER-1523: --- Summary: Add AtlasResourceMapper implementation for Apache Storm Key: RANGER-1523 URL: https://issues.apache.org/jira/browse/RANGER-1523 Project: Ranger Issue Type: New Feature Components: tagsync Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Fix For: 1.0.0 This task is to add a AtlasResourceMapper implementation for Apache Storm. This will allow us to sync tags associated with Storm topologies into Ranger via the REST aproach + to create tag based authorization policies as a result. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Review Request 58495: Ranger hive service definition to use hive metastore directly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58495/#review172181 --- The diff is just the pom diff...it looks like the rest of the patch is missing? - Colm O hEigeartaigh On April 18, 2017, 6:24 a.m., Ankita Sinha wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58495/ > --- > > (Updated April 18, 2017, 6:24 a.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-1483 > https://issues.apache.org/jira/browse/RANGER-1483 > > > Repository: ranger > > > Description > --- > > Added some libraries for Ranger to use Hive Metastore for Test connection and > Resource Lookup > > > Diffs > - > > pom.xml de2eeb2 > security-admin/pom.xml fd387a8 > > > Diff: https://reviews.apache.org/r/58495/diff/1/ > > > Testing > --- > > Tested on simple environment > > > Thanks, > > Ankita Sinha > >
Re: Review Request 58472: In RangerBasePlugin.java LOG.debug spelling error
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58472/#review172180
---
Ship it!
Ship It!
- Colm O hEigeartaigh
On April 17, 2017, 5:43 a.m., Qiang Zhang wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58472/
> ---
>
> (Updated April 17, 2017, 5:43 a.m.)
>
>
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
>
>
> Bugs: RANGER-1517
> https://issues.apache.org/jira/browse/RANGER-1517
>
>
> Repository: ranger
>
>
> Description
> ---
>
> In RangerBasePlugin.java LOG.debug spelling error for
> LOG.debug("Scheduled PolicyEngineRefresher to reorder policies nbased on
> number of evaluations in and every " + policyReorderIntervalMs + "
> milliseconds");
> change from "nbased " to "based "
>
>
> Diffs
> -
>
>
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
> acf8d15
>
>
> Diff: https://reviews.apache.org/r/58472/diff/1/
>
>
> Testing
> ---
>
> tested it
>
>
> Thanks,
>
> Qiang Zhang
>
>
Re: Review Request 58474: RANGER-1518:Do some code improvement for the error message in PolicyMgrUserGroupBuilder.java
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58474/#review172179
---
Ship it!
Ship It!
- Colm O hEigeartaigh
On April 17, 2017, 5:49 a.m., Qiang Zhang wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58474/
> ---
>
> (Updated April 17, 2017, 5:49 a.m.)
>
>
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan
> Neethiraj, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1518
> https://issues.apache.org/jira/browse/RANGER-1518
>
>
> Repository: ranger
>
>
> Description
> ---
>
> if (addUserGroupInfo(ugInfo) == null) {
> String msg = "Failed to
> add add user group info";
> LOG.error(msg);
> throw new
> Exception(msg);
> }
> Duplicate 'add' for this error message.
>
>
> Diffs
> -
>
>
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
> 070a39b
>
>
> Diff: https://reviews.apache.org/r/58474/diff/1/
>
>
> Testing
> ---
>
>
> Thanks,
>
> Qiang Zhang
>
>
Re: Review Request 58475: RANGER-1519:Error occurred after execute enable-hive-plugin.sh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58475/#review172178 --- I think it would be better to change the comment "Hive configuration directory" to "Hive installation directory". - Colm O hEigeartaigh On April 17, 2017, 6:57 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58475/ > --- > > (Updated April 17, 2017, 6:57 a.m.) > > > Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Ramesh Mani, > and Velmurugan Periasamy. > > > Bugs: RANGER-1519 > https://issues.apache.org/jira/browse/RANGER-1519 > > > Repository: ranger > > > Description > --- > > [root@zdh41 ranger-1.0.0-SNAPSHOT-hive-plugin]# ./enable-hive-plugin.sh > Custom user and group is available, using custom user and group. > ERROR: Unable to find the conf directory of component [hive]; dir > [/home/xiehh/rangerplugin/hive/conf] not found. > Exiting installation. > > > Diffs > - > > hive-agent/scripts/install.properties 2c109af > > > Diff: https://reviews.apache.org/r/58475/diff/1/ > > > Testing > --- > > > Thanks, > > Qiang Zhang > >
Re: Review Request 58476: RANGER-1520:Some codes do not follow the python language development rules in usersync. They are messy.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58476/#review172175 --- You could change "proprty" to "property" (3 times) - Colm O hEigeartaigh On April 17, 2017, 9:20 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58476/ > --- > > (Updated April 17, 2017, 9:20 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, and Velmurugan Periasamy. > > > Bugs: RANGER-1520 > https://issues.apache.org/jira/browse/RANGER-1520 > > > Repository: ranger > > > Description > --- > > Python is the language that uses indent control code.The indentation of the > rows should be consistent. Some codes do not follow the rule in usersync. > They are messy. > > > Diffs > - > > unixauthservice/scripts/setup.py d7872ea > > > Diff: https://reviews.apache.org/r/58476/diff/1/ > > > Testing > --- > > > Thanks, > > Qiang Zhang > >
[jira] [Created] (RANGER-1522) Update consolidated db schema script for SQLServer DB flavor to reduce execution time
Pradeep Agrawal created RANGER-1522: --- Summary: Update consolidated db schema script for SQLServer DB flavor to reduce execution time Key: RANGER-1522 URL: https://issues.apache.org/jira/browse/RANGER-1522 Project: Ranger Issue Type: Bug Components: Ranger Affects Versions: 1.0.0, 0.7.1 Reporter: Pradeep Agrawal Assignee: Pradeep Agrawal Fix For: 1.0.0, 0.7.1 Modify the schema file to exclude the GO statements or use the minimum number of GO Statement. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RANGER-1521) Ranger database script for mysql requires admin privileges for replicated database
[
https://issues.apache.org/jira/browse/RANGER-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972229#comment-15972229
]
Yan commented on RANGER-1521:
-
A few comments:
1) There are a few workarounds to address the data corruption issue. The one
mentioned above comes from
https://community.hortonworks.com/questions/23647/ranger-admin-install-fails-with-007-updateblankpol.html
2) Use of Now() probably won't serve the purpose of uniqueness for "transaction
ID" given its second-level granularity;
3) More knowledge about the use of "transaction id" in Ranger is desired. If it
is of no practical use, it should be considered to be removed at all; if it is
used for identification purpose, the uniqueness requirement would be important.
May the community shed more lights on this?
4) Ranger intrinsic logic to create transaction ID utilizes some random #
generation. Not sure whether it would be compatible with the timestamp-based
IDs as proposed here.
> Ranger database script for mysql requires admin privileges for replicated
> database
> --
>
> Key: RANGER-1521
> URL: https://issues.apache.org/jira/browse/RANGER-1521
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 0.6.2
>Reporter: Eric Yang
> Attachments: RANGER-1521.patch
>
>
> When running install or upgrade on replicated MySQL or MariaDB, ranger
> installation might fail with:
> {code}
> Error executing: CREATE FUNCTION `getTempPolicyCount`(assetId bigint, resId
> bigint) RETURNS int(11) BEGIN DECLARE tempPolicyCount int default 1; DECLARE
> dbResourceId bigint; DECLARE exitLoop int DEFAULT FALSE; DECLARE policyList
> CURSOR FOR SELECT id from x_resource where asset_id = assetId; DECLARE
> CONTINUE HANDLER FOR NOT FOUND SET exitLoop = true; OPEN policyList;
> readPolicy : LOOP FETCH policyList into dbResourceId; IF exitLoop THEN set
> tempPolicyCount = tempPolicyCount + 1; LEAVE readPolicy; END IF; IF (resId =
> dbResourceId) THEN LEAVE readPolicy; END IF; set tempPolicyCount =
> tempPolicyCount + 1; END LOOP; CLOSE policyList; RETURN tempPolicyCount; END
> java.sql.SQLException: This function has none of DETERMINISTIC, NO SQL, or
> READS SQL DATA in its declaration and binary logging is enabled (you *might*
> want to use the less safe log_bin_trust_function_creators variable)
> SQLException : SQL state: HY000 java.sql.SQLException: This function has none
> of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary
> logging is enabled (you *might* want to use the less safe
> log_bin_trust_function_creators variable) ErrorCode: 1418 2017-04-05
> 22:59:00,345 [JISQL]
> /usr/jdk64/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/bin/java -cp
> /usr/iop/current/ranger-admin/ews/lib/mysql-connector-java.jar:/usr/iop/current/ranger-admin/jisql/lib/*
> org.apache.util.sql.Jisql -driver mysqlconj -cstring
> jdbc:mysql://tmh21-3-dal10-bi-mn000.fyre.ibm.com/ranger -u 'ranger' -p
> '' -noheader -trim -c \; -query "delete from x_db_version_h where
> version='007' and active='N' and
> updated_by='tmh21-3-dal10-bi-mn004.fyre.ibm.com';" 2017-04-05 22:59:00,781
> [E] 007-updateBlankPolicyName.sql import failed!
> {code}
> Two files under ranger-admin/db/mysql/patches:
> 007-updateBlankPolicyName.sql and 008-removeTrailingSlash.sql are using
> rand() functions to generate transaction ID, which makes them
> non-deterministic functions and are causing failures on replicated MySQL
> database.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
