[jira] [Created] (RANGER-2052) The Ranger admin supports high availability
Qiang Zhang created RANGER-2052: --- Summary: The Ranger admin supports high availability Key: RANGER-2052 URL: https://issues.apache.org/jira/browse/RANGER-2052 Project: Ranger Issue Type: New Feature Components: admin Reporter: Qiang Zhang Assignee: Qiang Zhang In real business environment, such as banks application and so on, we should use other Ranger service to continuely supply service once a Ranger service exits abnormally. We call the feature as high availability. The Ranger high availability includes: 1. The Ranger admin is high availability; 2. The mysql service that saves meta data is high availability. For the Ranger admin high availability our scheme is as following: 1. Add a configuration item in plugin. The configuration item includes two the address of the Ranger admin service. 2. The plugin refreshed policy from other Ranger admin service once it refresh policy from a Ranger admin service fail. For the mysql service that saves meta data admin high availability, our scheme is as following: 1. Add a configuration item in the Ranger admin. The configuration item includes two the address of the mysql service. 2. Build two jdbc connection pools. The other will be used once a jdbc connection fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (RANGER-1809) Write install guide for RANGER-1672
[ https://issues.apache.org/jira/browse/RANGER-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Zhang resolved RANGER-1809. - Resolution: Fixed Fix Version/s: 1.1.0 master > Write install guide for RANGER-1672 > --- > > Key: RANGER-1809 > URL: https://issues.apache.org/jira/browse/RANGER-1809 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Affects Versions: 1.0.0, master >Reporter: Qiang Zhang >Assignee: Qiang Zhang >Priority: Major > Labels: newbie, patch > Fix For: master, 1.1.0 > > > Write install guide for RANGER-1672 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RANGER-2017) Ranger KMS encryption good practices
[ https://issues.apache.org/jira/browse/RANGER-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bhavik patel updated RANGER-2017: - Attachment: RANGER-2017-master.patch > Ranger KMS encryption good practices > > > Key: RANGER-2017 > URL: https://issues.apache.org/jira/browse/RANGER-2017 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Selvamohan Neethiraj >Assignee: bhavik patel >Priority: Critical > Fix For: 1.1.0 > > Attachments: RANGER-2017-master.patch > > > Change the default crypt algorithm to use a stronger cipher algorithm in > Ranger KMS (Similar to RANGER-1644 for Ranger Admin). Also make these options > configurable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Review Request 66357: RANGER-2017 : Ranger KMS encryption good practices
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66357/ --- Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2017 https://issues.apache.org/jira/browse/RANGER-2017 Repository: ranger Description --- Code Improvement To Follow Best Practices. Diffs - kms/config/kms-webapp/dbks-site.xml 2fc5177 kms/scripts/DBMK2HSM.sh 89c8c2d kms/scripts/HSMMK2DB.sh 2637cf6 kms/scripts/importJCEKSKeys.sh d72c93e kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16 Diff: https://reviews.apache.org/r/66357/diff/1/ Testing --- 1. Verified Ranger Kms is working as expected. 2. Import/Export of key's working as expected. Thanks, bhavik patel
FW: New Defects reported by Coverity Scan for Apache Ranger
Please review and resolve if needed. Thanks Bosco On 3/29/18, 12:58 AM, "scan-ad...@coverity.com" wrote: Hi, Please find the latest report on new defect(s) introduced to Apache Ranger found with Coverity Scan. 2 new defect(s) introduced to Apache Ranger found with Coverity Scan. 2 defect(s), reported by Coverity Scan earlier, were marked fixed in the recent build analyzed by Coverity Scan. New defect(s) Reported-by: Coverity Scan Showing 2 of 2 defect(s) ** CID 175091: FindBugs: Internationalization (FB.DM_DEFAULT_ENCODING) /plugin-kylin/src/test/java/org/apache/ranger/authorization/kylin/authorizer/RangerAdminClientImpl.java: 68 in org.apache.ranger.authorization.kylin.authorizer.RangerAdminClientImpl.getServicePoliciesIfUpdated(long, long)() *** CID 175091: FindBugs: Internationalization (FB.DM_DEFAULT_ENCODING) /plugin-kylin/src/test/java/org/apache/ranger/authorization/kylin/authorizer/RangerAdminClientImpl.java: 68 in org.apache.ranger.authorization.kylin.authorizer.RangerAdminClientImpl.getServicePoliciesIfUpdated(long, long)() 62 } 63 64 java.nio.file.Path cachePath = FileSystems.getDefault() 65 .getPath(basedir, "/src/test/resources/" + cacheFilename); 66 byte[] cacheBytes = Files.readAllBytes(cachePath); 67 >>> CID 175091: FindBugs: Internationalization (FB.DM_DEFAULT_ENCODING) >>> Found reliance on default encoding: new String(byte[]). 68 return gson.fromJson(new String(cacheBytes), ServicePolicies.class); 69 } 70 71 @Override 72 public void grantAccess(GrantRevokeRequest request) throws Exception { 73 ** CID 175090: Null pointer dereferences (FORWARD_NULL) *** CID 175090: Null pointer dereferences (FORWARD_NULL) /security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java: 2601 in org.apache.ranger.rest.ServiceREST.getServicePolicies(java.lang.String, org.apache.ranger.plugin.util.SearchFilter)() 2595 filter.setStartIndex(savedStartIndex); 2596filter.setMaxRows(savedMaxRows); 2597} 2598 2599servicePolicies = applyAdminAccessFilter(servicePolicies); 2600 >>> CID 175090: Null pointer dereferences (FORWARD_NULL) >>> Passing null pointer "filter" to "toRangerPolicyList", which dereferences it. 2601return toRangerPolicyList(servicePolicies, filter); 2602} 2603} catch(WebApplicationException excp) { 2604throw excp; 2605} catch (Throwable excp) { 2606LOG.error("getServicePolicies(" + serviceName + ") failed", excp); To view the defects in Coverity Scan visit, https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F-2BfV0V05UPxvVjWch-2Bd2MGckcRZSbhom32dlDl11LWEm9nX11zsOWMf5dv3Q9Mogo-2FGua3FsLRTFft2V-2FOFC9o0P2e0-3D_d04ZgyDzSjlwpjXIuOFYDNE6R93Lal83MDClQK32PZsnIXFWgRi957MYBuy-2FlDkytlroFFT4DodK8yWwhtaHLaX8VDTgQexZ1KpM2ghdONIK2s05PzHFJPHt35agqZ9nM-2FaTS-2BwGKR0yiHd9VqwUDlZAGDFlrUyAnUh6f6i-2B3anlkbZqOzVnGOBWkfRi-2BewYxNrjphgpV2suFoBTWVzO0g-3D-3D To manage Coverity Scan email notifications for "bo...@apache.org", click https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F-2BfV0V05UPxvVjWch-2Bd2MGckcRbVDbis712qZDP-2FA8y06Nq4HK0JLY-2BbZ-2FD0yvjg-2BbWSwquqqdEYtbR9nIDW-2BM81kI8TiABM2LsH3tiPfMWf-2FvOsjZSWngS5IRVC-2FH5Pl4zyaK1OE6Dh-2BhR6pXASEFJKZLM-3D_d04ZgyDzSjlwpjXIuOFYDNE6R93Lal83MDClQK32PZsnIXFWgRi957MYBuy-2FlDkyKfsBgXiTb4k-2FaNGDo8qEUl-2BH63yXDNcomvZoiNiF2WHM0-2BfOOmQbx4B5UGXDs4vgM4Iijd2No-2BKhKAZ5fWyDIvixetSLwUn6Ii5x-2FoDgj-2BkICrMEOuMN9xxY3hvndcP6NWUsLHw4lI958nIxF-2BKARg-3D-3D
Re: Review Request 66294: RANGER-1985: Code changes to support follow-up tasks
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66294/ --- (Updated March 29, 2018, 4:17 p.m.) Review request for ranger, Ankita Sinha, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy. Summary (updated) - RANGER-1985: Code changes to support follow-up tasks Bugs: RANGER-1985 https://issues.apache.org/jira/browse/RANGER-1985 Repository: ranger Description --- – Add other meta-data to sync source popup (like group search first enabled) – Distinguish between full sync and incremental sync events – Show synctime, last modified time in event time format – In addition to #users/groups sync'ed from source, show added/updated users (explore this to see how best to show this without confusing users) Diffs - security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 7892a6a5 security-admin/db/mysql/patches/031-create-schema-for-usersync-audit-info.sql 2405a3ef security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 1f03978c security-admin/db/oracle/patches/031-create-schema-for-usersync-audit-info.sql e03e8937 security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 5b3f4bef security-admin/db/postgres/patches/031-create-schema-for-usersync-audit-info.sql 2227f011 security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 42a8704e security-admin/db/sqlanywhere/patches/031-create-schema-for-usersync-audit-info.sql 26d857b5 security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 7421d4d5 security-admin/db/sqlserver/patches/031-create-schema-for-usersync-audit-info.sql 362f4152 security-admin/src/main/java/org/apache/ranger/entity/XXUgsyncAuditInfo.java 56098338 security-admin/src/main/java/org/apache/ranger/service/XUgsyncAuditInfoService.java 85a96512 security-admin/src/main/java/org/apache/ranger/service/XUgsyncAuditInfoServiceBase.java bf6f6e15 security-admin/src/main/java/org/apache/ranger/view/VXFileSyncSourceInfo.java d8c66ca1 security-admin/src/main/java/org/apache/ranger/view/VXLdapSyncSourceInfo.java fd2d8e11 security-admin/src/main/java/org/apache/ranger/view/VXUgsyncAuditInfo.java 6fef928a security-admin/src/main/java/org/apache/ranger/view/VXUnixSyncSourceInfo.java 47bd839d ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapDeltaUserGroupBuilder.java 7d85f33b ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapPolicyMgrUserGroupBuilder.java 54a519a9 ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapUserGroupBuilder.java 85cba3c5 ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java 45eeb1bc ugsync/src/main/java/org/apache/ranger/unixusersync/model/FileSyncSourceInfo.java 02387e11 ugsync/src/main/java/org/apache/ranger/unixusersync/model/LdapSyncSourceInfo.java 318138d4 ugsync/src/main/java/org/apache/ranger/unixusersync/model/UgsyncAuditInfo.java 9dfd3a55 ugsync/src/main/java/org/apache/ranger/unixusersync/model/UnixSyncSourceInfo.java 97b95a1f ugsync/src/main/java/org/apache/ranger/unixusersync/process/FileSourceUserGroupBuilder.java a50b0c76 ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java 9d3112dd ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java c58589ef ugsync/src/main/java/org/apache/ranger/usergroupsync/UserGroupSync.java f078cb8d ugsync/src/test/java/org/apache/ranger/usergroupsync/PolicyMgrUserGroupBuilderTest.java 312ea9b8 Diff: https://reviews.apache.org/r/66294/diff/1/ Testing --- 1. Performed functional tests with unix, file, and LDAP/AD as sync sources 2. Performed functional tests with different combinations of group search and user search with AD/LDAP as sync source 3. Also ran some functional tests with and without incremental sync and nested groups Thanks, Sailaja Polavarapu
[jira] [Comment Edited] (RANGER-1985) Auditing for Ranger Usersync operations
[ https://issues.apache.org/jira/browse/RANGER-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16414751#comment-16414751 ] Sailaja Polavarapu edited comment on RANGER-1985 at 3/29/18 4:18 PM: - Feedback items: – Show separate icon for showing sync source popup – Add other meta-data to sync source popup (like group search first enabled) – Distinguish between full sync and incremental sync events – Show synctime, last modified time in event time format – In addition to #users/groups sync'ed from source, show added/updated users (explore this to see how best to show this without confusing users) was (Author: spolavarapu): Following suggestions came up during the demo: – Show separate icon for showing sync source popup – Add other meta-data to sync source popup (like group search first enabled) – Distinguish between full sync and incremental sync events – Show synctime, last modified time in event time format – In addition to #users/groups sync'ed from source, show added/updated users (explore this to see how best to show this without confusing users) > Auditing for Ranger Usersync operations > --- > > Key: RANGER-1985 > URL: https://issues.apache.org/jira/browse/RANGER-1985 > Project: Ranger > Issue Type: New Feature > Components: Ranger, usersync >Reporter: Sailaja Polavarapu >Assignee: Sailaja Polavarapu >Priority: Major > Fix For: 1.1.0 > > Attachments: > 0001-RANGER-1985-Auditing-for-Ranger-usersync-operations.patch, > 0001-RANGER-1985-Code-changes-to-support-follow-up-tasks-.patch, > UI-RANGER-1985.patch > > > During every sync cycle, ranger usersync should audit some basic information > like number of users, number of groups that are sync'd for that cycle. Also > provide details on sync source like the unix, file, or ldap with relevant > configuration like ldap filters applied for that sync cycle, ldap host url, > etc... > Add a new tab in the ranger admin UI audits for usersync and show the above > information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (RANGER-2053) Import Ranger Policy(ies) failing with NullPointerException
Anuja Leekha created RANGER-2053: Summary: Import Ranger Policy(ies) failing with NullPointerException Key: RANGER-2053 URL: https://issues.apache.org/jira/browse/RANGER-2053 Project: Ranger Issue Type: Bug Components: Ranger Affects Versions: 1.1.0, 1.0.1 Reporter: Anuja Leekha Fix For: 1.1.0, 1.0.1 Trying to import policies through Ranger Admin UI fails with NPE. Error in logs: 2018-03-27 21:05:45,862 [http-bio-6080-exec-27] ERROR org.apache.ranger.rest.ServiceREST (ServiceREST.java:2185) - Error while importing policy from file!! java.lang.NullPointerException at org.apache.ranger.common.RangerSearchUtil.getSearchFilter(RangerSearchUtil.java:48) at org.apache.ranger.rest.ServiceREST.getServicePoliciesByName(ServiceREST.java:2541) at org.apache.ranger.rest.ServiceREST.deletePoliciesProvidedInServiceMap(ServiceREST.java:2315) at org.apache.ranger.rest.ServiceREST.importPoliciesFromFile(ServiceREST.java:2092) at org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (RANGER-2053) Import Ranger Policy(ies) failing with NullPointerException
[ https://issues.apache.org/jira/browse/RANGER-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal reassigned RANGER-2053: --- Assignee: Pradeep Agrawal > Import Ranger Policy(ies) failing with NullPointerException > --- > > Key: RANGER-2053 > URL: https://issues.apache.org/jira/browse/RANGER-2053 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 1.1.0, 1.0.1 >Reporter: Anuja Leekha >Assignee: Pradeep Agrawal >Priority: Critical > Fix For: 1.1.0, 1.0.1 > > > Trying to import policies through Ranger Admin UI fails with NPE. > Error in logs: > 2018-03-27 21:05:45,862 [http-bio-6080-exec-27] ERROR > org.apache.ranger.rest.ServiceREST (ServiceREST.java:2185) - Error while > importing policy from file!! > java.lang.NullPointerException > at > org.apache.ranger.common.RangerSearchUtil.getSearchFilter(RangerSearchUtil.java:48) > at > org.apache.ranger.rest.ServiceREST.getServicePoliciesByName(ServiceREST.java:2541) > at > org.apache.ranger.rest.ServiceREST.deletePoliciesProvidedInServiceMap(ServiceREST.java:2315) > at > org.apache.ranger.rest.ServiceREST.importPoliciesFromFile(ServiceREST.java:2092) > at > org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke() > at > org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) > at > org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2053) Import Ranger Policy(ies) failing with NullPointerException
[ https://issues.apache.org/jira/browse/RANGER-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419506#comment-16419506 ] Pradeep Agrawal commented on RANGER-2053: - This issue should be resolved after the commit of RANGER-2016, I will test again for the given case and provide updates. > Import Ranger Policy(ies) failing with NullPointerException > --- > > Key: RANGER-2053 > URL: https://issues.apache.org/jira/browse/RANGER-2053 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 1.1.0, 1.0.1 >Reporter: Anuja Leekha >Assignee: Pradeep Agrawal >Priority: Critical > Fix For: 1.1.0, 1.0.1 > > > Trying to import policies through Ranger Admin UI fails with NPE. > Error in logs: > 2018-03-27 21:05:45,862 [http-bio-6080-exec-27] ERROR > org.apache.ranger.rest.ServiceREST (ServiceREST.java:2185) - Error while > importing policy from file!! > java.lang.NullPointerException > at > org.apache.ranger.common.RangerSearchUtil.getSearchFilter(RangerSearchUtil.java:48) > at > org.apache.ranger.rest.ServiceREST.getServicePoliciesByName(ServiceREST.java:2541) > at > org.apache.ranger.rest.ServiceREST.deletePoliciesProvidedInServiceMap(ServiceREST.java:2315) > at > org.apache.ranger.rest.ServiceREST.importPoliciesFromFile(ServiceREST.java:2092) > at > org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke() > at > org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) > at > org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-1823) Allowed TRUNCATE and INSERT to partition table when the policy item is only "SELECT"
[ https://issues.apache.org/jira/browse/RANGER-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16420064#comment-16420064 ] Haihui Xu commented on RANGER-1823: --- [~rmani] This may be about with the vision of hive. I had test it using ranger-0.5.3 and apache hive-1.1.0 > Allowed TRUNCATE and INSERT to partition table when the policy item is only > "SELECT" > > > Key: RANGER-1823 > URL: https://issues.apache.org/jira/browse/RANGER-1823 > Project: Ranger > Issue Type: Bug > Components: plugins >Affects Versions: 0.5.3, 0.7.1 >Reporter: Haihui Xu >Assignee: Haihui Xu >Priority: Major > Fix For: 1.1.0 > > Attachments: RANGER-1823-Allowed TRUNCATE and INSERT to partition > table when the policy item is only SELECT.patch, Screen Shot 2018-01-24 at > 12.59.55 PM.png, clipboard.png > > > In beeline, useraa create table hive_test, such as : CREATE TABLE > hive_test(b string) PARTITIONED BY (a string); then in ranger admin UI add > a policy for userbb, the policy has only "SELECT" of table hive_test; in > beeline userbb execute "truncate table hive_test" / "insert into hive_test > partition(a=20171003) select 1 from hive_test" is allowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)