[jira] [Created] (RANGER-1732) Collection added to itself
JC created RANGER-1732:
--
Summary: Collection added to itself
Key: RANGER-1732
URL: https://issues.apache.org/jira/browse/RANGER-1732
Project: Ranger
Issue Type: Bug
Components: audit
Reporter: JC
Priority: Trivial
Hi
In a recent github mirror, I've found suspicious code.
Branch: master
Path:
agents-common/src/main/java/org/apache/ranger/plugin/audit/RangerMultiResourceAuditHandler.java
{code:java}
32 Collection auditEvents = new ArrayList<>();
...
42
43 @Override
44 public void logAuthzAudits(Collection auditEvents)
{
45 auditEvents.addAll(auditEvents);
46 }
{code}
In Line 45, `auditEvents.addAll' should be `this.auditEvents.addAll'? This
might not be an issue but I wanted to report just in case.
Thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (RANGER-1733) Duplicate RangerDaoManager fields in the services
Zsombor Gegesy created RANGER-1733: -- Summary: Duplicate RangerDaoManager fields in the services Key: RANGER-1733 URL: https://issues.apache.org/jira/browse/RANGER-1733 Project: Ranger Issue Type: Bug Components: admin Affects Versions: 1.0.0 Reporter: Zsombor Gegesy Assignee: Zsombor Gegesy Fix For: 1.0.0 The AbstractBaseResourceService contains two RangerDaoManager field, daoManager and daoMgr, the various subclasses also define a couple of RangerDaoManager fields, they are unnecessarily, it can be removed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1733) Duplicate RangerDaoManager fields in the services
[ https://issues.apache.org/jira/browse/RANGER-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-1733: --- Attachment: 0001-RANGER-1733-Remove-duplicate-RangerDaoManager-fields.patch > Duplicate RangerDaoManager fields in the services > - > > Key: RANGER-1733 > URL: https://issues.apache.org/jira/browse/RANGER-1733 > Project: Ranger > Issue Type: Bug > Components: admin >Affects Versions: 1.0.0 >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy > Labels: code-cleanup, reduce > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1733-Remove-duplicate-RangerDaoManager-fields.patch > > > The AbstractBaseResourceService contains two RangerDaoManager field, > daoManager and daoMgr, the various subclasses also define a couple of > RangerDaoManager fields, they are unnecessarily, it can be removed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Review Request 61624: RANGER-1733 - remove duplicate fields from the services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61624/ --- Review request for ranger. Bugs: RANGER-1733 https://issues.apache.org/jira/browse/RANGER-1733 Repository: ranger Description --- The same RangerDaoManager is injected multiple times, unnecessarily. One fields is more than enough. Diffs - security-admin/src/main/java/org/apache/ranger/service/AbstractBaseResourceService.java 3c2f263 security-admin/src/main/java/org/apache/ranger/service/AuthSessionService.java 249a695 security-admin/src/main/java/org/apache/ranger/service/UserService.java 91fb38b security-admin/src/main/java/org/apache/ranger/service/XAccessAuditService.java 8604bed security-admin/src/main/java/org/apache/ranger/service/XAssetService.java 3aab900 security-admin/src/main/java/org/apache/ranger/service/XAuditMapService.java e090391 security-admin/src/main/java/org/apache/ranger/service/XGroupPermissionService.java 6329e6b security-admin/src/main/java/org/apache/ranger/service/XGroupService.java 1253eec security-admin/src/main/java/org/apache/ranger/service/XGroupUserService.java 52f8b1a security-admin/src/main/java/org/apache/ranger/service/XModuleDefService.java f66cdfe security-admin/src/main/java/org/apache/ranger/service/XPermMapService.java 1a59975 security-admin/src/main/java/org/apache/ranger/service/XResourceService.java 004ded4 security-admin/src/main/java/org/apache/ranger/service/XTrxLogService.java 7618981 security-admin/src/main/java/org/apache/ranger/service/XUserPermissionService.java 76b7400 security-admin/src/main/java/org/apache/ranger/service/XUserService.java 294223b Diff: https://reviews.apache.org/r/61624/diff/1/ Testing --- Tested locally. https://travis-ci.org/gzsombor/ranger/builds/264508518 Thanks, Zsombor Gegesy
[jira] [Commented] (RANGER-1732) Collection added to itself
[
https://issues.apache.org/jira/browse/RANGER-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126682#comment-16126682
]
Don Bosco Durai commented on RANGER-1732:
-
Good catch. Let me know if you want to fix it? Else, I can fix it. Thanks
> Collection added to itself
> --
>
> Key: RANGER-1732
> URL: https://issues.apache.org/jira/browse/RANGER-1732
> Project: Ranger
> Issue Type: Bug
> Components: audit
>Reporter: JC
>Priority: Trivial
>
> Hi
> In a recent github mirror, I've found suspicious code.
> Branch: master
> Path:
> agents-common/src/main/java/org/apache/ranger/plugin/audit/RangerMultiResourceAuditHandler.java
> {code:java}
> 32 Collection auditEvents = new ArrayList<>();
> ...
> 42
> 43 @Override
> 44 public void logAuthzAudits(Collection
> auditEvents) {
> 45 auditEvents.addAll(auditEvents);
> 46 }
> {code}
> In Line 45, `auditEvents.addAll' should be `this.auditEvents.addAll'? This
> might not be an issue but I wanted to report just in case.
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
FW: New Defects reported by Coverity Scan for Apache Ranger
Contributors/Committers, please review if any of your commit is giving these
errors. And fix them if required.
Thanks
Bosco
On 8/13/17, 1:07 AM, "scan-ad...@coverity.com" wrote:
Hi,
Please find the latest report on new defect(s) introduced to Apache Ranger
found with Coverity Scan.
10 new defect(s) introduced to Apache Ranger found with Coverity Scan.
1 defect(s), reported by Coverity Scan earlier, were marked fixed in the
recent build analyzed by Coverity Scan.
New defect(s) Reported-by: Coverity Scan
Showing 10 of 10 defect(s)
** CID 166304: Incorrect expression (USELESS_CALL)
*** CID 166304: Incorrect expression (USELESS_CALL)
/security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java: 851
in org.apache.ranger.biz.TestXUserMgr.test30CreateVXUserGroupInfo()()
845 Assert.assertEquals("user1",
vxUserGroupTest.getXuserInfo().getName());
846 List result = vxUserGroupTest.getXgroupInfo();
847 List expected = new ArrayList();
848 expected.add(vXGroup1);
849 expected.add(vXGroup2);
850 Assert.assertTrue(result.containsAll(expected));
>>> CID 166304: Incorrect expression (USELESS_CALL)
>>> Calling
"(org.apache.ranger.db.RangerDaoManager)org.mockito.Mockito.verify(daoManager).getXXPortalUser()"
is only useful for its return value, which is ignored.
851 Mockito.verify(daoManager).getXXPortalUser();
852 Mockito.verify(portalUser).findByLoginId(vXUser.getName());
853 Mockito.verify(daoManager).getXXPortalUserRole();
854 Mockito.verify(userDao).findXPortalUserRolebyXPortalUserId(
855 Mockito.anyLong());
856
** CID 166303: High impact security (CSRF)
/security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java: 178 in
org.apache.ranger.rest.XUserREST.createXGroupUserFromMap(org.apache.ranger.view.VXGroupUserInfo)()
*** CID 166303: High impact security (CSRF)
/security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java: 178 in
org.apache.ranger.rest.XUserREST.createXGroupUserFromMap(org.apache.ranger.view.VXGroupUserInfo)()
172 }
173
174 @POST
175 @Path("/groups/groupinfo")
176 @Produces({ "application/xml", "application/json" })
177 @PreAuthorize("hasRole('ROLE_SYS_ADMIN')")
>>> CID 166303: High impact security (CSRF)
>>> No CSRF protection was detected anywhere in this application. If
this is not correct, please refer to the CSRF checker reference on how to
specify it via checker option.
178 public VXGroupUserInfo createXGroupUserFromMap(VXGroupUserInfo
vXGroupUserInfo) {
179 return
xUserMgr.createXGroupUserFromMap(vXGroupUserInfo);
180 }
181
182 @POST
183 @Path("/secure/groups")
** CID 166302: Exceptional resource leaks (RESOURCE_LEAK)
/hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java:
908 in
org.apache.ranger.services.hive.HIVERangerAuthorizerTest.testShowPrivileges()()
*** CID 166302: Exceptional resource leaks (RESOURCE_LEAK)
/hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java:
908 in
org.apache.ranger.services.hive.HIVERangerAuthorizerTest.testShowPrivileges()()
902
903 @Test
904 public void testShowPrivileges() throws Exception {
905 String initialUrl = "jdbc:hive2://localhost:" + port;
906 Connection connection =
DriverManager.getConnection(initialUrl, "admin", "admin");
907 Statement statement = connection.createStatement();
>>> CID 166302: Exceptional resource leaks (RESOURCE_LEAK)
>>> Variable "statement" going out of scope leaks the resource it
refers to.
908 Assert.assertTrue(statement.execute("show grant user
admin"));
909 statement.close();
910 }
911
** CID 166301: Incorrect expression (USELESS_CALL)
*** CID 166301: Incorrect expression (USELESS_CALL)
/security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java: 853
in org.apache.ranger.biz.TestXUserMgr.test30CreateVXUserGroupInfo()()
847 List expected = new ArrayList();
848
[jira] [Commented] (RANGER-1729) The hbase's performance will be significantly reduced after used Ranger to control and configure Hbase data security
[ https://issues.apache.org/jira/browse/RANGER-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126704#comment-16126704 ] Don Bosco Durai commented on RANGER-1729: - Hi [~zhangqiang2], when you are doing benchmarking with HBase there are few things you need to be aware of: 1. When you do load/performance testing (without Ranger), HBase will use all available CPU 2. HBase also does a lot of IO (for put operation) 3. When Ranger is enabled, a slice of processing is consumed by Ranger. But since every operation calls Ranger, there is a visible performance difference. 4. Considering enabling Audit is only affected the performance by 2.8% is pretty good. Consider that every read in HBase that could translate to write (for audit). Since HBase is super optimized for READ, audit can never outperform HBase itself. However, Ranger does optimization by consolidating/summarizing the audits and minimizing writes. I would suggest few things: 1. Come up with what would be an acceptable overhead for security. In the case of HBase and Kafka, it will be never free, because these service are super optimized for performance. So you need to be reasonable here. 2. Enable native HBase access control and see what is the difference wrt when Ranger is enabled 3. Modify the Ranger code to return "true" when the co-processor is called. This will help in understanding what are the overhead of co-processor and other non-Ranger code. Once we have this number, we can look into Ranger code. 4. Increase the time interval for consolidating/batch Ranger Audits. The default is 5 seconds. You can try changing this property in HBase xasecure.audit.provider.summary.interval.ms to 3 > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security > > > Key: RANGER-1729 > URL: https://issues.apache.org/jira/browse/RANGER-1729 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Attachments: Polling_interval_result_detail.png, result_detail.png > > > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security. I used following environment to > verify issue. > Test tools：ycsb-0.1.4 > Test environment： > Node number：4 > Node configuration detail is as following: > Node1、Node2: > CPU:32 core，Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz > memory：128GB > Node3、Node4 > CPU:48 core，Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz > memory：128GB > The test result: > 2. Add one policy and not write audit log, HBase's performance decreased > 10.10%; > 3. Add one policy and write audit log, HBase's performance decreased 12.90%; > Please refer to result_detail.png. > This decline is unacceptable in a real environment. It will seriously affect > the user to use Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1729) The hbase's performance will be significantly reduced after used Ranger to control and configure Hbase data security
[ https://issues.apache.org/jira/browse/RANGER-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126814#comment-16126814 ] Qiang Zhang commented on RANGER-1729: - Hi [~bosco], You are justified, but the results are unacceptable in real big project, because the performance decline is too large, more than 10%. I'm trying to optimize this performance. If there is a problem, I will talk with you further. > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security > > > Key: RANGER-1729 > URL: https://issues.apache.org/jira/browse/RANGER-1729 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Attachments: Polling_interval_result_detail.png, result_detail.png > > > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security. I used following environment to > verify issue. > Test tools：ycsb-0.1.4 > Test environment： > Node number：4 > Node configuration detail is as following: > Node1、Node2: > CPU:32 core，Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz > memory：128GB > Node3、Node4 > CPU:48 core，Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz > memory：128GB > The test result: > 2. Add one policy and not write audit log, HBase's performance decreased > 10.10%; > 3. Add one policy and write audit log, HBase's performance decreased 12.90%; > Please refer to result_detail.png. > This decline is unacceptable in a real environment. It will seriously affect > the user to use Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
