[
https://issues.apache.org/jira/browse/RANGER-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17407276#comment-17407276
]
caozhiqiang edited comment on RANGER-3386 at 8/31/21, 11:56 AM:
----------------------------------------------------------------
OK. I will make this changer to my company's private branch. Thank you.
was (Author: caozhiqiang):
OK. I will make this changer to my company's private branch. Thank you.
> apache CollectionUtils package reduce ranger agent performance and should be
> replaced
> -------------------------------------------------------------------------------------
>
> Key: RANGER-3386
> URL: https://issues.apache.org/jira/browse/RANGER-3386
> Project: Ranger
> Issue Type: Improvement
> Components: plugins
> Affects Versions: 2.0.1
> Reporter: caozhiqiang
> Assignee: caozhiqiang
> Priority: Major
> Attachments: CollectionUtils.png, RANGER-3386-branch-2.0.1.001.patch,
> RANGER-3386-branch-2.0.1.002.patch
>
>
> org.apache.commons.collections.CollectionUtils' performance is too poor and
> reduce the performance of ranger plugins, particularly with hdfs. There are
> too many places used CollectionUtils.isNotEmpty and CollectionUtils.isEmpty
> in agent component, so we should replace them.
> We can see many CollectionUtils call is take too much time in namenode
> benchmark's result.
> In this patch, I replace almost CollectionUtils functions in agents-common.
> After adding this patch, in creating file benchmark of hdfs, the performance
> can improve from creating *7000* files to *7600* files per second.
> By the way, I write a simple test code below. collection.isEmpty is almost
> take 0 milliseconds, but CollectionUtils.isNotEmpty take 5 milliseconds.
>
> {code:java}
> List<Integer> list = new ArrayList<Integer>();
> for(int i = 0; i<1000; i++)
> list.add(i);
> long startTime = System.currentTimeMillis();
> if(list != null && !list.isEmpty()){
> }
> long endTime = System.currentTimeMillis();
> long totalTime = endTime - startTime;
> System.out.println(totalTime);
> long startTime2 = System.currentTimeMillis();
> CollectionUtils.isNotEmpty(list);
> long endTime2 = System.currentTimeMillis();
> long totalTime2 = endTime2 - startTime2;
> System.out.println(totalTime2);
> {code}
>
> !CollectionUtils.png|width=680,height=324!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
