[ https://issues.apache.org/jira/browse/HADOOP-7761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420818#comment-13420818 ]
Scott Carey commented on HADOOP-7761: ------------------------------------- {quote} this is much slower {quote} It appears that I exaggerated. This is somewhat slower for small byte arrays and somewhat faster for larger ones on a 64 bit JVM in AVRO-939. There is more work to do to understand this and possibly improve all of it. > Improve performance of raw comparisons > -------------------------------------- > > Key: HADOOP-7761 > URL: https://issues.apache.org/jira/browse/HADOOP-7761 > Project: Hadoop Common > Issue Type: Improvement > Components: io, performance, util > Affects Versions: 0.23.0 > Reporter: Todd Lipcon > Assignee: Todd Lipcon > Fix For: 0.23.1 > > Attachments: hadoop-7761.txt, hadoop-7761.txt, hadoop-7761.txt, > hadoop-7761.txt > > > Guava has a nice implementation of lexicographical byte-array comparison that > uses sun.misc.Unsafe to compare unsigned byte arrays long-at-a-time. Their > benchmarks show it as being 2x more CPU-efficient than the equivalent > pure-Java implementation. We can easily integrate this into > WritableComparator.compareBytes to improve CPU performance in the shuffle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira