[ https://issues.apache.org/jira/browse/HADOOP-7761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419580#comment-13419580 ]
Todd Lipcon commented on HADOOP-7761: ------------------------------------- Hey Scott. That's interesting -- I did some initial benchmarks back when this was committed, which showed an improvement, _but_ I was benchmarking several patches at once, not just this one in isolation. Shame on me. Does anyone have time to run a terasort before/after (or simulated terasort workload in isolation) and compare CPU_TIME_MILLIS counters on the map side? > Improve performance of raw comparisons > -------------------------------------- > > Key: HADOOP-7761 > URL: https://issues.apache.org/jira/browse/HADOOP-7761 > Project: Hadoop Common > Issue Type: Improvement > Components: io, performance, util > Affects Versions: 0.23.0 > Reporter: Todd Lipcon > Assignee: Todd Lipcon > Fix For: 0.23.1 > > Attachments: hadoop-7761.txt, hadoop-7761.txt, hadoop-7761.txt, > hadoop-7761.txt > > > Guava has a nice implementation of lexicographical byte-array comparison that > uses sun.misc.Unsafe to compare unsigned byte arrays long-at-a-time. Their > benchmarks show it as being 2x more CPU-efficient than the equivalent > pure-Java implementation. We can easily integrate this into > WritableComparator.compareBytes to improve CPU performance in the shuffle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira