[ https://issues.apache.org/jira/browse/MAPREDUCE-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13945456#comment-13945456 ]
Hadoop QA commented on MAPREDUCE-5791: -------------------------------------- {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12636387/MAPREDUCE-5791.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4459//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4459//console This message is automatically generated. > Shuffle phase is slow in Windows - FadviseFileRegion::transferTo does not > read disks efficiently > ------------------------------------------------------------------------------------------------ > > Key: MAPREDUCE-5791 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5791 > Project: Hadoop Map/Reduce > Issue Type: Bug > Reporter: Nikola Vujic > Assignee: Nikola Vujic > Attachments: MAPREDUCE-5791.patch, MAPREDUCE-5791.patch, > MAPREDUCE-5791.patch > > > transferTo method in org.apache.hadoop.mapred.FadvisedFileRegion is using > transferTo method from a FileChannel to transfer data from a disk to socket. > This is performing slow in Windows, slower than in Linux. The reason is that > transferTo method for the java.nio is issuing 32K IO requests all the time. > In Windows, these 32K transfers are not optimal and we don't get the best > performance form the underlying IO subsystem. In order to achieve better > performance when reading from the drives, we need to read data in bigger > chunks, 512K for example. -- This message was sent by Atlassian JIRA (v6.2#6252)