[
https://issues.apache.org/jira/browse/HADOOP-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12554984
]
Hadoop QA commented on HADOOP-1719:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12372152/HADOOP-1719.patch
against trunk revision r607330.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests -1. The patch failed contrib unit tests.
Test results:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1431/testReport/
Findbugs warnings:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1431/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1431/artifact/trunk/build/test/checkstyle-errors.html
Console output:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1431/console
This message is automatically generated.
> Improve the utilization of shuffle copier threads
> -------------------------------------------------
>
> Key: HADOOP-1719
> URL: https://issues.apache.org/jira/browse/HADOOP-1719
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Devaraj Das
> Assignee: Amar Kamat
> Attachments: 1719.1.patch, 1719.patch, HADOOP-1719.patch,
> HADOOP-1719.patch
>
>
> In the current design, the scheduling of copies is done and the scheduler
> (the main loop in fetchOutputs) won't schedule anything until it hears back
> from at least one of the copier threads. Due to this, the main loop won't
> query the TaskTracker asking for new map locations and may not be using all
> the copiers effectively. This may not be an issue for small-sized map
> outputs, where at steady state, the frequency of such notifications is
> frequent.
> Ideally, we should schedule all what we can, and, depending on how busy we
> currently are, query the tasktracker for more map locations.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
