[ http://issues.apache.org/jira/browse/HADOOP-195?page=all ]
Owen O'Malley updated HADOOP-195:
---------------------------------
Attachment: data-transfer-chart.pdf
Here is a chart of the data transfers from Monday's run (blue) and Tuesday's
run (red). The parallel fetches seem help most in the early part of the curve.
You can also see the substantial rework that was caused by losing the tasks on
Monday.
> transfer map output transfer with http instead of rpc
> -----------------------------------------------------
>
> Key: HADOOP-195
> URL: http://issues.apache.org/jira/browse/HADOOP-195
> Project: Hadoop
> Type: Improvement
> Components: mapred
> Versions: 0.2
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.3
> Attachments: data-transfer-chart.pdf, netstat.log, netstat.xls
>
> The data transfer of the map output should be transfered via http instead
> rpc, because rpc is very slow for this application and the timeout behavior
> is suboptimal. (server sends data and client ignores it because it took more
> than 10 seconds to be received.)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
