[
https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558162#action_12558162
]
Mukund Madhugiri commented on HADOOP-1650:
------------------------------------------
I did a re-run on the 500 node cluster and don't see the
NotReplicatedYetException and SocketTimeoutException. I do see the
OutOfMemoryError when the sort job is running:
2008-01-11 20:54:10,375 INFO org.apache.hadoop.mapred.TaskInProgress: Error
from task_200801112025_0002_m_005337_0: java.lang.OutOfMemoryError: Java heap
space
at java.util.Arrays.copyOf(Arrays.java:2786)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.io.BytesWritable.write(BytesWritable.java:137)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:373)
at
org.apache.hadoop.mapred.lib.IdentityMapper.map(IdentityMapper.java:40)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:208)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2043)
Here is the sort data:
|*500 nodes*|*trunk*|*trunk + patch*|
|randomWriter (mins)|27|25|
|sort (mins)|86|107|
|sortValidation (mins)|22|21|
> Upgrade Jetty to 6.x
> --------------------
>
> Key: HADOOP-1650
> URL: https://issues.apache.org/jira/browse/HADOOP-1650
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Attachments: hadoop-1650-jetty6.1.5.patch,
> hadoop-jetty6.1.4-lib.tar.gz, hadoop-jetty6.1.6-lib.tar.gz,
> jetty-hadoop-6.1.6.patch, jetty-hbase.patch, jetty6.1.4.patch,
> jetty6.1.6.patch
>
>
> This is the third attempt at moving to jetty6. Apparently, the jetty-6.1.4
> has fixed some of the issues we discovered in jetty during HADOOP-736 and
> HADOOP-1273. I'd like to keep this issue open for sometime so that we have
> enough time to test out things.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
