Hi Guys,
Currently we are running the small Hadoop(1.2.1) cluster with 13 nodes,
today we getting OutOfMemory error in jobtracker,
java.io.IOException: Call to nn:8020 failed on local exception:
java.io.IOException: Couldn't set up IO streams
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1150)
at org.apache.hadoop.ipc.Client.call(Client.java:1118)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at $Proxy7.renewLease(Unknown Source)
at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at $Proxy7.renewLease(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:379)
at org.apache.hadoop.hdfs.LeaseRenewer.renew(LeaseRenewer.java:378)
at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:400)
at
org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69)
at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Couldn't set up IO streams
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:642)
at
org.apache.hadoop.ipc.Client$Connection.access$2200(Client.java:205)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1249)
at org.apache.hadoop.ipc.Client.call(Client.java:1093)
... 14 more
Caused by: java.lang.OutOfMemoryError: Java heap space
-----------------------------------------------------------------------------------------------------------------
2013-09-15 13:13:31,198 ERROR org.apache.hadoop.mapred.JobTracker: Job
initialization failed:
java.lang.OutOfMemoryError: Java heap space
at
com.sun.org.apache.xml.internal.serializer.ToUnknownStream.characters(ToUnknownStream.java:341)
at
com.sun.org.apache.xalan.internal.xsltc.trax.DOM2TO.parse(DOM2TO.java:240)
at
com.sun.org.apache.xalan.internal.xsltc.trax.DOM2TO.parse(DOM2TO.java:226)
at
com.sun.org.apache.xalan.internal.xsltc.trax.DOM2TO.parse(DOM2TO.java:226)
at
com.sun.org.apache.xalan.internal.xsltc.trax.DOM2TO.parse(DOM2TO.java:226)
at
com.sun.org.apache.xalan.internal.xsltc.trax.DOM2TO.parse(DOM2TO.java:132)
Please help to resolve this issue asap.
What are the best mapred/hadoop core configuration to resolve this.
--
Regards,
Viswa.J
