Hi guys,
Nodemanager process error log:
Could you give me some tips for this , why happened?
2015-12-16 17:24:38,605 WARN
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.Container
sMonitorImpl: Uncaught exception in ContainerMemoryManager while managing
memory of container_1448915696877_23390_01_000430
java.lang.ArrayIndexOutOfBoundsException
at
sun.misc.FormattedFloatingDecimal.dtoa(FormattedFloatingDecimal.java:848)
at
sun.misc.FormattedFloatingDecimal.<init>(FormattedFloatingDecimal.java:542)
at java.util.Formatter$FormatSpecifier.print(Formatter.java:3264)
at java.util.Formatter$FormatSpecifier.print(Formatter.java:3202)
at
java.util.Formatter$FormatSpecifier.printFloat(Formatter.java:2769)
at java.util.Formatter$FormatSpecifier.print(Formatter.java:2720)
at java.util.Formatter.format(Formatter.java:2500)
at java.util.Formatter.format(Formatter.java:2435)
at java.lang.String.format(String.java:2148)
at org.apache.hadoop.util.StringUtils.format(StringUtils.java:123)
at
org.apache.hadoop.util.StringUtils$TraditionalBinaryPrefix.long2String(Strin
gUtils.java:758)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.Container
sMonitorImpl$MonitoringThread.formatUsageString(ContainersMonitorImpl.java:4
87)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.Container
sMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:399)
2015-12-16 17:24:39,663 ERROR org.apache.hadoop.mapred.ShuffleHandler:
Shuffle error:
java.lang.ArrayIndexOutOfBoundsException
at sun.nio.cs.UTF_8$Encoder.encode(UTF_8.java:652)
at java.lang.StringCoding.encode(StringCoding.java:899)
at java.lang.String.getBytes(String.java:2221)
at
org.apache.hadoop.mapreduce.security.SecureShuffleUtils.verifyReply(SecureSh
uffleUtils.java:100)
at
org.apache.hadoop.mapred.ShuffleHandler$Shuffle.verifyRequest(ShuffleHandler
.java:558)
at
org.apache.hadoop.mapred.ShuffleHandler$Shuffle.messageReceived(ShuffleHandl
er.java:499)
at
org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWri
teHandler.java:142)
at
org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpC
hunkAggregator.java:148)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceive
d(FrameDecoder.java:459)
at
org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDe
coder.java:536)
at
org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(Replay
ingDecoder.java:435)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWork
er.java:107)
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelect
or.java:312)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.j
ava:88)
at
org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
57)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
27)
at java.lang.Thread.run(Thread.java:809)
2015-12-16 17:24:39,663 ERROR org.apache.hadoop.mapred.ShuffleHandler:
Shuffle error [id: 0x34ed4c97, /172.19.206.148:34641 =>
/172.19.206.142:8080] EXCEPTION: java.lang.ArrayIndexOutOfBoundsException
2015-12-16 17:24:39,663 FATAL
org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread Thread[Container
Monitor,5,main] threw an Error. Shutting down now...
java.lang.OutOfMemoryError: Java 堆空间(heapspace)
at java.util.HashMap.inflateTable(HashMap.java:328)
at java.util.HashMap.<init>(HashMap.java:308)
at
org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.updateProcessTree(ProcfsB
asedProcessTree.java:154)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.Container
sMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:390)
2015-12-16 17:24:39,666 INFO org.apache.hadoop.util.ExitUtil: Halt with
status -1 Message: HaltException
Best Regards,
Evan Yao
