[
http://issues.apache.org/jira/browse/HADOOP-401?page=comments#action_12425817 ]
Marco Nicosia commented on HADOOP-401:
--------------------------------------
And again, this time in VMThread!
Exception in thread "Server handler 3 on 50020"
java.lang.IncompatibleClassChangeError
at java.util.HashSet.iterator(HashSet.java:154)
at sun.nio.ch.SelectorImpl.processDeregisterQueue(SelectorImpl.java:127)
at sun.nio.ch.PollSelectorImpl.doSelect(PollSelectorImpl.java:53)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
at
org.mortbay.http.nio.SocketChannelOutputStream.flushBuffer(SocketChannelOutputStream.java:149)
at
org.mortbay.http.nio.SocketChannelOutputStream.write(SocketChannelOutputStream.java:103)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:487)
#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# SIGSEGV (0xb) at pc=0xf76e0b48, pid=921, tid=1890360240
#
# Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
# Problematic frame:
# V [libjvm.so+0x205b48]
#
# An error report file with more information is saved as hs_err_pid921.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
And the hs_err_pid921.log file:
#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# SIGSEGV (0xb) at pc=0xf76e0b48, pid=921, tid=1890360240
#
# Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
# Problematic frame:
# V [libjvm.so+0x205b48]
#
--------------- T H R E A D ---------------
Current thread (0x081111b8): VMThread [id=931]
siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x00330037
Registers:
EAX=0x00330037, EBX=0xf7a867f0, ECX=0x00000000, EDX=0x0033003f
ESP=0x70ac8eb0, EBP=0x70ac8ec8, ESI=0x70ac8ee0, EDI=0x70ac8ef0
EIP=0xf76e0b48, CR2=0x00330037, EFLAGS=0x00010246
Top of Stack: (sp=0x70ac8eb0)
0x70ac8eb0: 00000000 70ac8ee8 00000000 f7a867f0
0x70ac8ec0: 70ac8ef0 70ac8ee0 70ac9018 f79a001c
0x70ac8ed0: 70ac8ee0 70ac8ef0 70ac8efc 00000000
0x70ac8ee0: 79880008 00000000 00330037 70ac8f48
0x70ac8ef0: 79880008 00000000 00330037 6f0d9040
0x70ac8f00: 6f0d9038 6f0d9084 6f0d9088 6f0d9044
0x70ac8f10: 6f0d903c 6f0d9080 70ac8f88 f78e6c04
0x70ac8f20: 70ac8f40 6f0d8fac 6f0d8fb0 6f0d8fb4
Instructions: (pc=0xf76e0b48)
0xf76e0b38: 00 00 00 8b 45 f0 85 c0 74 15 8b 47 08 8b 50 fc
0xf76e0b48: 8b 38 8b 40 04 89 46 04 89 16 89 7e 08 eb b4 8b
Stack: [0x70a49000,0x70aca000), sp=0x70ac8eb0, free space=511k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x205b48]
V [libjvm.so+0x4c501c]
V [libjvm.so+0x4c7629]
V [libjvm.so+0x4a467f]
V [libjvm.so+0x4870b7]
V [libjvm.so+0x504e8b]
V [libjvm.so+0x504850]
V [libjvm.so+0x424338]
C [libpthread.so.0+0x5341]
--------------- P R O C E S S ---------------
[... most threads in _thread_blocked state]
Other Threads:
=>0x081111b8 VMThread [id=931]
0x08123170 WatcherThread [id=939]
VM state:at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x0805d368/0x0805d390] Safepoint_lock - owner thread: 0x081111b8
[0x0805d3e8/0x0805d410] Threads_lock - owner thread: 0x081111b8
Heap
PSYoungGen total 196288K, used 71209K [0xe4640000, 0xf2460000, 0xf2470000)
eden space 181696K, 35% used [0xe4640000,0xe85be468,0xef7b0000)
from space 14592K, 42% used [0xef7b0000,0xefdbc000,0xf05f0000)
to space 14208K, 0% used [0xf1680000,0xf1680000,0xf2460000)
PSOldGen total 359232K, used 347217K [0x75470000, 0x8b340000,
0xe4640000)
object space 359232K, 96% used [0x75470000,0x8a784528,0x8b340000)
PSPermGen total 16384K, used 8009K [0x71470000, 0x72470000, 0x75470000)
object space 16384K, 48% used [0x71470000,0x71c42470,0x72470000)
[...]
VM Arguments:
jvm_args: -Xmx2000m -Dhadoop.log.dir=/e/c/k/hadoop/current/../run2/log
-Dhadoop.log.file=hadoop-crawler-jobtracker-kry
1190.log -Dhadoop.home.dir=/e/c/k/hadoop/current -Dhadoop.id.str=crawler
-Dhadoop.root.logger=INFO,DRFA
java_command: org.apache.hadoop.mapred.JobTracker
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=/e/c/k/java/jdk
PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin
LD_LIBRARY_PATH=/e/c/k/java/jdk1.5.0_06/jre/lib/i386/server:/e/c/k/java/jdk1.5.0_06/jre/lib/i386:/e/c/k/java/jdk1.5.0_06/jre/../lib/i386
SHELL=/bin/tcsh
HOSTTYPE=x86_64-linux
OSTYPE=linux
MACHTYPE=x86_64
> JobTracker crashes Sun JVM
> --------------------------
>
> Key: HADOOP-401
> URL: http://issues.apache.org/jira/browse/HADOOP-401
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.4.0
> Environment: OS:Red Hat Enterprise Linux AS release 4 (Nahant Update
> 2)
> uname:Linux 2.6.9-22.12.y1-64 #5 SMP Fri Nov 4 13:39:56 PST 2005 x86_64
> libc:glibc 2.3.4 NPTL 2.3.4
> rlimit: STACK 10240k, CORE 0k, NPROC 32768, NOFILE 32768, AS infinity
> load average:0.20 0.21 0.18
> CPU:total 4 family 47, cmov, cx8, fxsr, mmx, sse, sse2, ht
> Memory: 4k page, physical 988k(220k free), swap 3070k(3070k free)
> vm_info: Java HotSpot(TM) Server VM (1.5.0_06-b05) for linux-x86, built on
> Nov 10 2005 10:56:33 by java_re with gcc 3.2.1-7a (J2SE release)
> Reporter: Marco Nicosia
>
> From our jobtacker's .out file:
> #
> # An unexpected error has been detected by HotSpot Virtual Machine:
> #
> # SIGSEGV (0xb) at pc=0xf27bf825, pid=24028, tid=1856498608
> #
> # Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
> # Problematic frame:
> # J org.mortbay.util.ByteArrayPool.getByteArray(I)[B
> #
> # An error report file with more information is saved as hs_err_pid24028.log
> #
> # If you would like to submit a bug report, please visit:
> # http://java.sun.com/webapps/bugreport/crash.jsp
> #
> And the relevant parts of that hs_err_pid24028.log file:
> #
> # An unexpected error has been detected by HotSpot Virtual Machine:
> #
> # SIGSEGV (0xb) at pc=0xf27bf825, pid=24028, tid=1856498608
> #
> # Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
> # Problematic frame:
> # J org.mortbay.util.ByteArrayPool.getByteArray(I)[B
> #
> --------------- T H R E A D ---------------
> Current thread (0x086e0088): JavaThread "SocketListener0-0"
> [_thread_in_Java, id=24112]
> siginfo:si_signo=11, si_errno=0, si_code=2, si_addr=0x00022008
> Registers:
> EAX=0x00022000, EBX=0x75a0fac8, ECX=0x00000000, EDX=0x75b31258
> ESP=0x6ea7ded0, EBP=0x6ea7df3c, ESI=0x71b6def5, EDI=0x00000402
> EIP=0xf27bf825, CR2=0x00022008, EFLAGS=0x00010217
> Top of Stack: (sp=0x6ea7ded0)
> 0x6ea7ded0: 71549561 6ea7def0 00002000 00000400
> 0x6ea7dee0: 339dc50d 086e0088 6ea7df3c 6ea7df50
> 0x6ea7def0: 00000000 71b6def5 7154d7c9 f2595a80
> 0x6ea7df00: 6ea7df3c 6ea7df50 7154d7d8 6ea7df14
> 0x6ea7df10: 6ea7df3c f2501a14 00002000 e57bd350
> 0x6ea7df20: 6ea7df20 71b6def5 6ea7df50 71b71798
> 0x6ea7df30: 00000000 71b6df80 6ea7df44 6ea7df78
> 0x6ea7df40: f2501aeb 719431e8 00002000 e57bd2d0
> Instructions: (pc=0xf27bf825)
> 0xf27bf815: 8b 98 a4 00 00 00 8b 4b 20 89 4c 24 20 8b 41 08
> 0xf27bf825: 8b 70 08 8b de 4b 8b 4c 24 10 23 cb 89 4c 24 10
> Stack: [0x6e9fe000,0x6ea7f000), sp=0x6ea7ded0, free space=511k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> J org.mortbay.util.ByteArrayPool.getByteArray(I)[B
> j org.mortbay.http.HttpInputStream.<init>(Ljava/io/InputStream;I)V+15
> j
> org.mortbay.http.HttpConnection.<init>(Lorg/mortbay/http/HttpListener;Ljava/net/InetAddress;Ljava/io/InputStream;Ljava/io/OutputStream;Ljava/lang/Object;)V+97
> j
> org.mortbay.http.SocketListener.createConnection(Ljava/net/Socket;)Lorg/mortbay/http/HttpConnection;+18
> j org.mortbay.http.SocketListener.handleConnection(Ljava/net/Socket;)V+2
> j org.mortbay.util.ThreadedServer.handle(Ljava/lang/Object;)V+19
> --------------- P R O C E S S ---------------
> Java Threads: ( => current thread )
> 0x086bb8a0 JavaThread "Thread-0" [_thread_blocked, id=24117]
> 0x0859dd38 JavaThread "Thread-20" [_thread_blocked, id=24116]
> 0x0859a8a8 JavaThread "Thread-19" [_thread_blocked, id=24115]
> 0x0859a1f8 JavaThread "Thread-18" [_thread_blocked, id=24114]
> 0x085d54d8 JavaThread "SocketListener0-1" [_thread_in_native, id=24113]
> =>0x086e0088 JavaThread "SocketListener0-0" [_thread_in_Java, id=24112]
> [...]
> Heap
> PSYoungGen total 21248K, used 19699K [0xe4640000, 0xe6a00000,
> 0xf2470000)
> eden space 19584K, 92% used [0xe4640000,0xe57def08,0xe5960000)
> from space 1664K, 99% used [0xe5960000,0xe5afe020,0xe5b00000)
> to space 8384K, 0% used [0xe61d0000,0xe61d0000,0xe6a00000)
> PSOldGen total 390080K, used 370693K [0x75470000, 0x8d160000,
> 0xe4640000)
> object space 390080K, 95% used [0x75470000,0x8be716e0,0x8d160000)
> PSPermGen total 16384K, used 8080K [0x71470000, 0x72470000, 0x75470000)
> object space 16384K, 49% used [0x71470000,0x71c54298,0x72470000)
> [...]
> VM Arguments:
> jvm_args: -Xmx2000m -Dhadoop.log.dir=/export//current/../run2/log
> -Dhadoop.log.file=hadoop-jobtracker.log -Dhadoop.home.dir=/export/current
> -Dhadoop.id.str=crawler -Dhadoop.root.logger=INFO,DRFA
> java_command: org.apache.hadoop.mapred.JobTracker
> Launcher Type: SUN_STANDARD
> Environment Variables:
> JAVA_HOME=/export//java/jdk
> PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/export//current/bin
> LD_LIBRARY_PATH=/export/java/jdk1.5.0_06/jre/lib/i386/server:/export//java/jdk1.5.0_06/jre/lib/i386:/export//java/jdk1.5.0_06/jre/../lib/i386
> SHELL=/bin/tcsh
> HOSTTYPE=x86_64-linux
> OSTYPE=linux
> MACHTYPE=x86_64
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
