ok,
[erl...@cohiba ~ ]$ cat java_native_frames_dn Attaching to process ID 3836, please wait... Debugger attached successfully. Server compiler detected. JVM version is 11.2-b01 Deadlock Detection: No deadlocks found. ----------------- 3864 ----------------- 0xf775a4f1 __libc_read + 0x41 0xf702298c readBytes + 0xdc 0xf701e717 Java_java_io_FileInputStream_readBytes + 0x47 0xf400b4aa * java.io.FileInputStream.readBytes(byte[], int, int) bci:0 (Interpreted frame) 0xf4003f69 * java.io.FileInputStream.read(byte[], int, int) bci:4 line:199 (Interpreted frame) 0xf4003f69 * java.io.BufferedInputStream.read1(byte[], int, int) bci:39 line:256 (Interpreted frame) 0xf4003f69 * java.io.BufferedInputStream.read(byte[], int, int) bci:49 line:317 (Interpreted frame) 0xf4003f69 * java.io.BufferedInputStream.fill() bci:175 line:218 (Interpreted frame) 0xf400408d * java.io.BufferedInputStream.read1(byte[], int, int) bci:44 line:258 (Interpreted frame) 0xf4003f69 * java.io.BufferedInputStream.read(byte[], int, int) bci:49 line:317 (Interpreted frame) 0xf4003f69 * sun.security.provider.SeedGenerator$URLSeedGenerator.getSeedByte() bci:12 line:453 (Interpreted frame) 0xf4003e61 * sun.security.provider.SeedGenerator.getSeedBytes(byte[]) bci:11 line:123 (Interpreted frame) 0xf400408d * sun.security.provider.SeedGenerator.generateSeed(byte[]) bci:4 line:118 (Interpreted frame) 0xf400408d * sun.security.provider.SecureRandom.engineGenerateSeed(int) bci:5 line:114 (Interpreted frame) 0xf4003f27 * sun.security.provider.SecureRandom.engineNextBytes(byte[]) bci:40 line:171 (Interpreted frame) 0xf400408d * java.security.SecureRandom.nextBytes(byte[]) bci:5 line:433 (Interpreted frame) 0xf400408d * java.security.SecureRandom.next(int) bci:17 line:455 (Interpreted frame) 0xf4003f69 * java.util.Random.nextLong() bci:3 line:284 (Interpreted frame) 0xf4003fab * org.mortbay.jetty.servlet.HashSessionIdManager.doStart() bci:73 line:139 (Interpreted frame) 0xf400408d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf4004569 * org.mortbay.jetty.servlet.AbstractSessionManager.doStart() bci:96 line:168 (Interpreted frame) 0xf400408d * org.mortbay.jetty.servlet.HashSessionManager.doStart() bci:12 line:67 (Interpreted frame) 0xf400408d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf4004569 * org.mortbay.jetty.servlet.SessionHandler.doStart() bci:4 line:115 (Interpreted frame) 0xf400408d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf4004569 * org.mortbay.jetty.handler.HandlerWrapper.doStart() bci:11 line:130 (Interpreted frame) 0xf400408d * org.mortbay.jetty.handler.ContextHandler.startContext() bci:1 line:537 (Interpreted frame) 0xf400408d * org.mortbay.jetty.servlet.Context.startContext() bci:1 line:136 (Interpreted frame) 0xf400408d * org.mortbay.jetty.webapp.WebAppContext.startContext() bci:123 line:1234 (Interpreted frame) 0xf400408d * org.mortbay.jetty.handler.ContextHandler.doStart() bci:140 line:517 (Interpreted frame) 0xf400408d * org.mortbay.jetty.webapp.WebAppContext.doStart() bci:170 line:460 (Interpreted frame) 0xf400408d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf4004569 * org.mortbay.jetty.handler.HandlerCollection.doStart() bci:32 line:152 (Interpreted frame) 0xf400408d * org.mortbay.jetty.handler.ContextHandlerCollection.doStart() bci:5 line:156 (Interpreted frame) 0xf400408d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf4004569 * org.mortbay.jetty.handler.HandlerWrapper.doStart() bci:11 line:130 (Interpreted frame) 0xf400408d * org.mortbay.jetty.Server.doStart() bci:201 line:222 (Interpreted frame) 0xf400408d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf400408d * org.apache.hadoop.http.HttpServer.start() bci:383 line:461 (Interpreted frame) 0xf400408d * org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(org.apache.hadoop.conf.Configuration, java.util.AbstractList) bci:916 line:375 (Interpreted frame) 0xf400408d * org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(org.apache.hadoop.conf.Configuration, java.util.AbstractList) bci:158 line:216 (Interpreted frame) 0xf400408d * org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(java.lang.String[], org.apache.hadoop.conf.Configuration) bci:96 line:1283 (Interpreted frame) 0xf4003f27 * org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(java.lang.String[], org.apache.hadoop.conf.Configuration) bci:92 line:1238 (Interpreted frame) 0xf4003f27 * org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(java.lang.String[], org.apache.hadoop.conf.Configuration) bci:2 line:1246 (Interpreted frame) 0xf4003f27 * org.apache.hadoop.hdfs.server.datanode.DataNode.main(java.lang.String[]) bci:12 line:1368 (Interpreted frame) 0xf400134c <StubRoutines> 0x06375c9d _ZN9JavaCalls11call_helperEP9JavaValueP12methodHandleP17JavaCallArgumentsP6Thread + 0x15d 0x065057f8 _ZN2os20os_exception_wrapperEPFvP9JavaValueP12methodHandleP17JavaCallArgumentsP6ThreadES1_S3_S5_S7_ + 0x18 0x06375b30 _ZN9JavaCalls4callEP9JavaValue12methodHandleP17JavaCallArgumentsP6Thread + 0x20 0x0639f2e6 _Z17jni_invoke_staticP7JNIEnv_P9JavaValueP8_jobject11JNICallTypeP10_jmethodIDP18JNI_ArgumentPusherP6Thread + 0x116 0x06390ae8 jni_CallStaticVoidMethod + 0xf8 0x08049b98 JavaMain + 0x2c8 0xf7753f50 start_thread + 0xa0 ----------------- 3865 ----------------- 0xf77578d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x064ed92d _ZN7Monitor5IWaitEP6Threadx + 0xcd 0x064ee071 _ZN7Monitor4waitEblb + 0x221 0x063272f6 _ZN13GCTaskManager8get_taskEj + 0x66 0x0632843b _ZN12GCTaskThread3runEv + 0xdb 0x06506929 _Z10java_startP6Thread + 0x139 0xf7753f50 start_thread + 0xa0 ----------------- 3866 ----------------- 0xf77578d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x064ed92d _ZN7Monitor5IWaitEP6Threadx + 0xcd 0x064ee071 _ZN7Monitor4waitEblb + 0x221 0x063272f6 _ZN13GCTaskManager8get_taskEj + 0x66 0x0632843b _ZN12GCTaskThread3runEv + 0xdb 0x06506929 _Z10java_startP6Thread + 0x139 0xf7753f50 start_thread + 0xa0 ----------------- 3869 ----------------- 0xf7757be8 __pthread_cond_timedwait + 0xe8 0x39d73c58 ???????? sun.jvm.hotspot.debugger.UnmappedAddressException at sun.jvm.hotspot.debugger.PageCache.checkPage(PageCache.java:208) at sun.jvm.hotspot.debugger.PageCache.getData(PageCache.java:63) at sun.jvm.hotspot.debugger.DebuggerBase.readBytes(DebuggerBase.java:205) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:471) at sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:442) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readAddress(LinuxDebuggerLocal.java:423) at sun.jvm.hotspot.debugger.linux.LinuxAddress.getAddressAt(LinuxAddress.java:74) at sun.jvm.hotspot.debugger.linux.x86.LinuxX86CFrame.sender(LinuxX86CFrame.java:60) at sun.jvm.hotspot.tools.PStack.run(PStack.java:161) at sun.jvm.hotspot.tools.PStack.run(PStack.java:53) at sun.jvm.hotspot.tools.PStack.run(PStack.java:48) at sun.jvm.hotspot.tools.JStack.run(JStack.java:60) at sun.jvm.hotspot.tools.Tool.start(Tool.java:221) at sun.jvm.hotspot.tools.JStack.main(JStack.java:86) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.tools.jstack.JStack.runJStackTool(JStack.java:118) at sun.tools.jstack.JStack.main(JStack.java:84) ----------------- 3870 ----------------- 0xf77578d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x065a1a31 _ZN13ObjectMonitor4waitExbP6Thread + 0x571 0x0659f3a2 _ZN18ObjectSynchronizer4waitE6HandlexP6Thread + 0x42 0x063c834a JVM_MonitorWait + 0x1da 0xf400b4aa * java.lang.Object.wait(long) bci:0 (Interpreted frame) 0xf400408d * java.lang.Object.wait() bci:2 line:485 (Interpreted frame) 0xf400408d * java.lang.ref.Reference$ReferenceHandler.run() bci:46 line:116 (Interpreted frame) 0xf400134c <StubRoutines> 0x06375c9d _ZN9JavaCalls11call_helperEP9JavaValueP12methodHandleP17JavaCallArgumentsP6Thread + 0x15d 0x065057f8 _ZN2os20os_exception_wrapperEPFvP9JavaValueP12methodHandleP17JavaCallArgumentsP6ThreadES1_S3_S5_S7_ + 0x18 0x063755b0 _ZN9JavaCalls12call_virtualEP9JavaValue11KlassHandle12symbolHandleS3_P17JavaCallArgumentsP6Thread + 0xa0 0x0637563d _ZN9JavaCalls12call_virtualEP9JavaValue6Handle11KlassHandle12symbolHandleS4_P6Thread + 0x5d 0x063e5ca5 _Z12thread_entryP10JavaThreadP6Thread + 0x75 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf7753f50 start_thread + 0xa0 ----------------- 3871 ----------------- 0xf77578d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x065a1a31 _ZN13ObjectMonitor4waitExbP6Thread + 0x571 0x0659f3a2 _ZN18ObjectSynchronizer4waitE6HandlexP6Thread + 0x42 0x063c834a JVM_MonitorWait + 0x1da 0xf400b4aa * java.lang.Object.wait(long) bci:0 (Interpreted frame) 0xf400408d * java.lang.ref.ReferenceQueue.remove(long) bci:44 line:116 (Interpreted frame) 0xf4003f27 * java.lang.ref.ReferenceQueue.remove() bci:2 line:132 (Interpreted frame) 0xf4003f27 * java.lang.ref.Finalizer$FinalizerThread.run() bci:3 line:159 (Interpreted frame) 0xf400134c <StubRoutines> 0x06375c9d _ZN9JavaCalls11call_helperEP9JavaValueP12methodHandleP17JavaCallArgumentsP6Thread + 0x15d 0x065057f8 _ZN2os20os_exception_wrapperEPFvP9JavaValueP12methodHandleP17JavaCallArgumentsP6ThreadES1_S3_S5_S7_ + 0x18 0x063755b0 _ZN9JavaCalls12call_virtualEP9JavaValue11KlassHandle12symbolHandleS3_P17JavaCallArgumentsP6Thread + 0xa0 0x0637563d _ZN9JavaCalls12call_virtualEP9JavaValue6Handle11KlassHandle12symbolHandleS4_P6Thread + 0x5d 0x063e5ca5 _Z12thread_entryP10JavaThreadP6Thread + 0x75 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf7753f50 start_thread + 0xa0 ----------------- 3872 ----------------- 0xf77598b0 __new_sem_wait + 0x40 0x06503c0d _ZN2os11signal_waitEv + 0xd 0x06500d28 _Z19signal_thread_entryP10JavaThreadP6Thread + 0x28 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf7753f50 start_thread + 0xa0 ----------------- 3873 ----------------- 0xf77578d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x064ed92d _ZN7Monitor5IWaitEP6Threadx + 0xcd 0x064edf21 _ZN7Monitor4waitEblb + 0xd1 0x062ada2f _ZN12CompileQueue3getEv + 0x5f 0x062af2b4 _ZN13CompileBroker20compiler_thread_loopEv + 0x184 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf7753f50 start_thread + 0xa0 ----------------- 3874 ----------------- 0xf77578d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x064ed92d _ZN7Monitor5IWaitEP6Threadx + 0xcd 0x064edf21 _ZN7Monitor4waitEblb + 0xd1 0x062ada2f _ZN12CompileQueue3getEv + 0x5f 0x062af2b4 _ZN13CompileBroker20compiler_thread_loopEv + 0x184 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf7753f50 start_thread + 0xa0 ----------------- 3875 ----------------- 0xf77578d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x064ed92d _ZN7Monitor5IWaitEP6Threadx + 0xcd 0x064ee071 _ZN7Monitor4waitEblb + 0x221 0x064bc935 _ZN17LowMemoryDetector32low_memory_detector_thread_entryEP10JavaThreadP6Thread + 0x85 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf7753f50 start_thread + 0xa0 ----------------- 3878 ----------------- 0xf775a6fe __libc_accept + 0x3e 0xb0c41a19 Java_java_net_PlainSocketImpl_socketAccept + 0x129 0xf400b4aa * java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) bci:0 (Interpreted frame) 0xf400408d * java.net.PlainSocketImpl.accept(java.net.SocketImpl) bci:7 line:384 (Interpreted frame) 0xf400408d * java.net.ServerSocket.implAccept(java.net.Socket) bci:60 line:453 (Interpreted frame) 0xf400408d * java.net.ServerSocket.accept() bci:48 line:421 (Interpreted frame) 0xf4003f27 * sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept() bci:1 line:34 (Interpreted frame) 0xf4003f27 * sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop() bci:55 line:369 (Interpreted frame) 0xf400408d * sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run() bci:1 line:341 (Interpreted frame) 0xf4004569 * java.lang.Thread.run() bci:11 line:619 (Interpreted frame) 0xf400134c <StubRoutines> 0x06375c9d _ZN9JavaCalls11call_helperEP9JavaValueP12methodHandleP17JavaCallArgumentsP6Thread + 0x15d 0x065057f8 _ZN2os20os_exception_wrapperEPFvP9JavaValueP12methodHandleP17JavaCallArgumentsP6ThreadES1_S3_S5_S7_ + 0x18 0x063755b0 _ZN9JavaCalls12call_virtualEP9JavaValue11KlassHandle12symbolHandleS3_P17JavaCallArgumentsP6Thread + 0xa0 0x0637563d _ZN9JavaCalls12call_virtualEP9JavaValue6Handle11KlassHandle12symbolHandleS4_P6Thread + 0x5d 0x063e5ca5 _Z12thread_entryP10JavaThreadP6Thread + 0x75 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf7753f50 start_thread + 0xa0 ----------------- 3879 ----------------- 0xf7757be8 __pthread_cond_timedwait + 0xe8 0x1bef7958 ???????? sun.jvm.hotspot.debugger.UnmappedAddressException at sun.jvm.hotspot.debugger.PageCache.checkPage(PageCache.java:208) at sun.jvm.hotspot.debugger.PageCache.getData(PageCache.java:63) at sun.jvm.hotspot.debugger.DebuggerBase.readBytes(DebuggerBase.java:205) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:471) at sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:442) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readAddress(LinuxDebuggerLocal.java:423) at sun.jvm.hotspot.debugger.linux.LinuxAddress.getAddressAt(LinuxAddress.java:74) at sun.jvm.hotspot.debugger.linux.x86.LinuxX86CFrame.sender(LinuxX86CFrame.java:60) at sun.jvm.hotspot.tools.PStack.run(PStack.java:161) at sun.jvm.hotspot.tools.PStack.run(PStack.java:53) at sun.jvm.hotspot.tools.PStack.run(PStack.java:48) at sun.jvm.hotspot.tools.JStack.run(JStack.java:60) at sun.jvm.hotspot.tools.Tool.start(Tool.java:221) at sun.jvm.hotspot.tools.JStack.main(JStack.java:86) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.tools.jstack.JStack.runJStackTool(JStack.java:118) at sun.tools.jstack.JStack.main(JStack.java:84) ----------------- 4055 ----------------- 0xf7757be8 __pthread_cond_timedwait + 0xe8 0x14e31390 ???????? sun.jvm.hotspot.debugger.UnmappedAddressException at sun.jvm.hotspot.debugger.PageCache.checkPage(PageCache.java:208) at sun.jvm.hotspot.debugger.PageCache.getData(PageCache.java:63) at sun.jvm.hotspot.debugger.DebuggerBase.readBytes(DebuggerBase.java:205) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:471) at sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:442) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readAddress(LinuxDebuggerLocal.java:423) at sun.jvm.hotspot.debugger.linux.LinuxAddress.getAddressAt(LinuxAddress.java:74) at sun.jvm.hotspot.debugger.linux.x86.LinuxX86CFrame.sender(LinuxX86CFrame.java:60) at sun.jvm.hotspot.tools.PStack.run(PStack.java:161) at sun.jvm.hotspot.tools.PStack.run(PStack.java:53) at sun.jvm.hotspot.tools.PStack.run(PStack.java:48) at sun.jvm.hotspot.tools.JStack.run(JStack.java:60) at sun.jvm.hotspot.tools.Tool.start(Tool.java:221) at sun.jvm.hotspot.tools.JStack.main(JStack.java:86) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.tools.jstack.JStack.runJStackTool(JStack.java:118) at sun.tools.jstack.JStack.main(JStack.java:84) ----------------- 4060 ----------------- 0xf7757be8 __pthread_cond_timedwait + 0xe8 0x358d0f88 ???????? sun.jvm.hotspot.debugger.UnalignedAddressException: Trying to read at address: 0x4bbbaf53 with alignment: 4 at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1.checkAlignment(LinuxDebuggerLocal.java:181) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:469) at sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:442) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readAddress(LinuxDebuggerLocal.java:423) at sun.jvm.hotspot.debugger.linux.LinuxAddress.getAddressAt(LinuxAddress.java:74) at sun.jvm.hotspot.debugger.linux.x86.LinuxX86CFrame.sender(LinuxX86CFrame.java:60) at sun.jvm.hotspot.tools.PStack.run(PStack.java:161) at sun.jvm.hotspot.tools.PStack.run(PStack.java:53) at sun.jvm.hotspot.tools.PStack.run(PStack.java:48) at sun.jvm.hotspot.tools.JStack.run(JStack.java:60) at sun.jvm.hotspot.tools.Tool.start(Tool.java:221) at sun.jvm.hotspot.tools.JStack.main(JStack.java:86) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.tools.jstack.JStack.runJStackTool(JStack.java:118) at sun.tools.jstack.JStack.main(JStack.java:84) ----------------- 4062 ----------------- 0xf7757be8 __pthread_cond_timedwait + 0xe8 0x35c10c20 ???????? sun.jvm.hotspot.debugger.UnalignedAddressException: Trying to read at address: 0x4bbbaf53 with alignment: 4 at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1.checkAlignment(LinuxDebuggerLocal.java:181) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:469) at sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:442) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readAddress(LinuxDebuggerLocal.java:423) at sun.jvm.hotspot.debugger.linux.LinuxAddress.getAddressAt(LinuxAddress.java:74) at sun.jvm.hotspot.debugger.linux.x86.LinuxX86CFrame.sender(LinuxX86CFrame.java:60) at sun.jvm.hotspot.tools.PStack.run(PStack.java:161) at sun.jvm.hotspot.tools.PStack.run(PStack.java:53) at sun.jvm.hotspot.tools.PStack.run(PStack.java:48) at sun.jvm.hotspot.tools.JStack.run(JStack.java:60) at sun.jvm.hotspot.tools.Tool.start(Tool.java:221) at sun.jvm.hotspot.tools.JStack.main(JStack.java:86) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.tools.jstack.JStack.runJStackTool(JStack.java:118) at sun.tools.jstack.JStack.main(JStack.java:84) ----------------- 7801 ----------------- 0xf775a6fe __libc_accept + 0x3e 0x061c1a78 _ZN14AttachListener7dequeueEv + 0x68 0x061c0e55 _Z28attach_listener_thread_entryP10JavaThreadP6Thread + 0x35 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf7753f50 start_thread + 0xa0 ----------------- 3836 ----------------- 0xf7754452 pthread_join + 0x112 0x0804dd78 ContinueInNewThread + 0x88 0x080497f6 main + 0x276 0xf7614775 ???????? [erl...@cohiba ~ ]$ cat java_native_frames_tt Attaching to process ID 22510, please wait... Debugger attached successfully. Server compiler detected. JVM version is 11.2-b01 Deadlock Detection: No deadlocks found. ----------------- 22544 ----------------- 0xf76e34f1 __libc_read + 0x41 0xf6fab98c readBytes + 0xdc 0xf6fa7717 Java_java_io_FileInputStream_readBytes + 0x47 0xf3f944aa * java.io.FileInputStream.readBytes(byte[], int, int) bci:0 (Interpreted frame) 0xf3f8cf69 * java.io.FileInputStream.read(byte[], int, int) bci:4 line:199 (Interpreted frame) 0xf3f8cf69 * java.io.BufferedInputStream.read1(byte[], int, int) bci:39 line:256 (Interpreted frame) 0xf3f8cf69 * java.io.BufferedInputStream.read(byte[], int, int) bci:49 line:317 (Interpreted frame) 0xf3f8cf69 * java.io.BufferedInputStream.fill() bci:175 line:218 (Interpreted frame) 0xf3f8d08d * java.io.BufferedInputStream.read1(byte[], int, int) bci:44 line:258 (Interpreted frame) 0xf3f8cf69 * java.io.BufferedInputStream.read(byte[], int, int) bci:49 line:317 (Interpreted frame) 0xf3f8cf69 * sun.security.provider.SeedGenerator$URLSeedGenerator.getSeedByte() bci:12 line:453 (Interpreted frame) 0xf3f8ce61 * sun.security.provider.SeedGenerator.getSeedBytes(byte[]) bci:11 line:123 (Interpreted frame) 0xf3f8d08d * sun.security.provider.SeedGenerator.generateSeed(byte[]) bci:4 line:118 (Interpreted frame) 0xf3f8d08d * sun.security.provider.SecureRandom.engineGenerateSeed(int) bci:5 line:114 (Interpreted frame) 0xf3f8cf27 * sun.security.provider.SecureRandom.engineNextBytes(byte[]) bci:40 line:171 (Interpreted frame) 0xf3f8d08d * java.security.SecureRandom.nextBytes(byte[]) bci:5 line:433 (Interpreted frame) 0xf3f8d08d * java.security.SecureRandom.next(int) bci:17 line:455 (Interpreted frame) 0xf3f8cf69 * java.util.Random.nextLong() bci:3 line:284 (Interpreted frame) 0xf3f8cfab * org.mortbay.jetty.servlet.HashSessionIdManager.doStart() bci:73 line:139 (Interpreted frame) 0xf3f8d08d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf3f8d569 * org.mortbay.jetty.servlet.AbstractSessionManager.doStart() bci:96 line:168 (Interpreted frame) 0xf3f8d08d * org.mortbay.jetty.servlet.HashSessionManager.doStart() bci:12 line:67 (Interpreted frame) 0xf3f8d08d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf3f8d569 * org.mortbay.jetty.servlet.SessionHandler.doStart() bci:4 line:115 (Interpreted frame) 0xf3f8d08d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf3f8d569 * org.mortbay.jetty.handler.HandlerWrapper.doStart() bci:11 line:130 (Interpreted frame) 0xf3f8d08d * org.mortbay.jetty.handler.ContextHandler.startContext() bci:1 line:537 (Interpreted frame) 0xf3f8d08d * org.mortbay.jetty.servlet.Context.startContext() bci:1 line:136 (Interpreted frame) 0xf3f8d08d * org.mortbay.jetty.webapp.WebAppContext.startContext() bci:123 line:1234 (Interpreted frame) 0xf3f8d08d * org.mortbay.jetty.handler.ContextHandler.doStart() bci:140 line:517 (Interpreted frame) 0xf3f8d08d * org.mortbay.jetty.webapp.WebAppContext.doStart() bci:170 line:460 (Interpreted frame) 0xf3f8d08d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf3f8d569 * org.mortbay.jetty.handler.HandlerCollection.doStart() bci:32 line:152 (Interpreted frame) 0xf3f8d08d * org.mortbay.jetty.handler.ContextHandlerCollection.doStart() bci:5 line:156 (Interpreted frame) 0xf3f8d08d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf3f8d569 * org.mortbay.jetty.handler.HandlerWrapper.doStart() bci:11 line:130 (Interpreted frame) 0xf3f8d08d * org.mortbay.jetty.Server.doStart() bci:201 line:222 (Interpreted frame) 0xf3f8d08d * org.mortbay.component.AbstractLifeCycle.start() bci:31 line:50 (Interpreted frame) 0xf3f8d08d * org.apache.hadoop.http.HttpServer.start() bci:383 line:461 (Interpreted frame) 0xf3f8d08d * org.apache.hadoop.mapred.TaskTracker.<init>(org.apache.hadoop.mapred.JobConf) bci:519 line:931 (Interpreted frame) 0xf3f8d08d * org.apache.hadoop.mapred.TaskTracker.main(java.lang.String[]) bci:52 line:2833 (Interpreted frame) 0xf3f8a34c <StubRoutines> 0x06375c9d _ZN9JavaCalls11call_helperEP9JavaValueP12methodHandleP17JavaCallArgumentsP6Thread + 0x15d 0x065057f8 _ZN2os20os_exception_wrapperEPFvP9JavaValueP12methodHandleP17JavaCallArgumentsP6ThreadES1_S3_S5_S7_ + 0x18 0x06375b30 _ZN9JavaCalls4callEP9JavaValue12methodHandleP17JavaCallArgumentsP6Thread + 0x20 0x0639f2e6 _Z17jni_invoke_staticP7JNIEnv_P9JavaValueP8_jobject11JNICallTypeP10_jmethodIDP18JNI_ArgumentPusherP6Thread + 0x116 0x06390ae8 jni_CallStaticVoidMethod + 0xf8 0x08049b98 JavaMain + 0x2c8 0xf76dcf50 start_thread + 0xa0 ----------------- 22545 ----------------- 0xf76e08d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x064ed92d _ZN7Monitor5IWaitEP6Threadx + 0xcd 0x064ee071 _ZN7Monitor4waitEblb + 0x221 0x063272f6 _ZN13GCTaskManager8get_taskEj + 0x66 0x0632843b _ZN12GCTaskThread3runEv + 0xdb 0x06506929 _Z10java_startP6Thread + 0x139 0xf76dcf50 start_thread + 0xa0 ----------------- 22546 ----------------- 0xf76e08d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x064ed92d _ZN7Monitor5IWaitEP6Threadx + 0xcd 0x064ee071 _ZN7Monitor4waitEblb + 0x221 0x063272f6 _ZN13GCTaskManager8get_taskEj + 0x66 0x0632843b _ZN12GCTaskThread3runEv + 0xdb 0x06506929 _Z10java_startP6Thread + 0x139 0xf76dcf50 start_thread + 0xa0 ----------------- 22549 ----------------- 0xf76e0be8 __pthread_cond_timedwait + 0xe8 0x340707e0 ???????? sun.jvm.hotspot.debugger.UnmappedAddressException at sun.jvm.hotspot.debugger.PageCache.checkPage(PageCache.java:208) at sun.jvm.hotspot.debugger.PageCache.getData(PageCache.java:63) at sun.jvm.hotspot.debugger.DebuggerBase.readBytes(DebuggerBase.java:205) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:471) at sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:442) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readAddress(LinuxDebuggerLocal.java:423) at sun.jvm.hotspot.debugger.linux.LinuxAddress.getAddressAt(LinuxAddress.java:74) at sun.jvm.hotspot.debugger.linux.x86.LinuxX86CFrame.sender(LinuxX86CFrame.java:60) at sun.jvm.hotspot.tools.PStack.run(PStack.java:161) at sun.jvm.hotspot.tools.PStack.run(PStack.java:53) at sun.jvm.hotspot.tools.PStack.run(PStack.java:48) at sun.jvm.hotspot.tools.JStack.run(JStack.java:60) at sun.jvm.hotspot.tools.Tool.start(Tool.java:221) at sun.jvm.hotspot.tools.JStack.main(JStack.java:86) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.tools.jstack.JStack.runJStackTool(JStack.java:118) at sun.tools.jstack.JStack.main(JStack.java:84) ----------------- 22550 ----------------- 0xf76e08d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x065a1a31 _ZN13ObjectMonitor4waitExbP6Thread + 0x571 0x0659f3a2 _ZN18ObjectSynchronizer4waitE6HandlexP6Thread + 0x42 0x063c834a JVM_MonitorWait + 0x1da 0xf3f944aa * java.lang.Object.wait(long) bci:0 (Interpreted frame) 0xf3f8d08d * java.lang.Object.wait() bci:2 line:485 (Interpreted frame) 0xf3f8d08d * java.lang.ref.Reference$ReferenceHandler.run() bci:46 line:116 (Interpreted frame) 0xf3f8a34c <StubRoutines> 0x06375c9d _ZN9JavaCalls11call_helperEP9JavaValueP12methodHandleP17JavaCallArgumentsP6Thread + 0x15d 0x065057f8 _ZN2os20os_exception_wrapperEPFvP9JavaValueP12methodHandleP17JavaCallArgumentsP6ThreadES1_S3_S5_S7_ + 0x18 0x063755b0 _ZN9JavaCalls12call_virtualEP9JavaValue11KlassHandle12symbolHandleS3_P17JavaCallArgumentsP6Thread + 0xa0 0x0637563d _ZN9JavaCalls12call_virtualEP9JavaValue6Handle11KlassHandle12symbolHandleS4_P6Thread + 0x5d 0x063e5ca5 _Z12thread_entryP10JavaThreadP6Thread + 0x75 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf76dcf50 start_thread + 0xa0 ----------------- 22551 ----------------- 0xf76e08d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x065a1a31 _ZN13ObjectMonitor4waitExbP6Thread + 0x571 0x0659f3a2 _ZN18ObjectSynchronizer4waitE6HandlexP6Thread + 0x42 0x063c834a JVM_MonitorWait + 0x1da 0xf3f944aa * java.lang.Object.wait(long) bci:0 (Interpreted frame) 0xf3f8d08d * java.lang.ref.ReferenceQueue.remove(long) bci:44 line:116 (Interpreted frame) 0xf3f8cf27 * java.lang.ref.ReferenceQueue.remove() bci:2 line:132 (Interpreted frame) 0xf3f8cf27 * java.lang.ref.Finalizer$FinalizerThread.run() bci:3 line:159 (Interpreted frame) 0xf3f8a34c <StubRoutines> 0x06375c9d _ZN9JavaCalls11call_helperEP9JavaValueP12methodHandleP17JavaCallArgumentsP6Thread + 0x15d 0x065057f8 _ZN2os20os_exception_wrapperEPFvP9JavaValueP12methodHandleP17JavaCallArgumentsP6ThreadES1_S3_S5_S7_ + 0x18 0x063755b0 _ZN9JavaCalls12call_virtualEP9JavaValue11KlassHandle12symbolHandleS3_P17JavaCallArgumentsP6Thread + 0xa0 0x0637563d _ZN9JavaCalls12call_virtualEP9JavaValue6Handle11KlassHandle12symbolHandleS4_P6Thread + 0x5d 0x063e5ca5 _Z12thread_entryP10JavaThreadP6Thread + 0x75 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf76dcf50 start_thread + 0xa0 ----------------- 22554 ----------------- 0xf76e28b0 __new_sem_wait + 0x40 0x06503c0d _ZN2os11signal_waitEv + 0xd 0x06500d28 _Z19signal_thread_entryP10JavaThreadP6Thread + 0x28 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf76dcf50 start_thread + 0xa0 ----------------- 22555 ----------------- 0xf76e08d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x064ed92d _ZN7Monitor5IWaitEP6Threadx + 0xcd 0x064edf21 _ZN7Monitor4waitEblb + 0xd1 0x062ada2f _ZN12CompileQueue3getEv + 0x5f 0x062af2b4 _ZN13CompileBroker20compiler_thread_loopEv + 0x184 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf76dcf50 start_thread + 0xa0 ----------------- 22556 ----------------- 0xf76e08d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x064ed92d _ZN7Monitor5IWaitEP6Threadx + 0xcd 0x064edf21 _ZN7Monitor4waitEblb + 0xd1 0x062ada2f _ZN12CompileQueue3getEv + 0x5f 0x062af2b4 _ZN13CompileBroker20compiler_thread_loopEv + 0x184 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf76dcf50 start_thread + 0xa0 ----------------- 22557 ----------------- 0xf76e08d0 __pthread_cond_wait + 0x90 0x065060c9 _ZN2os13PlatformEvent4parkEv + 0xb9 0x064ed92d _ZN7Monitor5IWaitEP6Threadx + 0xcd 0x064ee071 _ZN7Monitor4waitEblb + 0x221 0x064bc935 _ZN17LowMemoryDetector32low_memory_detector_thread_entryEP10JavaThreadP6Thread + 0x85 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf76dcf50 start_thread + 0xa0 ----------------- 22558 ----------------- 0xf76e0be8 __pthread_cond_timedwait + 0xe8 0x38ed9008 ???????? sun.jvm.hotspot.debugger.UnalignedAddressException: Trying to read at address: 0x4bbbb06f with alignment: 4 at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1.checkAlignment(LinuxDebuggerLocal.java:181) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:469) at sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:442) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readAddress(LinuxDebuggerLocal.java:423) at sun.jvm.hotspot.debugger.linux.LinuxAddress.getAddressAt(LinuxAddress.java:74) at sun.jvm.hotspot.debugger.linux.x86.LinuxX86CFrame.sender(LinuxX86CFrame.java:60) at sun.jvm.hotspot.tools.PStack.run(PStack.java:161) at sun.jvm.hotspot.tools.PStack.run(PStack.java:53) at sun.jvm.hotspot.tools.PStack.run(PStack.java:48) at sun.jvm.hotspot.tools.JStack.run(JStack.java:60) at sun.jvm.hotspot.tools.Tool.start(Tool.java:221) at sun.jvm.hotspot.tools.JStack.main(JStack.java:86) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.tools.jstack.JStack.runJStackTool(JStack.java:118) at sun.tools.jstack.JStack.main(JStack.java:84) ----------------- 22644 ----------------- 0xf76e0be8 __pthread_cond_timedwait + 0xe8 0x07b9bcc8 ???????? sun.jvm.hotspot.debugger.UnalignedAddressException: Trying to read at address: 0x4bbbb087 with alignment: 4 at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1.checkAlignment(LinuxDebuggerLocal.java:181) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:469) at sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:442) at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readAddress(LinuxDebuggerLocal.java:423) at sun.jvm.hotspot.debugger.linux.LinuxAddress.getAddressAt(LinuxAddress.java:74) at sun.jvm.hotspot.debugger.linux.x86.LinuxX86CFrame.sender(LinuxX86CFrame.java:60) at sun.jvm.hotspot.tools.PStack.run(PStack.java:161) at sun.jvm.hotspot.tools.PStack.run(PStack.java:53) at sun.jvm.hotspot.tools.PStack.run(PStack.java:48) at sun.jvm.hotspot.tools.JStack.run(JStack.java:60) at sun.jvm.hotspot.tools.Tool.start(Tool.java:221) at sun.jvm.hotspot.tools.JStack.main(JStack.java:86) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.tools.jstack.JStack.runJStackTool(JStack.java:118) at sun.tools.jstack.JStack.main(JStack.java:84) ----------------- 24136 ----------------- 0xf76e36fe __libc_accept + 0x3e 0x061c1a78 _ZN14AttachListener7dequeueEv + 0x68 0x061c0e55 _Z28attach_listener_thread_entryP10JavaThreadP6Thread + 0x35 0x065cde5d _ZN10JavaThread3runEv + 0xfd 0x06506929 _Z10java_startP6Thread + 0x139 0xf76dcf50 start_thread + 0xa0 ----------------- 22510 ----------------- 0xf76dd452 pthread_join + 0x112 0x0804dd78 ContinueInNewThread + 0x88 0x080497f6 main + 0x276 0xf759d775 ???????? [erl...@cohiba ~ ]$ cat thread_dump_dn Attaching to process ID 3836, please wait... Debugger attached successfully. Server compiler detected. JVM version is 11.2-b01 Deadlock Detection: No deadlocks found. Thread 7801: (state = BLOCKED) Thread 4062: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - org.mortbay.thread.QueuedThreadPool$PoolThread.run() @bci=275, line=565 (Interpreted frame) Thread 4060: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - org.mortbay.thread.QueuedThreadPool$PoolThread.run() @bci=275, line=565 (Interpreted frame) Thread 4055: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame) - org.apache.hadoop.fs.DU$DURefreshThread.run() @bci=17, line=80 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=619 (Interpreted frame) Thread 3878: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame) - java.net.PlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=384 (Interpreted frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=453 (Interpreted frame) - java.net.ServerSocket.accept() @bci=48, line=421 (Interpreted frame) - sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept() @bci=1, line=34 (Interpreted frame) - sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop() @bci=55, line=369 (Interpreted frame) - sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run() @bci=1, line=341 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=619 (Interpreted frame) Thread 3872: (state = BLOCKED) Thread 3871: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=116 (Interpreted frame) - java.lang.ref.ReferenceQueue.remove() @bci=2, line=132 (Interpreted frame) - java.lang.ref.Finalizer$FinalizerThread.run() @bci=3, line=159 (Interpreted frame) Thread 3870: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame) - java.lang.ref.Reference$ReferenceHandler.run() @bci=46, line=116 (Interpreted frame) Thread 3864: (state = IN_NATIVE) - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Interpreted frame) - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=199 (Interpreted frame) - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=256 (Interpreted frame) - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=317 (Interpreted frame) - java.io.BufferedInputStream.fill() @bci=175, line=218 (Interpreted frame) - java.io.BufferedInputStream.read1(byte[], int, int) @bci=44, line=258 (Interpreted frame) - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=317 (Interpreted frame) - sun.security.provider.SeedGenerator$URLSeedGenerator.getSeedByte() @bci=12, line=453 (Interpreted frame) - sun.security.provider.SeedGenerator.getSeedBytes(byte[]) @bci=11, line=123 (Interpreted frame) - sun.security.provider.SeedGenerator.generateSeed(byte[]) @bci=4, line=118 (Interpreted frame) - sun.security.provider.SecureRandom.engineGenerateSeed(int) @bci=5, line=114 (Interpreted frame) - sun.security.provider.SecureRandom.engineNextBytes(byte[]) @bci=40, line=171 (Interpreted frame) - java.security.SecureRandom.nextBytes(byte[]) @bci=5, line=433 (Interpreted frame) - java.security.SecureRandom.next(int) @bci=17, line=455 (Interpreted frame) - java.util.Random.nextLong() @bci=3, line=284 (Interpreted frame) - org.mortbay.jetty.servlet.HashSessionIdManager.doStart() @bci=73, line=139 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.mortbay.jetty.servlet.AbstractSessionManager.doStart() @bci=96, line=168 (Interpreted frame) - org.mortbay.jetty.servlet.HashSessionManager.doStart() @bci=12, line=67 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.mortbay.jetty.servlet.SessionHandler.doStart() @bci=4, line=115 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.mortbay.jetty.handler.HandlerWrapper.doStart() @bci=11, line=130 (Interpreted frame) - org.mortbay.jetty.handler.ContextHandler.startContext() @bci=1, line=537 (Interpreted frame) - org.mortbay.jetty.servlet.Context.startContext() @bci=1, line=136 (Interpreted frame) - org.mortbay.jetty.webapp.WebAppContext.startContext() @bci=123, line=1234 (Interpreted frame) - org.mortbay.jetty.handler.ContextHandler.doStart() @bci=140, line=517 (Interpreted frame) - org.mortbay.jetty.webapp.WebAppContext.doStart() @bci=170, line=460 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.mortbay.jetty.handler.HandlerCollection.doStart() @bci=32, line=152 (Interpreted frame) - org.mortbay.jetty.handler.ContextHandlerCollection.doStart() @bci=5, line=156 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.mortbay.jetty.handler.HandlerWrapper.doStart() @bci=11, line=130 (Interpreted frame) - org.mortbay.jetty.Server.doStart() @bci=201, line=222 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.apache.hadoop.http.HttpServer.start() @bci=383, line=461 (Interpreted frame) - org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(org.apache.hadoop.conf.Configuration, java.util.AbstractList) @bci=916, line=375 (Interpreted frame) - org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(org.apache.hadoop.conf.Configuration, java.util.AbstractList) @bci=158, line=216 (Interpreted frame) - org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(java.lang.String[], org.apache.hadoop.conf.Configuration) @bci=96, line=1283 (Interpreted frame) - org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(java.lang.String[], org.apache.hadoop.conf.Configuration) @bci=92, line=1238 (Interpreted frame) - org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(java.lang.String[], org.apache.hadoop.conf.Configuration) @bci=2, line=1246 (Interpreted frame) - org.apache.hadoop.hdfs.server.datanode.DataNode.main(java.lang.String[]) @bci=12, line=1368 (Interpreted frame) [erl...@cohiba ~ ]$ cat thread_dump_tt Attaching to process ID 22510, please wait... Debugger attached successfully. Server compiler detected. JVM version is 11.2-b01 Deadlock Detection: No deadlocks found. Thread 24136: (state = BLOCKED) Thread 22644: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - org.mortbay.thread.QueuedThreadPool$PoolThread.run() @bci=275, line=565 (Interpreted frame) Thread 22554: (state = BLOCKED) Thread 22551: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=116 (Interpreted frame) - java.lang.ref.ReferenceQueue.remove() @bci=2, line=132 (Interpreted frame) - java.lang.ref.Finalizer$FinalizerThread.run() @bci=3, line=159 (Interpreted frame) Thread 22550: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame) - java.lang.ref.Reference$ReferenceHandler.run() @bci=46, line=116 (Interpreted frame) Thread 22544: (state = IN_NATIVE) - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Interpreted frame) - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=199 (Interpreted frame) - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=256 (Interpreted frame) - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=317 (Interpreted frame) - java.io.BufferedInputStream.fill() @bci=175, line=218 (Interpreted frame) - java.io.BufferedInputStream.read1(byte[], int, int) @bci=44, line=258 (Interpreted frame) - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=317 (Interpreted frame) - sun.security.provider.SeedGenerator$URLSeedGenerator.getSeedByte() @bci=12, line=453 (Interpreted frame) - sun.security.provider.SeedGenerator.getSeedBytes(byte[]) @bci=11, line=123 (Interpreted frame) - sun.security.provider.SeedGenerator.generateSeed(byte[]) @bci=4, line=118 (Interpreted frame) - sun.security.provider.SecureRandom.engineGenerateSeed(int) @bci=5, line=114 (Interpreted frame) - sun.security.provider.SecureRandom.engineNextBytes(byte[]) @bci=40, line=171 (Interpreted frame) - java.security.SecureRandom.nextBytes(byte[]) @bci=5, line=433 (Interpreted frame) - java.security.SecureRandom.next(int) @bci=17, line=455 (Interpreted frame) - java.util.Random.nextLong() @bci=3, line=284 (Interpreted frame) - org.mortbay.jetty.servlet.HashSessionIdManager.doStart() @bci=73, line=139 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.mortbay.jetty.servlet.AbstractSessionManager.doStart() @bci=96, line=168 (Interpreted frame) - org.mortbay.jetty.servlet.HashSessionManager.doStart() @bci=12, line=67 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.mortbay.jetty.servlet.SessionHandler.doStart() @bci=4, line=115 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.mortbay.jetty.handler.HandlerWrapper.doStart() @bci=11, line=130 (Interpreted frame) - org.mortbay.jetty.handler.ContextHandler.startContext() @bci=1, line=537 (Interpreted frame) - org.mortbay.jetty.servlet.Context.startContext() @bci=1, line=136 (Interpreted frame) - org.mortbay.jetty.webapp.WebAppContext.startContext() @bci=123, line=1234 (Interpreted frame) - org.mortbay.jetty.handler.ContextHandler.doStart() @bci=140, line=517 (Interpreted frame) - org.mortbay.jetty.webapp.WebAppContext.doStart() @bci=170, line=460 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.mortbay.jetty.handler.HandlerCollection.doStart() @bci=32, line=152 (Interpreted frame) - org.mortbay.jetty.handler.ContextHandlerCollection.doStart() @bci=5, line=156 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.mortbay.jetty.handler.HandlerWrapper.doStart() @bci=11, line=130 (Interpreted frame) - org.mortbay.jetty.Server.doStart() @bci=201, line=222 (Interpreted frame) - org.mortbay.component.AbstractLifeCycle.start() @bci=31, line=50 (Interpreted frame) - org.apache.hadoop.http.HttpServer.start() @bci=383, line=461 (Interpreted frame) - org.apache.hadoop.mapred.TaskTracker.<init>(org.apache.hadoop.mapred.JobConf) @bci=519, line=931 (Interpreted frame) - org.apache.hadoop.mapred.TaskTracker.main(java.lang.String[]) @bci=52, line=2833 (Interpreted frame) Edson Ramiro On 6 April 2010 19:47, Todd Lipcon <t...@cloudera.com> wrote: > Hi Edson, > > Your attachments did not come through - can you put them on pastebin? > > -Todd > > On Tue, Apr 6, 2010 at 3:37 PM, Edson Ramiro <erlfi...@gmail.com> wrote: > > Hi Todd, > > > > I'm getting this behavior in another cluster too, there the same thing > > happens. > > and as I don't have a jstack installed in the first cluster and > > I'm not the admin, I'm sending the results of the second cluster. > > > > These are the results: > > > > [erl...@cohiba ~ ]$ jstack -l 22510 > > 22510: well-known file is not secure > > [erl...@cohiba ~ ]$ jstack -l 3836 > > 3836: well-known file is not secure > > > > The jstack -F result is in thread_dump files and the jstack -m result is > in > > java_native_frames. > > > > The files ending with nn are the namenode results and the files ending > with > > dn are the datanode results. > > > > Thanks, > > > > Edson Ramiro > > > > > > On 6 April 2010 18:19, Todd Lipcon <t...@cloudera.com> wrote: > >> > >> Hi Edson, > >> > >> Can you please run "jstack" on the daemons in question and paste the > >> output here? > >> > >> -Todd > >> > >> On Tue, Apr 6, 2010 at 12:44 PM, Edson Ramiro <erlfi...@gmail.com> > wrote: > >> > Hi all, > >> > > >> > I configured the Hadoop in a cluster and the NameNode and JobTracker > are > >> > running ok, but the DataNode and TaskTracker Doesn't start, they stop > >> > and > >> > keep waiting > >> > when they are going to start the Jetty > >> > > >> > I observed that Jetty can't start the _SelectChannelConnector_ > >> > > >> > Is there any Jetty configuration that should be changed ? > >> > > >> > There is no log message in the NN and JT when I try to start the DN > and > >> > TT. > >> > > >> > The kernel I'm using is: > >> > Linux bl05 2.6.32.10 #2 SMP Tue Apr 6 12:33:42 BRT 2010 x86_64 > GNU/Linux > >> > > >> > This is the message when I start the DN. It happens with TT too. > >> > > >> > ram...@bl05:~/hadoop-0.20.1+169.56$ ./bin/hadoop datanode > >> > 10/04/06 16:24:14 INFO datanode.DataNode: STARTUP_MSG: > >> > /************************************************************ > >> > STARTUP_MSG: Starting DataNode > >> > STARTUP_MSG: host = bl05.ctinfra.ufpr.br/192.168.1.115 > >> > STARTUP_MSG: args = [] > >> > STARTUP_MSG: version = 0.20.1+169.56 > >> > STARTUP_MSG: build = -r 8e662cb065be1c4bc61c55e6bff161e09c1d36f3; > >> > compiled by 'chad' on Tue Feb 2 13:27:17 PST 2010 > >> > ************************************************************/ > >> > 10/04/06 16:24:14 INFO datanode.DataNode: Registered > >> > FSDatasetStatusMBean > >> > 10/04/06 16:24:14 INFO datanode.DataNode: Opened info server at 50010 > >> > 10/04/06 16:24:14 INFO datanode.DataNode: Balancing bandwith is > 1048576 > >> > bytes/s > >> > 10/04/06 16:24:14 INFO mortbay.log: Logging to > >> > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > >> > org.mortbay.log.Slf4jLog > >> > 10/04/06 16:24:14 INFO http.HttpServer: Port returned by > >> > webServer.getConnectors()[0].getLocalPort() before open() is -1. > Opening > >> > the > >> > listener on 50075 > >> > 10/04/06 16:24:14 INFO http.HttpServer: listener.getLocalPort() > returned > >> > 50075 webServer.getConnectors()[0].getLocalPort() returned 50075 > >> > 10/04/06 16:24:14 INFO http.HttpServer: Jetty bound to port 50075 > >> > 10/04/06 16:24:14 INFO mortbay.log: jetty-6.1.14 > >> > > >> > Thanks in Advance, > >> > > >> > Edson Ramiro > >> > > >> > >> > >> > >> -- > >> Todd Lipcon > >> Software Engineer, Cloudera > > > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera >