[ https://issues.apache.org/jira/browse/CASSANDRA-14711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Saurabh updated CASSANDRA-14711: -------------------------------- Description: Hi Team, I am using Apache Cassandra 3.2 with Java 1.8.0_161-b12.. Issue: Cassandra is continuously crashing with generating an HEAP dump log. There are no errors reported in system.log OR Debug.log. Exception in hs_err_PID.log: # Problematic frame: # J 8283 C2 org.apache.cassandra.db.marshal.TimestampType.compareCustom(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I (6 bytes) @ 0x00002b7d3d417fb4 [0x00002b7d3d417c80+0x334] #--------------- P R O C E S S --------------- Java Threads: ( => current thread ) 0x00002b7da57924a0 JavaThread "MemtableReclaimMemory:52" daemon [_thread_blocked, id=117880, stack(0x00002b7d917ff000,0x00002b7d91840000)] 0x00002b7d39f6a9e0 JavaThread "PerDiskMemtableFlushWriter_0:52" daemon [_thread_blocked, id=117879, stack(0x00002b7e4ea94000,0x00002b7e4ead5000)] 0x00002b7d39d0f520 JavaThread "MemtablePostFlush:53" daemon [_thread_blocked, id=117878, stack(0x00002b7e407dd000,0x00002b7e4081e000)] 0x00002b7df31a9150 JavaThread "MemtableFlushWriter:52" daemon [_thread_blocked, id=117877, stack(0x00002b7e406d9000,0x00002b7e4071a000)] 0x00002b7e53e60110 JavaThread "RMI TCP Connection(1795)-127.0.0.1" daemon : : lot of threads in BLOCKED status Other Threads: 0x00002b7d38de5ea0 VMThread [stack: 0x00002b7d8208d000,0x00002b7d8218d000] [id=32098] 0x00002b7d38fa9de0 WatcherThread [stack: 0x00002b7d88ee9000,0x00002b7d88fe9000] [id=32108] VM state:not at safepoint (normal execution) VM Mutex/Monitor currently owned by a thread: None Heap: garbage-first heap total 8388608K, used 6791168K [0x00000003c0000000, 0x00000003c0404000, 0x00000007c0000000) region size 4096K, 785 young (3215360K), 55 survivors (225280K) Metaspace used 40915K, capacity 42044K, committed 42368K, reserved 1087488K class space used 4429K, capacity 4646K, committed 4736K, reserved 1048576K Heap Regions: (Y=young(eden), SU=young(survivor), HS=humongous(starts), HC=humongous(continues), CS=collection set, F=free, TS=gc time stamp, PTAMS=previous top-at-mark-start, NTAMS=next top-at-mark-start) AC 0 O TS 0 PTAMS 0x00000003c0400000 NTAMS 0x00000003c0400000 space 4096K, 100% used [0x00000003c0000000, 0x00000003c0400000) AC 0 O TS 0 PTAMS 0x00000003c0800000 NTAMS 0x00000003c0800000 space 4096K, 100% used [0x00000003c0400000, 0x00000003c0800000) AC 0 O TS 9 PTAMS 0x00000003c0800000 NTAMS 0x00000003c0800000 space 4096K, 100% used [0x00000003c0800000, 0x00000003c0c00000) AC 0 O TS 11 PTAMS 0x00000003c0c00000 NTAMS 0x00000003c0c00000 space 4096K, 100% used [0x00000003c0c00000, 0x00000003c1000000) AC 0 O TS 11 PTAMS 0x00000003c1000000 NTAMS 0x00000003c1000000 space 4096K, 100% used [0x00000003c1000000, 0x00000003c1400000) AC 0 O TS 11 PTAMS 0x00000003c1400000 NTAMS 0x00000003c1400000 space 4096K, 100% used [0x00000003c1400000, 0x00000003c1800000) : : lot of such messages was: Hi Team, I am using Apache Cassandra 3.2 with Java 1.8.0_161-b12.. Issue: Cassandra is continuously crashing with generating an HEAP dump log. There are no errors reported in system.log OR Debug.log. Exception in hs_err_PID.log: # Problematic frame: # J 8283 C2 org.apache.cassandra.db.marshal.TimestampType.compareCustom(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I (6 bytes) @ 0x00002b7d3d417fb4 [0x00002b7d3d417c80+0x334] # --------------- T H R E A D --------------- Current thread (0x00002b7d3a1033e0): JavaThread "SharedPool-Worker-1" daemon [_thread_in_Java, id=32216, stack(0x00002b7e4085f000,0x00002b7e408a0000)] siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000014914c69 Registers: RAX=0x0000000000000001, RBX=0x0000000000000000, RCX=0x000000009f1fbef0, RDX=0x00000004f8fdf798 RSP=0x00002b7e4089e4b0, RBP=0x0000000000000001, RSI=0x0000000014907800, RDI=0x0000000000000000 R8 =0x000000000000d469, R9 =0x0000000000000000, R10=0x00000004a41764c8, R11=0x0000000000000000 R12=0x0000000000000000, R13=0x0000000000000000, R14=0x000000000000d469, R15=0x00002b7d3a1033e0 RIP=0x00002b7d3d417fb4, EFLAGS=0x0000000000010283, CSGSFS=0x0000000000000033, ERR=0x0000000000000004 TRAPNO=0x000000000000000e [error occurred during error reporting (printing register info), id 0xb] Stack: [0x00002b7e4085f000,0x00002b7e408a0000], sp=0x00002b7e4089e4b0, free space=253k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) J 8283 C2 org.apache.cassandra.db.marshal.TimestampType.compareCustom(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I (6 bytes) @ 0x00002b7d3d417fb4 [0x00002b7d3d417c80+0x334] J 12970 C2 org.apache.cassandra.db.Slice$Bound.compareTo(Lorg/apache/cassandra/db/ClusteringComparator;Ljava/util/List;)I (119 bytes) @ 0x00002b7d3e0291c0 [0x00002b7d3e028900+0x8c0] J 16245 C2 org.apache.cassandra.db.Slices$ArrayBackedSlices.intersects(Ljava/util/List;Ljava/util/List;)Z (46 bytes) @ 0x00002b7d3e619cfc [0x00002b7d3e619b20+0x1dc] J 18878 C2 org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(Lorg/apache/cassandra/db/ColumnFamilyStore;Z)Lorg/apache/cassandra/db/rows/UnfilteredRowIterator; (822 bytes) @ 0x00002b7d3ebcabf4 [0x00002b7d3ebc7be0+0x3014] J 9377 C2 org.apache.cassandra.db.ReadCommand.executeLocally(Lorg/apache/cassandra/db/ReadExecutionController;)Lorg/apache/cassandra/db/partitions/UnfilteredPartitionIterator; (219 bytes) @ 0x00002b7d3d80cde8 [0x00002b7d3d80c0a0+0xd48] J 14198 C2 org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(Lorg/apache/cassandra/net/MessageIn;I)V (328 bytes) @ 0x00002b7d3c8bcbd0 [0x00002b7d3c8bca20+0x1b0] J 9731 C2 org.apache.cassandra.net.MessageDeliveryTask.run()V (187 bytes) @ 0x00002b7d3d158d60 [0x00002b7d3d158bc0+0x1a0] J 18999% C2 org.apache.cassandra.concurrent.SEPWorker.run()V (253 bytes) @ 0x00002b7d3eaa10ec [0x00002b7d3eaa0960+0x78c] j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub V [libjvm.so+0x695ae6] JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x1056 V [libjvm.so+0x695ff1] JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x321 V [libjvm.so+0x696497] JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*, Symbol*, Thread*)+0x47 V [libjvm.so+0x731cb0] thread_entry(JavaThread*, Thread*)+0xa0 V [libjvm.so+0xa7eaa3] JavaThread::thread_main_inner()+0x103 V [libjvm.so+0xa7ebec] JavaThread::run()+0x11c V [libjvm.so+0x92da28] java_start(Thread*)+0x108 C [libpthread.so.0+0x7e25] start_thread+0xc5 --------------- P R O C E S S --------------- Java Threads: ( => current thread ) 0x00002b7da57924a0 JavaThread "MemtableReclaimMemory:52" daemon [_thread_blocked, id=117880, stack(0x00002b7d917ff000,0x00002b7d91840000)] 0x00002b7d39f6a9e0 JavaThread "PerDiskMemtableFlushWriter_0:52" daemon [_thread_blocked, id=117879, stack(0x00002b7e4ea94000,0x00002b7e4ead5000)] 0x00002b7d39d0f520 JavaThread "MemtablePostFlush:53" daemon [_thread_blocked, id=117878, stack(0x00002b7e407dd000,0x00002b7e4081e000)] 0x00002b7df31a9150 JavaThread "MemtableFlushWriter:52" daemon [_thread_blocked, id=117877, stack(0x00002b7e406d9000,0x00002b7e4071a000)] 0x00002b7e53e60110 JavaThread "RMI TCP Connection(1795)-127.0.0.1" daemon : : lot of threads in BLOCKED status Other Threads: 0x00002b7d38de5ea0 VMThread [stack: 0x00002b7d8208d000,0x00002b7d8218d000] [id=32098] 0x00002b7d38fa9de0 WatcherThread [stack: 0x00002b7d88ee9000,0x00002b7d88fe9000] [id=32108] VM state:not at safepoint (normal execution) VM Mutex/Monitor currently owned by a thread: None Heap: garbage-first heap total 8388608K, used 6791168K [0x00000003c0000000, 0x00000003c0404000, 0x00000007c0000000) region size 4096K, 785 young (3215360K), 55 survivors (225280K) Metaspace used 40915K, capacity 42044K, committed 42368K, reserved 1087488K class space used 4429K, capacity 4646K, committed 4736K, reserved 1048576K Heap Regions: (Y=young(eden), SU=young(survivor), HS=humongous(starts), HC=humongous(continues), CS=collection set, F=free, TS=gc time stamp, PTAMS=previous top-at-mark-start, NTAMS=next top-at-mark-start) AC 0 O TS 0 PTAMS 0x00000003c0400000 NTAMS 0x00000003c0400000 space 4096K, 100% used [0x00000003c0000000, 0x00000003c0400000) AC 0 O TS 0 PTAMS 0x00000003c0800000 NTAMS 0x00000003c0800000 space 4096K, 100% used [0x00000003c0400000, 0x00000003c0800000) AC 0 O TS 9 PTAMS 0x00000003c0800000 NTAMS 0x00000003c0800000 space 4096K, 100% used [0x00000003c0800000, 0x00000003c0c00000) AC 0 O TS 11 PTAMS 0x00000003c0c00000 NTAMS 0x00000003c0c00000 space 4096K, 100% used [0x00000003c0c00000, 0x00000003c1000000) AC 0 O TS 11 PTAMS 0x00000003c1000000 NTAMS 0x00000003c1000000 space 4096K, 100% used [0x00000003c1000000, 0x00000003c1400000) AC 0 O TS 11 PTAMS 0x00000003c1400000 NTAMS 0x00000003c1400000 space 4096K, 100% used [0x00000003c1400000, 0x00000003c1800000) : : lot of such messages > Apache Cassandra 3.2 crashing with exception > org.apache.cassandra.db.marshal.TimestampType.compareCustom > -------------------------------------------------------------------------------------------------------- > > Key: CASSANDRA-14711 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14711 > Project: Cassandra > Issue Type: Bug > Reporter: Saurabh > Priority: Minor > Attachments: hs_err_pid32069.log > > > Hi Team, > I am using Apache Cassandra 3.2 with Java 1.8.0_161-b12.. > Issue: > Cassandra is continuously crashing with generating an HEAP dump log. There > are no errors reported in system.log OR Debug.log. > Exception in hs_err_PID.log: > # Problematic frame: > # J 8283 C2 > org.apache.cassandra.db.marshal.TimestampType.compareCustom(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I > (6 bytes) @ 0x00002b7d3d417fb4 [0x00002b7d3d417c80+0x334] > #--------------- P R O C E S S --------------- > Java Threads: ( => current thread ) > 0x00002b7da57924a0 JavaThread "MemtableReclaimMemory:52" daemon > [_thread_blocked, id=117880, stack(0x00002b7d917ff000,0x00002b7d91840000)] > 0x00002b7d39f6a9e0 JavaThread "PerDiskMemtableFlushWriter_0:52" daemon > [_thread_blocked, id=117879, stack(0x00002b7e4ea94000,0x00002b7e4ead5000)] > 0x00002b7d39d0f520 JavaThread "MemtablePostFlush:53" daemon > [_thread_blocked, id=117878, stack(0x00002b7e407dd000,0x00002b7e4081e000)] > 0x00002b7df31a9150 JavaThread "MemtableFlushWriter:52" daemon > [_thread_blocked, id=117877, stack(0x00002b7e406d9000,0x00002b7e4071a000)] > 0x00002b7e53e60110 JavaThread "RMI TCP Connection(1795)-127.0.0.1" daemon > : > : > lot of threads in BLOCKED status > Other Threads: > 0x00002b7d38de5ea0 VMThread [stack: 0x00002b7d8208d000,0x00002b7d8218d000] > [id=32098] > 0x00002b7d38fa9de0 WatcherThread [stack: > 0x00002b7d88ee9000,0x00002b7d88fe9000] [id=32108] > VM state:not at safepoint (normal execution) > VM Mutex/Monitor currently owned by a thread: None > Heap: > garbage-first heap total 8388608K, used 6791168K [0x00000003c0000000, > 0x00000003c0404000, 0x00000007c0000000) > region size 4096K, 785 young (3215360K), 55 survivors (225280K) > Metaspace used 40915K, capacity 42044K, committed 42368K, reserved 1087488K > class space used 4429K, capacity 4646K, committed 4736K, reserved 1048576K > Heap Regions: (Y=young(eden), SU=young(survivor), HS=humongous(starts), > HC=humongous(continues), CS=collection set, F=free, TS=gc time stamp, > PTAMS=previous top-at-mark-start, NTAMS=next top-at-mark-start) > AC 0 O TS 0 PTAMS 0x00000003c0400000 NTAMS 0x00000003c0400000 space 4096K, > 100% used [0x00000003c0000000, 0x00000003c0400000) > AC 0 O TS 0 PTAMS 0x00000003c0800000 NTAMS 0x00000003c0800000 space 4096K, > 100% used [0x00000003c0400000, 0x00000003c0800000) > AC 0 O TS 9 PTAMS 0x00000003c0800000 NTAMS 0x00000003c0800000 space 4096K, > 100% used [0x00000003c0800000, 0x00000003c0c00000) > AC 0 O TS 11 PTAMS 0x00000003c0c00000 NTAMS 0x00000003c0c00000 space 4096K, > 100% used [0x00000003c0c00000, 0x00000003c1000000) > AC 0 O TS 11 PTAMS 0x00000003c1000000 NTAMS 0x00000003c1000000 space 4096K, > 100% used [0x00000003c1000000, 0x00000003c1400000) > AC 0 O TS 11 PTAMS 0x00000003c1400000 NTAMS 0x00000003c1400000 space 4096K, > 100% used [0x00000003c1400000, 0x00000003c1800000) > : > : > lot of such messages -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org