Hi all - cluster had a power outage and one of the nodes in a 14 nodes
cluster isn't starting with:
DEBUG [MemtableFlushWriter:1] 2023-02-03 13:52:45,468
ColumnFamilyStore.java:1329 - Flushed to
[BigTableReader(path='/data/2/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8407-big-Data.db'),
BigTableReader(path='/data/3/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8408-big-Data.db'),
BigTableReader(path='/data/4/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8409-big-Data.db'),
BigTableReader(path='/data/5/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8410-big-Data.db'),
BigTableReader(path='/data/6/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8411-big-Data.db'),
BigTableReader(path='/data/8/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8412-big-Data.db'),
BigTableReader(path='/data/9/cassandra/data/doc/correlation_counts-ca4e8c0080b311edbd6d4d9b3bfd78b2/nb-8413-big-Data.db')]
(7 sstables, 92.858MiB), biggest 15.420MiB, smallest 10.307MiB
INFO [main] 2023-02-03 13:52:45,621 CommitLogReader.java:257 - Finished
reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126163.log
DEBUG [main] 2023-02-03 13:52:45,622 CommitLogReader.java:266 - Reading
/var/lib/cassandra/commitlog/CommitLog-7-1674161126164.log (CL version
7, messaging version 12, compression null)
INFO [main] 2023-02-03 13:52:46,811 CommitLogReader.java:257 - Finished
reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126164.log
DEBUG [main] 2023-02-03 13:52:46,811 CommitLogReader.java:266 - Reading
/var/lib/cassandra/commitlog/CommitLog-7-1674161126165.log (CL version
7, messaging version 12, compression null)
INFO [main] 2023-02-03 13:52:47,985 CommitLogReader.java:257 - Finished
reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126165.log
DEBUG [main] 2023-02-03 13:52:47,986 CommitLogReader.java:266 - Reading
/var/lib/cassandra/commitlog/CommitLog-7-1674161126166.log (CL version
7, messaging version 12, compression null)
INFO [main] 2023-02-03 13:52:49,282 CommitLogReader.java:257 - Finished
reading /var/lib/cassandra/commitlog/CommitLog-7-1674161126166.log
DEBUG [main] 2023-02-03 13:52:49,283 CommitLogReader.java:266 - Reading
/var/lib/cassandra/commitlog/CommitLog-7-1674161126167.log (CL version
7, messaging version 12, compression null)
ERROR [main] 2023-02-03 13:52:49,651 JVMStabilityInspector.java:196 -
Exiting due to error while processing commit log during initialization.
org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException:
Mutation checksum failure at 11231154 in Next section at 11230925 in
CommitLog-7-1674161126167.log
at
org.apache.cassandra.db.commitlog.CommitLogReader.readSection(CommitLogReader.java:387)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:244)
at
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:147)
at
org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:191)
at
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:200)
at
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:181)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:357)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:752)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:876)
How to proceed?
Thank you!
-Joe
--
This email has been checked for viruses by AVG antivirus software.
www.avg.com
