Startup fails due to cassandra trying to delete inexisting file
---------------------------------------------------------------
Key: CASSANDRA-2206
URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Environment: linux
Reporter: Thibaut
Fix For: 0.7.3
Hi,
On one of our nodes, startup fails due to cassandra trying to delete a
nonexistant "Data" file (see below).
Why that file is missing is another mistery... The log file entries don't show
any ERROR messages before cassandra restarted (for reasons I don't know) and
this error occured.
Directory listing:
total 109M
-rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
-rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
-rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
-rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
-rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
-rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
-rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
-rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
-rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
-rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
-rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
-rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
-rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
-rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
-rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
-rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
-rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
-rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
-rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
-rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
-rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
-rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
-rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
-rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
-rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
-rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
-rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
Cassandra log:
/software/cassandra/bin/cassandra
rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
INFO 12:47:29,030 Heap size: 2614493184/2614493184
INFO 12:47:29,031 JNA not found. Native methods will be disabled.
INFO 12:47:29,038 Loading settings from
file:/software/cassandra/conf/cassandra.yaml
INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
INFO 12:47:29,332 Creating new commitlog segment
/hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
WARN 12:47:29,623 Schema definitions were defined both locally and in
cassandra.yaml. Definitions in cassandra.yaml were ignored.
ERROR 12:47:29,638 Exception encountered during startup.
java.io.IOError: java.io.IOException: Failed to delete
/cassandra/data/table_task/table_task-tmp-f-1707-Data.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
at
org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
at
org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
Caused by: java.io.IOException: Failed to delete
/cassandra/data/table_task/table_task-tmp-f-1707-Data.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
... 4 more
Exception encountered during startup.
java.io.IOError: java.io.IOException: Failed to delete
/cassandra/data/table_task/table_task-tmp-f-1707-Data.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
at
org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
at
org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
Caused by: java.io.IOException: Failed to delete
/cassandra/data/table_task/table_task-tmp-f-1707-Data.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
