[jira] [Commented] (CASSANDRA-6298) Duplicate hard links; Cannot take snapshot (1.2)

2013-11-08 Thread Elias Ross (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13817670#comment-13817670
 ] 

Elias Ross commented on CASSANDRA-6298:
---

As it turns out, I was using symlinks. I had my last data directory pointing to 
the same data directory.

lrwxrwxrwx1 root root 6 Oct 30 18:37 data01 - /data1
lrwxrwxrwx1 root root 6 Oct 30 18:37 data02 - /data2
lrwxrwxrwx1 root root 6 Oct 30 18:37 data03 - /data3
lrwxrwxrwx1 root root 6 Oct 30 18:37 data04 - /data4
lrwxrwxrwx1 root root 6 Oct 30 18:37 data05 - /data5
lrwxrwxrwx1 root root 6 Nov  8 18:46 data06 - /data5

As you can see (not so clearly) the last link data06 is pointing to data5.

Maybe Cassandra can do some basic checking (at startup) to check that the data 
directories aren't repeated or pointing to the same location.

In any case, not a bug in the server.


 Duplicate hard links; Cannot take snapshot (1.2)
 

 Key: CASSANDRA-6298
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6298
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux, multiple data directories, Java 6
Reporter: Elias Ross

 I've been seeing this. It could be related to CASSANDRA-6093 in some way. The 
 version is 1.2.9.
 Caused by: java.lang.RuntimeException: Tried to create duplicate hard link to 
 /data05/rhq/data/system/NodeIdInfo/snapshots/1383588365276/system-NodeIdInfo-ic-
 1-TOC.txt
 at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:70)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1081)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1567)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1612)
 at org.apache.cassandra.db.Table.snapshot(Table.java:194)
 at 
 org.apache.cassandra.service.StorageService.takeSnapshot(StorageService.java:2203)
 $ ./nodetool -p 7299 snapshot
 Requested creating snapshot for: all keyspaces 
 Exception in thread main java.lang.RuntimeException: Tried to create 
 duplicate hard link to 
 /data06/rhq/data/system/NodeIdInfo/snapshots/1383602745114/system-NodeIdInfo-ic-1-Summary.db
 data:
 /data02/rhq/data/system/peers/snapshots
 /data02/rhq/data/system/peers/snapshots/1383602745114
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Data.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Statistics.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-TOC.txt
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Summary.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Filter.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Summary.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Data.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Digest.sha1
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-TOC.txt
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Index.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Index.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Digest.sha1
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Statistics.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Filter.db
 /data02/rhq/data/system/local/snapshots
 /data02/rhq/data/system/local/snapshots/1383602745114
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-TOC.txt
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Digest.sha1
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Index.db
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Summary.db
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Statistics.db
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Filter.db
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Data.db
 /data03/rhq/data/system/local/snapshots
 /data03/rhq/data/system/local/snapshots/1383602745114
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-Index.db
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-Statistics.db
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-Digest.sha1
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-TOC.txt
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-Summary.db

[jira] [Commented] (CASSANDRA-6298) Duplicate hard links; Cannot take snapshot (1.2)

2013-11-07 Thread Robert Coli (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13816890#comment-13816890
 ] 

Robert Coli commented on CASSANDRA-6298:


Per OP on cassandra-user ML, multiple data directories are in use, so this 
potentially relates to multiple data dir support. All filesystems are ext3.

multiple data dirs defined as follows :

{noformat}
data_file_directories:
- /data01/rhq/data
- /data02/rhq/data
- /data03/rhq/data
- /data04/rhq/data
- /data05/rhq/data
- /data06/rhq/data
{noformat}

 Duplicate hard links; Cannot take snapshot (1.2)
 

 Key: CASSANDRA-6298
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6298
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux, multiple data directories, Java 6
Reporter: Elias Ross

 I've been seeing this. It could be related to CASSANDRA-6093 in some way. The 
 version is 1.2.9.
 Caused by: java.lang.RuntimeException: Tried to create duplicate hard link to 
 /data05/rhq/data/system/NodeIdInfo/snapshots/1383588365276/system-NodeIdInfo-ic-
 1-TOC.txt
 at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:70)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1081)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1567)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1612)
 at org.apache.cassandra.db.Table.snapshot(Table.java:194)
 at 
 org.apache.cassandra.service.StorageService.takeSnapshot(StorageService.java:2203)
 $ ./nodetool -p 7299 snapshot
 Requested creating snapshot for: all keyspaces 
 Exception in thread main java.lang.RuntimeException: Tried to create 
 duplicate hard link to 
 /data06/rhq/data/system/NodeIdInfo/snapshots/1383602745114/system-NodeIdInfo-ic-1-Summary.db
 data:
 /data02/rhq/data/system/peers/snapshots
 /data02/rhq/data/system/peers/snapshots/1383602745114
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Data.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Statistics.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-TOC.txt
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Summary.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Filter.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Summary.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Data.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Digest.sha1
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-TOC.txt
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Index.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Index.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Digest.sha1
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-32-Statistics.db
 /data02/rhq/data/system/peers/snapshots/1383602745114/system-peers-ic-33-Filter.db
 /data02/rhq/data/system/local/snapshots
 /data02/rhq/data/system/local/snapshots/1383602745114
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-TOC.txt
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Digest.sha1
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Index.db
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Summary.db
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Statistics.db
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Filter.db
 /data02/rhq/data/system/local/snapshots/1383602745114/system-local-ic-58-Data.db
 /data03/rhq/data/system/local/snapshots
 /data03/rhq/data/system/local/snapshots/1383602745114
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-Index.db
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-Statistics.db
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-Digest.sha1
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-TOC.txt
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-Summary.db
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-Filter.db
 /data03/rhq/data/system/local/snapshots/1383602745114/system-local-ic-57-Data.db
 /data05/rhq/data/system/NodeIdInfo/snapshots
 /data05/rhq/data/system/NodeIdInfo/snapshots/1383602745114
 /data05/rhq/data/system/NodeIdInfo/snapshots/1383602745114/system-NodeIdInfo-ic-1-Data.db