Re: Possibly losing data with corrupted SSTables

2014-03-14 Thread Robert Coli
On Wed, Feb 12, 2014 at 9:20 AM, Francisco Nogueira Calmon Sobral < fsob...@igcorp.com.br> wrote: > I've removed the corrupted sstables and 'nodetool repair' ran successfully > for the column family. I'm not sure whether or not we've lost data. > If you read/write at CL.ONE, there is a non-zero c

Re: Possibly losing data with corrupted SSTables

2014-02-12 Thread sankalp kohli
You might want to look at this JIRA i filed today CASSANDRA-6696 You are good if you are fine with data reappearing. On Wed, Feb 12, 2014 at 9:20 AM, Francisco Nogueira Calmon Sobral < fsob...@igcorp.com.br> wrote: > Hi, Rahul. > > I've rem

Re: Possibly losing data with corrupted SSTables

2014-02-12 Thread Francisco Nogueira Calmon Sobral
Hi, Rahul. I've removed the corrupted sstables and 'nodetool repair' ran successfully for the column family. I'm not sure whether or not we've lost data. Best regards, Francisco Sobral On Jan 30, 2014, at 3:58 PM, Rahul Menon wrote: > Yes should delete all files related to -ib--.db > > Run

Re: Possibly losing data with corrupted SSTables

2014-01-30 Thread Rahul Menon
Yes should delete all files related to -ib--.db Run a repair after deletion On Thu, Jan 30, 2014 at 10:17 PM, Francisco Nogueira Calmon Sobral < fsob...@igcorp.com.br> wrote: > Ok. I'll try this idea with one sstable. But, should I delete all the > files associated with it? I mean, there is a d

Re: Possibly losing data with corrupted SSTables

2014-01-30 Thread Francisco Nogueira Calmon Sobral
Ok. I'll try this idea with one sstable. But, should I delete all the files associated with it? I mean, there is a difference in the number of files between the BAD sstable and a GOOD one, as I've already shown: BAD -- -rw-r--r-- 8 cassandra cassandra 991M Nov 8 15:11 Sessions-Users-ib-251

Re: Possibly losing data with corrupted SSTables

2014-01-30 Thread Rahul Menon
Looks like the sstables are corrupt. I dont believe there is a method to recover those sstables. I would delete them and run a repair to ensure data consistency. Rahul On Wed, Jan 29, 2014 at 11:29 PM, Francisco Nogueira Calmon Sobral < fsob...@igcorp.com.br> wrote: > Hi, Rahul. > > I've run no

Re: Possibly losing data with corrupted SSTables

2014-01-29 Thread Francisco Nogueira Calmon Sobral
Hi, Rahul. I've run nodetool upgradesstable only in the problematic CF. It throwed the following exception: Error occurred while upgrading the sstables for keyspace Sessions java.util.concurrent.ExecutionException: org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.IOException: d

Re: Possibly losing data with corrupted SSTables

2014-01-29 Thread Rahul Menon
Francisco, the sstables with *-ib-* is something that was from a previous version of c*. The *-ib-* naming convention started at c* 1.2.1 but 1.2.10 onwards im sure it has the *-ic-* convention. You could try running a nodetool sstableupgrade which should ideally upgrade the sstables with the *-ib

Possibly losing data with corrupted SSTables

2014-01-29 Thread Francisco Nogueira Calmon Sobral
Dear experts, We are facing a annoying problem in our cluster. We have 9 amazon extra large linux nodes, running Cassandra 1.2.11. The short story is that after moving the data from one cluster to another, we've been unable to run 'nodetool repair'. It get stuck due to a CorruptSSTableExceptio