Re: [Solved] column index offset miscalculation (was: Existing column(s) not readable)

2011-10-13 Thread Sylvain Lebresne
JIRA is not read-only, you should be able to create a ticket at https://issues.apache.org/jira/browse/CASSANDRA, though that probably require that you create an account. -- Sylvain On Thu, Oct 13, 2011 at 3:20 PM, Thomas Richter wrote: > Hi Aaron, > > the fix does the trick. I wonder why nobody

[Solved] column index offset miscalculation (was: Existing column(s) not readable)

2011-10-13 Thread Thomas Richter
Hi Aaron, the fix does the trick. I wonder why nobody else ran into this before... I checked org/apache/cassandra/db/ColumnIndexer.java in 0.7.9, 0.8.7 and 1.0.0-rc2 and all seem to be affected. Looks like public Jira is readonly - so I'm not sure about how to continue. Best, Thomas On 10/13/2

Re: Existing column(s) not readable

2011-10-13 Thread Thomas Richter
Hi Aaron, I guess i found it :-). I added logging for the used IndexInfo to SSTableNamesIterator.readIndexedColumns and got negative index postions for the missing columns. This is the reason why the columns are not loaded from sstable. So I had a look at ColumnIndexer.serializeInternal and ther

Re: Existing column(s) not readable

2011-10-12 Thread Thomas Richter
Hi Aaron, I cannot read the column with a slice query. The slice query only returns data till a certain column and after that i only get empty results. I added log output to QueryFilter.isRelevant to see if the filter is dropping the column(s) but it doesn't even show up there. Next thing i will

Re: Existing column(s) not readable

2011-10-11 Thread aaron morton
kewl, > * Row is not deleted (other columns can be read, row survives compaction > with GCGraceSeconds=0) IIRC row tombstones can hang around for a while (until gc grace has passed), and they only have an effect on columns that have a lower timstamp. So it's possible to read columns from a row

Re: Existing column(s) not readable

2011-10-11 Thread Thomas Richter
Hi Aaron, i invalidated the caches but nothing changed. I didn't get the mentioned log line either, but as I read the code SliceByNamesReadCommand uses NamesQueryFilter and not SliceQueryFilter. Next, there is only one SSTable. I can rule out that the row is deleted because I deleted all other r

Re: Existing column(s) not readable

2011-10-11 Thread aaron morton
Nothing jumps out. The obvious answer is that the column has been deleted. Did you check all the SSTables ? It looks like query returned from row cache, otherwise you would see this as well… DEBUG [ReadStage:34] 2011-10-11 21:11:11,484 SliceQueryFilter.java (line 123) collecting 0 of 214748364

Re: Existing column(s) not readable

2011-10-10 Thread Thomas Richter
Hi Aaron, normally we use hector to access cassandra, but for debugging I switched to cassandra-cli. Column can not be read by a simple get CFName['rowkey']['colname']; Response is "Value was not found" if i query another column, everything is just fine. Serverlog for unsuccessful read (keyspac

Re: Existing column(s) not readable

2011-10-10 Thread aaron morton
How are they unreadable ? You need to go into some details about what is going wrong. What sort of read ? What client ? What is in the logging on client and server side ? Try turning the logging up to DEBUG on the server to watch what happens. Cheers - Aaron Morton Freel

Re: Existing column(s) not readable

2011-10-10 Thread Thomas Richter
Hi, no errors in the server logs. The columns are unreadable on all nodes at any consistency level (ONE, QUORUM, ALL). We started with 0.7.3 and upgraded to 0.7.6-2 two months ago. Best, Thomas On 10/10/2011 10:03 AM, aaron morton wrote: > What error are you seeing in the server logs ? Are the

Re: Existing column(s) not readable

2011-10-10 Thread aaron morton
What error are you seeing in the server logs ? Are the columns unreadable at all Consistency Levels ? i.e. are the columns unreadable on all nodes. What is the upgrade history of the cluster ? What version did it start at ? Cheers - Aaron Morton Freelance Cassandra Developer

Re: Existing column(s) not readable

2011-10-09 Thread Thomas Richter
Hi, here is some further information. Compaction did not help, but data is still there when I dump the row with sstable2json. Best, Thomas On 10/08/2011 11:30 PM, Thomas Richter wrote: > Hi, > > we are running a 3 node cassandra (0.7.6-2) cluster and some of our > column families contain quite

Existing column(s) not readable

2011-10-08 Thread Thomas Richter
Hi, we are running a 3 node cassandra (0.7.6-2) cluster and some of our column families contain quite large rows (400k+ columns, 4-6GB row size). Replicaton factor is 3 for all keyspaces. The cluster is running fine for several months now and we never experienced any serious trouble. Some days ag