"What version of cassandra are you using ?" 1.1.2
"Can you explain this further?" I had an unexplained amount of reads (up to 1800 r/s and 90 Mo/s) on one server the other was doing about 200 r/s and 5 Mo/s max. I fixed it by rebooting the server. This server is dedicated to cassandra. I can't tell you more about it 'cause I don't get it... But a simple Cassandra restart wasn't enough. "Was something writing to the cluster ?" Yes we are having some activity and perform about 600 w/s. "Did you drain for the upgrade ?" We upgrade a long time ago and to 1.1.2. This warning is about the version 1.1.6. "What changes did you make ?" In the cassandra.yaml I just change the "compaction_throughput_mb_per_sec" property to slow down my compaction a bit. I don't think the problem come from here. "Are you saying that a particular counter column is giving different values for different reads ?" Yes, this is exactly what I was saying. Sorry if something is wrong with my English, it's not my mother tongue. "What CL are you using ?" I think this can be what causes the issue. I'm writing and reading at CL ONE. I didn't drain before stopping Cassandra and this may have produce a fail in the current counters (those which were being written when I stopped a server). But isn't Cassandra suppose to handle a server crash ? When a server crashes I guess it don't drain before... Thank you for your time Aaron, once again. Alain 2012/10/31 aaron morton <aa...@thelastpickle.com> > What version of cassandra are you using ? > > I finally restart Cassandra. It didn't solve the problem so I stopped >> Cassandra again on that node and restart my ec2 server. This solved the >> issue (1800 r/s to 100 r/s). > > Can you explain this further? > Was something writing to the cluster ? > Did you drain for the upgrade ? > https://github.com/apache/cassandra/blob/cassandra-1.1/NEWS.txt#L17 > > Today I changed my cassandra.yml and restart this same server to apply my >> conf. > > What changes did you make ? > > I just noticed that my homepage (which uses a Cassandra counter and >> refreshes every sec) shows me 4 different values. 2 of them repeatedly >> (5000 and 4000) and the 2 other some rare times (5500 and 3800) > > Are you saying that a particular counter column is giving different values > for different reads ? > What CL are you using ? > > Cheers > > ----------------- > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 31/10/2012, at 3:39 AM, Jason Wee <peich...@gmail.com> wrote: > > maybe enable the debug in log4j-server.properties and going through the > log to see what actually happen? > > On Tue, Oct 30, 2012 at 7:31 PM, Alain RODRIGUEZ <arodr...@gmail.com>wrote: > >> Hi, >> >> I have an issue with counters, yesterday I had a lot of ununderstandable >> reads/sec on one server. I finally restart Cassandra. It didn't solve the >> problem so I stopped Cassandra again on that node and restart my ec2 >> server. This solved the issue (1800 r/s to 100 r/s). >> >> Today I changed my cassandra.yml and restart this same server to apply my >> conf. >> >> I just noticed that my homepage (which uses a Cassandra counter and >> refreshes every sec) shows me 4 different values. 2 of them repeatedly >> (5000 and 4000) and the 2 other some rare times (5500 and 3800) >> >> Only the counters made today and yesterday are concerned. >> >> I performed a repair without success. These data are the heart of our >> business so if someone had any clue on it, I would be really grateful... >> >> The sooner the better, I am in production with these random counters. >> >> Alain >> >> INFO: >> >> My environnement is 2 nodes (EC2 large), RF 2, CL.ONE (R & W), Random >> Partitioner. >> >> xxx.xxx.xxx.241 eu-west 1b Up Normal 151.95 GB >> 50.00% 0 >> xxx.xxx.xxx.109 eu-west 1b Up Normal 117.71 GB >> 50.00% 85070591730234615865843651857942052864 >> >> Here is my conf: http://pastebin.com/5cMuBKDt >> >> >> > >
