And the final solution
http://unbxd.com/blog/2012/07/java-and-ksoftirqd-100-cpu-due-to-leap-second/
Doing $ date -s "`date`" solved the problem.
30.07.2012, 16:09, "Nikolay Kоvshov" :
> You mean using swap memory? I have total of 48G of RAM and Cassandra never
> used more than 2G, swap is d
You mean using swap memory? I have total of 48G of RAM and Cassandra never used
more than 2G, swap is disabled.
But as I have little clues, I can give this a try. Is there any fresh
instruction on running Cassandra with JNA ?
30.07.2012, 16:01, "Mateusz Korniak" :
> On Monday 30 of July 2012, N
On Monday 30 of July 2012, Nikolay Kоvshov wrote:
> - JNA is not installed on both machines
So your GC times may be strongly [1] affected by swapping.
IIRC, also snapshotting is more expensive and may trigger more swapping.
I would start with turning JNA mlockall on [2].
[1]:
Not sure if up to
- JNA is not installed on both machines
30.07.2012, 14:44, "Mateusz Korniak" :
> On Monday 30 of July 2012, Nikolay Kоvshov wrote:
>
>> What I plan to compare between 'bad' cluster and 'good' cluster:
>>
>> - Configs, schemas, data etc: same
>> - java version : same
>> - RAM and CPU : 'bad'
On Monday 30 of July 2012, Nikolay Kоvshov wrote:
> What I plan to compare between 'bad' cluster and 'good' cluster:
>
> - Configs, schemas, data etc: same
> - java version : same
> - RAM and CPU : 'bad' cluster has more
> - Ubuntu version
> - Networking
> - What else???
- JNA. If it's working an
There is one peculiar thing about this problem
When I copy the cluster to other 2 servers (much weaker servers in means of CPU
and RAM) and run cassandra there, I see no massive CPU load and no huge GC
times. Thus I suppose it has something to do with hardware or software
installed on the machi
On Thu, Jul 26, 2012 at 1:25 AM, Nikolay Kоvshov wrote:
>
> And I have remembered that I do a lot of get_range_slices requests, each
> time on a range of 100, if this could be important.
If you have large rows, try lowering the number of rows you fetch at once
from 100 to 25. Pulling in a larg
Sun Java 6 didn't help it at all
Sar shows no special activity on the long GC times
And I have remembered that I do a lot of get_range_slices requests, each time
on a range of 100, if this could be important.
Still my GC times are growing incredible
INFO [ScheduledTasks:1] 2012-07-25 12:17:04
Sun Java 6 didn't help it at all
Sar shows no special activity on the long GC times
And I have remembered that I do a lot of get_range_slices requests, each time
on a range of 100, if this could be important.
Still my GC times are growing incredible
INFO [ScheduledTasks:1] 2012-07-25 12:17:04
You are better off using Sun Java 6 to run Cassandra. In the past
there were issues reported on 7. Can you try running it on Sun Java 6?
kind regards
Joost
On Tue, Jul 24, 2012 at 10:04 AM, Nikolay Kоvshov wrote:
> 48 G of Ram on that machine, swap is not used. I will disable swap at all
> jus
I ran sar only recently after your advice and did not meet any huge GC-s on
that server
At 08:14 there was a GC lasting 4.5 seconds, that's not five minutes of course,
but also quite an unpleasant value;
Still I'm waiting for big GC values and will provide according sar logs.
07:25:01 PM pgp
48 G of Ram on that machine, swap is not used. I will disable swap at all just
in case
I have 4 cassandra processes (parts of 4 different clusters), each allocated 8
GB and using 4 of them
>java -version
java version "1.7.0"
Java(TM) SE Runtime Environment (build 1.7.0-b147)
Java HotSpot(TM) 64-
Can you provide output from sar command for the time period when long
GC occurred ?
Regards,
Wojciech Meler
Howmuch memory do you have on the machine. Seems like you have 8G
reserved for the Cassandra java process, If this is all the memory on
the machine you might be swapping. Also which jvm do you use?
kind regards
Joost
On Mon, Jul 23, 2012 at 10:07 AM, Nikolay Kоvshov wrote:
> 21th I have mirgat
21th I have mirgated to cassandra 1.1.2 but see no improvement
cat /var/log/cassandra/Earth1.log | grep "GC for"
INFO [ScheduledTasks:1] 2012-05-22 17:42:48,445 GCInspector.java (line 123) GC
for ParNew: 345 ms for 1 collections, 82451888 used; max is 8464105472
INFO [ScheduledTasks:1] 2012-05-
Assuming all the memory and yaml settings default that does not sound right.
The first thought would be the memory meter not counting correctly...
Do you do a lot of deletes ?
Do you have a lot of CF's and/or secondary indexes ?
Can you see log lines about the "liveRatio" for your cf's ?
I
I have similar problem with cassandra 0.8.10. After digging a while
I've found that my problem is somehow related to system page scanning
activities.
After turning on -Xloggc I've found high system cpu usage during
ParNew. After looking into sar -B I've found this:
08:00:08 AM pgpgin/s pgpgout/s
This is a cluster of 2 nodes, each having 8G of operating memory,
replicationfactor=2
Write/read pressure is quite low and almost never exceeds 10/second
>From time to time (2-3 times in a month) I see GC activity in logs and for
>this time cassandra stops responding to requests which results i
18 matches
Mail list logo
