Assuming all the memory and yaml settings default that does not sound right.
The first thought would be the memory meter not counting correctly... Do you do a lot of deletes ? Do you have a lot of CF's and/or secondary indexes ? Can you see log lines about the "liveRatio" for your cf's ? I would upgrade to 1.0.10 before getting too carried away though. Cheers ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/07/2012, at 8:14 PM, Nikolay Kоvshov wrote: > > This is a cluster of 2 nodes, each having 8G of operating memory, > replicationfactor=2 > Write/read pressure is quite low and almost never exceeds 10/second > > From time to time (2-3 times in a month) I see GC activity in logs and for > this time cassandra stops responding to requests which results in a timeout > in upper-layer application. Total time of unavailability can be over 5 minues > (like in the following case) > > What can I do with that? Wiil it become much worse when my cluster grows up? > > INFO [GossipTasks:1] 2012-07-16 13:10:50,055 Gossiper.java (line 736) > InetAddress /10.220.50.9 is now dead. > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,056 GCInspector.java (line 123) > GC for ParNew: 391383 ms for 1 collections, 2025808488 used; max is 8464105472 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,067 StatusLogger.java (line 50) > Pool Name Active Pending Blocked > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,071 StatusLogger.java (line 65) > ReadStage 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,071 StatusLogger.java (line 65) > RequestResponseStage 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,072 StatusLogger.java (line 65) > ReadRepairStage 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,072 StatusLogger.java (line 65) > MutationStage 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,073 StatusLogger.java (line 65) > ReplicateOnWriteStage 0 0 0 > INFO [GossipStage:1] 2012-07-16 13:10:50,074 Gossiper.java (line 722) > InetAddress /10.220.50.9 is now UP > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,075 StatusLogger.java (line 65) > GossipStage 1 59 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,075 StatusLogger.java (line 65) > AntiEntropyStage 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,076 StatusLogger.java (line 65) > MigrationStage 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,076 StatusLogger.java (line 65) > StreamStage 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,077 StatusLogger.java (line 65) > MemtablePostFlusher 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,077 StatusLogger.java (line 65) > FlushWriter 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,077 StatusLogger.java (line 65) > MiscStage 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,078 StatusLogger.java (line 65) > InternalResponseStage 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,078 StatusLogger.java (line 65) > HintedHandoff 0 0 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,079 StatusLogger.java (line 69) > CompactionManager n/a 0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,079 StatusLogger.java (line 81) > MessagingService n/a 0,0 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,080 StatusLogger.java (line 85) > ColumnFamily Memtable ops,data Row cache size/cap Key cache > size/cap > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,080 StatusLogger.java (line 88) > Keyspace1.PSS 36712,343842617 0/0 > 97995/1000000 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,085 StatusLogger.java (line 88) > Keyspace1.Standard1 128679,162567721 0/0 > 0/1000000 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,085 StatusLogger.java (line 88) > system.NodeIdInfo 0,0 0/0 > 0/1 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,086 StatusLogger.java (line 88) > system.IndexInfo 0,0 0/0 > 0/1 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,086 StatusLogger.java (line 88) > system.LocationInfo 0,0 0/0 > 1/2 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,086 StatusLogger.java (line 88) > system.Migrations 0,0 0/0 > 3/3 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,087 StatusLogger.java (line 88) > system.HintsColumnFamily 1,528 0/0 > 0/1 > INFO [ScheduledTasks:1] 2012-07-16 13:10:50,087 StatusLogger.java (line 88) > system.Schema 0,0 0/0 > 2/3 > -------- Завершение пересылаемого сообщения --------
