Re: large system hint partition

2016-09-20 Thread Romain Hardouin
Hi, > More recent (I think 2.2) don't have this problem since they write hints to >the file system as per the commit log Flat files hints were implemented starting from 3.0   https://issues.apache.org/jira/browse/CASSANDRA-6230 Best, Romain

Re: large system hint partition

2016-09-19 Thread Graham Sanderson
The reason for large partitions is that the partition key is just the uuid of the target node More recent (I think 2.2) don't have this problem since they write hints to the file system as per the commit log Sadly the large partitions make things worse when you are hinting hence presumably und

Re: large system hint partition

2016-09-18 Thread Ezra Stuetzel
Yeah I tried that, but oddly the table had nothing in it. I changed the compaction strategy from leveled to sizetierd and ran a major compaction on each node. I haven't seen the message logged on any node in a few days which makes me think that fixed it because it is normally logged multiple times

Re: large system hint partition

2016-09-18 Thread Carlos Alonso
By inspecting the contents on your system.hints table, specifically the host_id column, you can see which is the destination host of those hints and check if it is one of the alive or dead ones. Carlos Alonso | Software Engineer | @calonso On 18 September 2016 at 04:

Re: large system hint partition

2016-09-17 Thread Ezra Stuetzel
Hey Nicolas, There are no dead nodes. 'nodetool status' and 'nodetool describecluster' both show 4 healthy nodes. In the past we have had some nodes we eliminated by using 'nodetool assassinate'. However, I checked system.peers table on all 4 of our nodes and they each show 3 peers as expected. So

Re: large system hint partition

2016-09-16 Thread Nicolas Douillet
Hi Erza, Have you a dead node in your cluster? Because the coordinator stores a hint about dead replicas in the local system.hints when a node is dead or didn't respond to a write request. -- Nicolas Le sam. 17 sept. 2016 à 00:12, Ezra Stuetzel a écrit : > What would be the likely causes of

large system hint partition

2016-09-16 Thread Ezra Stuetzel
What would be the likely causes of large system hint partitions? Normally large partition warnings are for user defined tables which they are writing large partitions to. In this case, it appears C* is writing large partitions to the system.hints table. Gossip is not backed up. version: C* 2.2.7