Re: A Question About Hints

2019-03-04 Thread shalom sagges
Everyone really should move off of the 2.x versions just like you are doing. Tell me about it... But since there are a lot of groups involved, these things take time unfortunately. Thanks for your assistance Kenneth On Mon, Mar 4, 2019 at 11:04 PM Kenneth Brotman wrote: > Since you are in

RE: A Question About Hints

2019-03-04 Thread Kenneth Brotman
Since you are in the process of upgrading, I’d do nothing on the settings right now. But if you wanted to do something on the settings in the meantime, based on my read of the information available, I’d maybe double the default settings. The upgrade will help a lot of things as you know.

Re: A Question About Hints

2019-03-04 Thread shalom sagges
See my comments inline. Do the 8 nodes clusters have the problem too? Yes To the same extent? It depends on the throughput, but basically the smaller clusters get low throughput, so the problem is naturally smaller. Is it any cluster across multi-DC’s? Yes Do all the clusters use nodes

RE: A Question About Hints

2019-03-04 Thread Kenneth Brotman
Makes sense. If you have time and don’t mind, could you answer the following: Do the 8 nodes clusters have the problem too? To the same extent? Is it just the clusters with the large node count? Is it any cluster across multi-DC’s? Do all the clusters use nodes with similar specs?

Re: A Question About Hints

2019-03-04 Thread shalom sagges
It varies... Some clusters have 48 nodes, others 24 nodes and some 8 nodes. Both settings are on default. I’d try making a single conservative change to one or the other, measure and reassess. Then do same to other setting. That's the plan, but I thought I might first get some valuable

RE: A Question About Hints

2019-03-04 Thread Kenneth Brotman
It sounds like your use case might be appropriate for tuning those two settings some. How many nodes are in the cluster? Are both settings definitely on the default values currently? I’d try making a single conservative change to one or the other, measure and reassess. Then do same to

Re: A Question About Hints

2019-03-04 Thread shalom sagges
Hi Kenneth, The concern is that in some cases, hints accumulate on nodes, and it takes a while until they are delivered (multi DCs). I see that whenever there are a lot of hints in play,like after a rolling restart, the cluster works harder. That's why I want to decrease the hints delivery time.

RE: A Question About Hints

2019-03-04 Thread Kenneth Brotman
What is the concern? Why are you looking there? The casssandra.yml file has some notes about it. Did you read them? From: shalom sagges [mailto:shalomsag...@gmail.com] Sent: Monday, March 04, 2019 7:22 AM To: user@cassandra.apache.org Subject: A Question About Hints Hi All, Does

A Question About Hints

2019-03-04 Thread shalom sagges
Hi All, Does anyone know what is the most optimal hints configuration (multiple DCs) in terms of max_hints_delivery_threads and hinted_handoff_throttle_in_kb? If it's different for various use cases, is there a rule of thumb I can work with? I found this post but it's quite old:

RE: MV's stuck in build state

2019-03-04 Thread Kenneth Brotman
Hi Dipan, What is the state of the dev and production clusters now? How big is the production cluster? How many nodes on each cluster spin out of control? On the production cluster, the data is 67 MB, you'd have use a value at least twice that size as the commitlog_segment_size. Of

Re: MV's stuck in build state

2019-03-04 Thread Dipan Shah
Hello Kenneth, Apologies for the late reply. 1) On production the value of x was 67 MB and y was 16 MV as value of commitlog_segment_size_in_mb is 32. 2) On Dev the value of x was 18 MB and y was 16 MV as value of commitlog_segment_size_in_mb was 32 initially. I had bumped up the value of