Re: Restarting nodes and reported load

Regarding mtime, I'm just talking about using something like the following (assuming you are on linux) "find *pathtoyourdatadir *-mtime -1 -ls" which will find all files in your datadir last modifed within the past 24h. You can compare increase in your reported nodetool load within the past N days

Re: Restarting nodes and reported load

I'll try to capture answer to questions in the last 2 messages.Network traffic looks pretty steady overall. About 0.5 up to 2 megabytes/s. The cluster handles about 100k to 500k operations per minute, right now the read/write comparison is about 50/50 right now, eventually though it will probably b

Re: How to avoid flush if the data can fit into memtable

Kevin, Stefan thanks for the positive feedback and questions. Stefan in the blog post I am writing generally based on Apache Cassandra defaults. The meltable cleanup threshold is 1/(1+ memtable_flush_writers). By default the meltable_flush_writers defaults to two. This comes to 33 percent of th

Re: Cassandra Server 3.10 unable to Start after crash - commitlog needs to be removed

You can also manually delete the corrupt log file. Just check is name on the logs. Of course you are losing some data or not! Cheers On 01-06-2017 20:01, Peter Reilly wrote: > Please, how do you do this? > > Peter > > > On Fri, May 19, 2017 at 7:13 PM, Varun Gupta > wro

Re: Cassandra Server 3.10 unable to Start after crash - commitlog needs to be removed

Please, how do you do this? Peter On Fri, May 19, 2017 at 7:13 PM, Varun Gupta wrote: > Yes the bugs need to be fixed, but as a work around on dev environment, > you can enable cassandra.yaml option to override any corrupted commit log > file. > > > Thanks, > Varun > > > On May 19, 2017, at 11

Re: Restarting nodes and reported load

Hi Daniel, In my experience when a node shows DN and then comes back up by itself that sounds some sort of gc pause (especially if nodtool status when run from the "DN" node itself shows it is up-- assuming there isn't a spotty network issue). Perhaps I missed this info due to length of thread but

MemtablePostFlush pending

Hello Folks, I'm adding another node on my 14 node open source apache cassandra 2.2.8 cluster. New node is taking long time to join the cluster. I see there are bunch of pending [memtablepostflush] threads. I did increase memtable_flush_writers from 8 to 24 , though it is not helping with situa

Information on Cassandra

Hello All, I'm about 3 months into support several clusters of Cassandra databases. I recently subscribed to this email list and I receive lots of interesting emails most of which I don't understand. I feel like I have a pretty good grasp on Cassandra, I would like to know what types of this sh

Re: Restarting nodes and reported load

Some random thoughts; I would like to thank you for giving us an interesting problem. Cassandra can get boring sometimes, it is too stable. - Do you have a way to monitor the network traffic to see if it is increasing between restarts or does it seem relatively flat? - What activities are happenin

Re: Restoring a table cassandra - compactions

This is done to avoid overlap in levels > 0 There is this though: https://issues.apache.org/jira/browse/CASSANDRA-13425 If you are restoring an entire node, starting with an empty data directory, you should probably stop cassandra, copy the snapshot in, and restart, that will keep the levels On

Restoring a table cassandra - compactions

Hello. During the restore of a table using its snapshot and nodetool refresh, I could see that cassandra starts to make a lot of compactions (depending on the size of the data). I wanted to know why and I found this in the code of cassandra 2.1.14. for CASSANDRA-4872 +// force forei

Re: Restarting nodes and reported load

I am just restarting cassandra. I'm not having any disk space issues I think, but we're having issues where operations have increased latency, and these are fixed by a restart. It seemed like the load reported by nodetool status might be helpful in understanding what is going wrong but I'm not sure

Re: How to avoid flush if the data can fit into memtable

Hello Akhil, thanks for your great blog post. One thing I cannot bring together: In the answer mail you write: "Note the cleanup threshold is .50 of 1GB and not a combination of heap and off heap space." In your blog post you write: "memtable_cleanup_threshold is the default value i.e. 33 percent

Re: Convert single node C* to cluster (rebalancing problem)

Did you run "nodetool cleanup" on first node after second was bootstrapped? It should clean rows not belonging to node after tokens changed. Best regards, Vladimir Yudovin, Winguzone - Cloud Cassandra Hosting On Wed, 31 May 2017 03:55:54 -0400 Junaid Nasir wrote -

Re: Stable version apache cassandra 3.X /3.0.X

We are running 3.9x in prod, it has been pretty stable, but admittedly our footprint is pretty small(half a dozen nodes with less than hundred gigs of actual data) and not extremely tricky(no realized tables, etc). We have been running tick-tock since 3.3x and upgrading a could weeks after release

Re: Convert single node C* to cluster (rebalancing problem)

When you bootstrapped the node for the first time did you see log similar to the following: INFO [main] 2017-06-01 07:19:45,199 StorageService.java:1435 - JOINING: waiting for schema information to complete INFO [main] 2017-06-01 07:19:45,250 StorageService.java:1435 - JOINING: schema complete