Re: Increasing size of Batch of prepared statements
Thanks Shane. best, /Shahab On Fri, Oct 3, 2014 at 6:51 PM, Shane Hansen shanemhan...@gmail.com wrote: It appears to be configurable in cassandra.yaml using batch_size_warn_threshold https://issues.apache.org/jira/browse/CASSANDRA-6487 On Fri, Oct 3, 2014 at 10:47 AM, shahab shahab.mok...@gmail.com wrote: Hi, I am getting the following warning in the cassandra log: BatchStatement.java:258 - Batch of prepared statements for [mydb.mycf] is of size 3272725, exceeding specified threshold of 5120 by 3267605. Apparently it relates to the (default) size of prepared insert statement . Is there any way to change the default value? thanks /Shahab
Why results of Cassandra Stress Toll is much worse than normal reading/writing from Cassandra?
Hi, I know that this question might look silly, but I really need to know how the cassandra stress tool works. I developed my data model and used Cassandra-Stress tool with u option where you pass your own data-model for column family (Table in CQL )and distribution of each column in the column family. Then I run it but when I compare the latency of it with what I could get from simple measurement of the time it took to execute a read/write operation for 100 cases. I could see that the results of Stress-Tool is much more (100 times) than what I have obtained in my own code. I know that stress test should stress the system and thus higher rates (.e.g. read/write latency) is reasonable, but 100 time more? I am sure I am missing something in interpreting the results of Stress-Tool (mainly the concept of latency). But there is very little vague documentation there. I do appreciate of any one could help me to understand the output of Stress-Tool? BTW, I have already seen this one (but still the documentation is quite poor): http://www.datastax.com/documentation/cassandra/2.1/cassandra/tools/toolsCStressOutput_c.html best, /Shahab
Re: Increasing size of Batch of prepared statements
Shabab,If you are hitting this limit because you are inserting a lot of (CQL) rows in a single batch I suggest you split the statement up in multiple smaller batches. Generally, large inserts like this will not perform very well. Cheers, Jens — Sent from Mailbox On Fri, Oct 3, 2014 at 6:47 PM, shahab shahab.mok...@gmail.com wrote: Hi, I am getting the following warning in the cassandra log: BatchStatement.java:258 - Batch of prepared statements for [mydb.mycf] is of size 3272725, exceeding specified threshold of 5120 by 3267605. Apparently it relates to the (default) size of prepared insert statement . Is there any way to change the default value? thanks /Shahab
Exploring Simply Queueing
Hi, I have put together some thoughts on realizing simple queues with Cassandra. https://github.com/algermissen/cassandra-ruby-queue The design is inspired by (the much more sophisticated) Netfilx approach[1] but very reduced. Given that I am still a C* newbie, I’d be very glad to hear some thoughts on the design path I took. Jan [1] https://github.com/Netflix/astyanax/wiki/Message-Queue
Re: Exploring Simply Queueing
It appears you are aware of the tombstones affect that leads people to label this an anti-pattern. Without due or any time based value being part of the partition key means you will still get a lot of buildup. You only have 1 partition per shard which just linearly decreases the tombstones. That isn't likely to be enough to really help in a situation of high queue throughput, especially with the default of 4 shards. You may want to consider switching to LCS from the default STCS since re-writing to same partitions a lot. It will still use STCS in L0 so in high write/delete scenarios, with low enough gc_grace, when it never gets higher then L1 it will be sameish write throughput. In scenarios where you get more LCS will shine I suspect by reducing number of obsolete tombstones. Would be hard to identify difference in small tests I think. Whats the plan to prevent two consumers from reading same message off of a queue? You mention in docs you will address it at a later point in time but its kinda a biggy. Big lock batch reads like astyanax recipe? --- Chris Lohfink On Oct 5, 2014, at 6:03 PM, Jan Algermissen jan.algermis...@nordsc.com wrote: Hi, I have put together some thoughts on realizing simple queues with Cassandra. https://github.com/algermissen/cassandra-ruby-queue The design is inspired by (the much more sophisticated) Netfilx approach[1] but very reduced. Given that I am still a C* newbie, I’d be very glad to hear some thoughts on the design path I took. Jan [1] https://github.com/Netflix/astyanax/wiki/Message-Queue
