Re: Increasing size of "Batch of prepared statements"

[shahab]

Thanks Tyler for sharing this. It is exactly what I was looking for to know.

best,
/Shahab

On Thu, Oct 23, 2014 at 5:37 PM, Tyler Hobbs  wrote:

> CASSANDRA-8091 (Stress tool creates too large batches) is relevant:
> https://issues.apache.org/jira/browse/CASSANDRA-8091
>
> On Thu, Oct 23, 2014 at 6:28 AM, shahab  wrote:
>
>> OK, Thanks again Jens.
>>
>> best,
>> /Shahab
>>
>> On Thu, Oct 23, 2014 at 1:22 PM, Jens Rantil  wrote:
>>
>>> Hi again Shabab,
>>>
>>> Yes, it seems that way. I have no experience with the “cassandra stress
>>> tool”, but wouldn’t be surprised if the batch size could be tweaked.
>>>
>>> Cheers,
>>> Jens
>>>
>>> ——— Jens Rantil Backend engineer Tink AB Email: jens.ran...@tink.se
>>> Phone: +46 708 84 18 32 Web: www.tink.se Facebook Linkedin Twitter
>>>
>>>
>>> On Thu, Oct 23, 2014 at 10:00 AM, shahab 
>>> wrote:
>>>
 Thanks Jens for the comments.

 As I am trying "cassandra stress tool", does it mean that the tool is
 executing batch of "Insert" statements (probably hundreds, or thousands)
  to the cassandra (for the sake of stressing Cassnadra ?

 best,
 /Shahab

 On Wed, Oct 22, 2014 at 8:14 PM, Jens Rantil 
 wrote:

>  Shabab,
>
> Apologize for the late answer.
>
> On Mon, Oct 6, 2014 at 2:38 PM, shahab 
> wrote:
>
>> But do you mean that inserting columns with large size (let's say a
>> text with 20-30 K) is potentially problematic in Cassandra?
>
>
> AFAIK, the size _warning_ you are getting relates to the size of the
> batch of prepared statements (INSERT INTO mykeyspace.mytable VALUES
> (?,?,?,?)). That is, it has nothing to do with the actual content of
> your row. 20-30 K shouldn't be a problem. But it's considered good 
> practise
> to split larger files (maybe > 5 MB into chunks) since it makes operations
> easier to your cluster more likely to spread more evenly across cluster.
>
>
>> What shall i do if I want columns with large size?
>
>
> Just don't insert to many rows in a single batch and you should be
> fine. Like Shane's JIRA ticket said, the warning is to let you know you 
> are
> not following best practice when adding too many rows in a single batch. 
> It
> can create bottlenecks in a single Cassandra node.
>
> Cheers,
> Jens
>
> --
> Jens Rantil
> Backend engineer
> Tink AB
>
> Email: jens.ran...@tink.se
> Phone: +46 708 84 18 32
> Web: www.tink.se
>
>  Facebook  Linkedin
> 
>  Twitter 
>


>>>
>>
>
>
> --
> Tyler Hobbs
> DataStax 
>

Re: Increasing size of "Batch of prepared statements"

[Tyler Hobbs]

CASSANDRA-8091 (Stress tool creates too large batches) is relevant:
https://issues.apache.org/jira/browse/CASSANDRA-8091

On Thu, Oct 23, 2014 at 6:28 AM, shahab  wrote:

> OK, Thanks again Jens.
>
> best,
> /Shahab
>
> On Thu, Oct 23, 2014 at 1:22 PM, Jens Rantil  wrote:
>
>> Hi again Shabab,
>>
>> Yes, it seems that way. I have no experience with the “cassandra stress
>> tool”, but wouldn’t be surprised if the batch size could be tweaked.
>>
>> Cheers,
>> Jens
>>
>> ——— Jens Rantil Backend engineer Tink AB Email: jens.ran...@tink.se
>> Phone: +46 708 84 18 32 Web: www.tink.se Facebook Linkedin Twitter
>>
>>
>> On Thu, Oct 23, 2014 at 10:00 AM, shahab  wrote:
>>
>>> Thanks Jens for the comments.
>>>
>>> As I am trying "cassandra stress tool", does it mean that the tool is
>>> executing batch of "Insert" statements (probably hundreds, or thousands)
>>>  to the cassandra (for the sake of stressing Cassnadra ?
>>>
>>> best,
>>> /Shahab
>>>
>>> On Wed, Oct 22, 2014 at 8:14 PM, Jens Rantil 
>>> wrote:
>>>
  Shabab,

 Apologize for the late answer.

 On Mon, Oct 6, 2014 at 2:38 PM, shahab  wrote:

> But do you mean that inserting columns with large size (let's say a
> text with 20-30 K) is potentially problematic in Cassandra?


 AFAIK, the size _warning_ you are getting relates to the size of the
 batch of prepared statements (INSERT INTO mykeyspace.mytable VALUES
 (?,?,?,?)). That is, it has nothing to do with the actual content of
 your row. 20-30 K shouldn't be a problem. But it's considered good practise
 to split larger files (maybe > 5 MB into chunks) since it makes operations
 easier to your cluster more likely to spread more evenly across cluster.


> What shall i do if I want columns with large size?


 Just don't insert to many rows in a single batch and you should be
 fine. Like Shane's JIRA ticket said, the warning is to let you know you are
 not following best practice when adding too many rows in a single batch. It
 can create bottlenecks in a single Cassandra node.

 Cheers,
 Jens

 --
 Jens Rantil
 Backend engineer
 Tink AB

 Email: jens.ran...@tink.se
 Phone: +46 708 84 18 32
 Web: www.tink.se

  Facebook  Linkedin
 
  Twitter 

>>>
>>>
>>
>


-- 
Tyler Hobbs
DataStax 

Re: Increasing size of "Batch of prepared statements"

[shahab]

OK, Thanks again Jens.

best,
/Shahab

On Thu, Oct 23, 2014 at 1:22 PM, Jens Rantil  wrote:

> Hi again Shabab,
>
> Yes, it seems that way. I have no experience with the “cassandra stress
> tool”, but wouldn’t be surprised if the batch size could be tweaked.
>
> Cheers,
> Jens
>
> ——— Jens Rantil Backend engineer Tink AB Email: jens.ran...@tink.se
> Phone: +46 708 84 18 32 Web: www.tink.se Facebook Linkedin Twitter
>
>
> On Thu, Oct 23, 2014 at 10:00 AM, shahab  wrote:
>
>> Thanks Jens for the comments.
>>
>> As I am trying "cassandra stress tool", does it mean that the tool is
>> executing batch of "Insert" statements (probably hundreds, or thousands)
>>  to the cassandra (for the sake of stressing Cassnadra ?
>>
>> best,
>> /Shahab
>>
>> On Wed, Oct 22, 2014 at 8:14 PM, Jens Rantil  wrote:
>>
>>>  Shabab,
>>>
>>> Apologize for the late answer.
>>>
>>> On Mon, Oct 6, 2014 at 2:38 PM, shahab  wrote:
>>>
 But do you mean that inserting columns with large size (let's say a
 text with 20-30 K) is potentially problematic in Cassandra?
>>>
>>>
>>> AFAIK, the size _warning_ you are getting relates to the size of the
>>> batch of prepared statements (INSERT INTO mykeyspace.mytable VALUES
>>> (?,?,?,?)). That is, it has nothing to do with the actual content of
>>> your row. 20-30 K shouldn't be a problem. But it's considered good practise
>>> to split larger files (maybe > 5 MB into chunks) since it makes operations
>>> easier to your cluster more likely to spread more evenly across cluster.
>>>
>>>
 What shall i do if I want columns with large size?
>>>
>>>
>>> Just don't insert to many rows in a single batch and you should be fine.
>>> Like Shane's JIRA ticket said, the warning is to let you know you are not
>>> following best practice when adding too many rows in a single batch. It can
>>> create bottlenecks in a single Cassandra node.
>>>
>>> Cheers,
>>> Jens
>>>
>>> --
>>> Jens Rantil
>>> Backend engineer
>>> Tink AB
>>>
>>> Email: jens.ran...@tink.se
>>> Phone: +46 708 84 18 32
>>> Web: www.tink.se
>>>
>>>  Facebook  Linkedin
>>> 
>>>  Twitter 
>>>
>>
>>
>

Re: Increasing size of "Batch of prepared statements"

[Jens Rantil]

Hi again Shabab,




Yes, it seems that way. I have no experience with the “cassandra stress tool”, 
but wouldn’t be surprised if the batch size could be tweaked.




Cheers,

Jens


———
Jens Rantil
Backend engineer
Tink AB

Email: jens.ran...@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook Linkedin Twitter

On Thu, Oct 23, 2014 at 10:00 AM, shahab  wrote:

> Thanks Jens for the comments.
> As I am trying "cassandra stress tool", does it mean that the tool is
> executing batch of "Insert" statements (probably hundreds, or thousands)
>  to the cassandra (for the sake of stressing Cassnadra ?
> best,
> /Shahab
> On Wed, Oct 22, 2014 at 8:14 PM, Jens Rantil  wrote:
>> Shabab,
>>
>> Apologize for the late answer.
>>
>> On Mon, Oct 6, 2014 at 2:38 PM, shahab  wrote:
>>
>>> But do you mean that inserting columns with large size (let's say a text
>>> with 20-30 K) is potentially problematic in Cassandra?
>>
>>
>> AFAIK, the size _warning_ you are getting relates to the size of the batch
>> of prepared statements (INSERT INTO mykeyspace.mytable VALUES (?,?,?,?)).
>> That is, it has nothing to do with the actual content of your row. 20-30 K
>> shouldn't be a problem. But it's considered good practise to split larger
>> files (maybe > 5 MB into chunks) since it makes operations easier to your
>> cluster more likely to spread more evenly across cluster.
>>
>>
>>> What shall i do if I want columns with large size?
>>
>>
>> Just don't insert to many rows in a single batch and you should be fine.
>> Like Shane's JIRA ticket said, the warning is to let you know you are not
>> following best practice when adding too many rows in a single batch. It can
>> create bottlenecks in a single Cassandra node.
>>
>> Cheers,
>> Jens
>>
>> --
>> Jens Rantil
>> Backend engineer
>> Tink AB
>>
>> Email: jens.ran...@tink.se
>> Phone: +46 708 84 18 32
>> Web: www.tink.se
>>
>> Facebook  Linkedin
>> 
>>  Twitter 
>>

Re: Increasing size of "Batch of prepared statements"

[shahab]

Thanks Jens for the comments.

As I am trying "cassandra stress tool", does it mean that the tool is
executing batch of "Insert" statements (probably hundreds, or thousands)
 to the cassandra (for the sake of stressing Cassnadra ?

best,
/Shahab

On Wed, Oct 22, 2014 at 8:14 PM, Jens Rantil  wrote:

> Shabab,
>
> Apologize for the late answer.
>
> On Mon, Oct 6, 2014 at 2:38 PM, shahab  wrote:
>
>> But do you mean that inserting columns with large size (let's say a text
>> with 20-30 K) is potentially problematic in Cassandra?
>
>
> AFAIK, the size _warning_ you are getting relates to the size of the batch
> of prepared statements (INSERT INTO mykeyspace.mytable VALUES (?,?,?,?)).
> That is, it has nothing to do with the actual content of your row. 20-30 K
> shouldn't be a problem. But it's considered good practise to split larger
> files (maybe > 5 MB into chunks) since it makes operations easier to your
> cluster more likely to spread more evenly across cluster.
>
>
>> What shall i do if I want columns with large size?
>
>
> Just don't insert to many rows in a single batch and you should be fine.
> Like Shane's JIRA ticket said, the warning is to let you know you are not
> following best practice when adding too many rows in a single batch. It can
> create bottlenecks in a single Cassandra node.
>
> Cheers,
> Jens
>
> --
> Jens Rantil
> Backend engineer
> Tink AB
>
> Email: jens.ran...@tink.se
> Phone: +46 708 84 18 32
> Web: www.tink.se
>
> Facebook  Linkedin
> 
>  Twitter 
>

Re: Increasing size of "Batch of prepared statements"

[Jens Rantil]

Shabab,

Apologize for the late answer.

On Mon, Oct 6, 2014 at 2:38 PM, shahab  wrote:

> But do you mean that inserting columns with large size (let's say a text
> with 20-30 K) is potentially problematic in Cassandra?


AFAIK, the size _warning_ you are getting relates to the size of the batch
of prepared statements (INSERT INTO mykeyspace.mytable VALUES (?,?,?,?)).
That is, it has nothing to do with the actual content of your row. 20-30 K
shouldn't be a problem. But it's considered good practise to split larger
files (maybe > 5 MB into chunks) since it makes operations easier to your
cluster more likely to spread more evenly across cluster.


> What shall i do if I want columns with large size?


Just don't insert to many rows in a single batch and you should be fine.
Like Shane's JIRA ticket said, the warning is to let you know you are not
following best practice when adding too many rows in a single batch. It can
create bottlenecks in a single Cassandra node.

Cheers,
Jens

-- 
Jens Rantil
Backend engineer
Tink AB

Email: jens.ran...@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook  Linkedin

 Twitter 

Re: Increasing size of "Batch of prepared statements"

[shahab]

Thanks Jens for the comment. Actually I am using Cassandra Stress Tool and
this is the tools who inserts such a large statements.

But do you mean that inserting columns with large size (let's say a text
with 20-30 K) is potentially problematic in Cassandra? What shall i do if I
want columns with large size?

best,
/Shahab

On Sun, Oct 5, 2014 at 6:03 PM, Jens Rantil  wrote:

> Shabab,
> If you are hitting this limit because you are inserting a lot of (CQL)
> rows in a single batch I suggest you split the statement up in multiple
> smaller batches. Generally, large inserts like this will not perform very
> well.
>
> Cheers,
> Jens
>
> —
> Sent from Mailbox 
>
>
> On Fri, Oct 3, 2014 at 6:47 PM, shahab  wrote:
>
>> Hi,
>>
>> I am getting the following warning in the cassandra log:
>> " BatchStatement.java:258 - Batch of prepared statements for [mydb.mycf]
>> is of size 3272725, exceeding specified threshold of 5120 by 3267605."
>>
>> Apparently it relates to the (default) size of prepared  insert statement
>> . Is there any way to change the default value?
>>
>> thanks
>> /Shahab
>>
>
>

Re: Increasing size of "Batch of prepared statements"

[Jens Rantil]

Shabab,If you are hitting this limit because you are inserting a lot of (CQL) 
rows in a single batch I suggest you split the statement up in multiple smaller 
batches. Generally, large inserts like this will not perform very well.




Cheers,

Jens


—
Sent from Mailbox

On Fri, Oct 3, 2014 at 6:47 PM, shahab  wrote:

> Hi,
> I am getting the following warning in the cassandra log:
> " BatchStatement.java:258 - Batch of prepared statements for [mydb.mycf] is
> of size 3272725, exceeding specified threshold of 5120 by 3267605."
> Apparently it relates to the (default) size of prepared  insert statement .
> Is there any way to change the default value?
> thanks
> /Shahab

Re: Increasing size of "Batch of prepared statements"

[shahab]

Thanks Shane.

best,
/Shahab

On Fri, Oct 3, 2014 at 6:51 PM, Shane Hansen  wrote:

> It appears to be configurable in cassandra.yaml
> using batch_size_warn_threshold
>
> https://issues.apache.org/jira/browse/CASSANDRA-6487
>
>
> On Fri, Oct 3, 2014 at 10:47 AM, shahab  wrote:
>
>> Hi,
>>
>> I am getting the following warning in the cassandra log:
>> " BatchStatement.java:258 - Batch of prepared statements for [mydb.mycf]
>> is of size 3272725, exceeding specified threshold of 5120 by 3267605."
>>
>> Apparently it relates to the (default) size of prepared  insert statement
>> . Is there any way to change the default value?
>>
>> thanks
>> /Shahab
>>
>
>

Re: Increasing size of "Batch of prepared statements"

[Shane Hansen]

It appears to be configurable in cassandra.yaml
using batch_size_warn_threshold

https://issues.apache.org/jira/browse/CASSANDRA-6487


On Fri, Oct 3, 2014 at 10:47 AM, shahab  wrote:

> Hi,
>
> I am getting the following warning in the cassandra log:
> " BatchStatement.java:258 - Batch of prepared statements for [mydb.mycf]
> is of size 3272725, exceeding specified threshold of 5120 by 3267605."
>
> Apparently it relates to the (default) size of prepared  insert statement
> . Is there any way to change the default value?
>
> thanks
> /Shahab
>

Increasing size of "Batch of prepared statements"

[shahab]

Hi,

I am getting the following warning in the cassandra log:
" BatchStatement.java:258 - Batch of prepared statements for [mydb.mycf] is
of size 3272725, exceeding specified threshold of 5120 by 3267605."

Apparently it relates to the (default) size of prepared  insert statement .
Is there any way to change the default value?

thanks
/Shahab