Re: Cassandra Config as per server hardware for heavy write

[Manoj Khangaonkar]

Hi,

What is your write consistency setting ?

regards

On Wed, Nov 23, 2016 at 3:48 AM, Vladimir Yudovin <vla...@winguzone.com>
wrote:

> Try to build cluster with *.withPoolingOptions*
>
> Best regards, Vladimir Yudovin,
> *Winguzone <https://winguzone.com?from=list> - Cloud Cassandra Hosting*
>
>
>  On Wed, 23 Nov 2016 05:57:58 -0500*Abhishek Kumar Maheshwari
> <abhishek.maheshw...@timesinternet.in
> <abhishek.maheshw...@timesinternet.in>>* wrote 
>
> Yes, i also try with async mode but I got max speed on 2500 request/sec
> per server.
>
>
>
>ExecutorService service=Executors.*newFixedThreadPool*(1000);
>
> *for*(*final* AdLog adLog:li){
>
> service.submit(()->{
>
> *session*.executeAsync(ktest.adImprLogToStatement(adLog.getAdLogType(),
> adLog.getAdImprLog()));
>
> inte.incrementAndGet();
>
>  });
>
>   }
>
>
>
> *Thanks & Regards,*
> *Abhishek Kumar Maheshwari*
> *+91- 805591 <%2B91-%C2%A0805591> (Mobile)*
>
> Times Internet Ltd. | A Times of India Group Company
>
> FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
>
> *P** Please do not print this email unless it is absolutely necessary.
> Spread environmental awareness.*
>
>
>
> *From:* Benjamin Roth [mailto:benjamin.r...@jaumo.com]
> *Sent:* Wednesday, November 23, 2016 4:09 PM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Cassandra Config as per server hardware for heavy write
>
>
>
> This has nothing to do with sync/async operations. An async operation is
> also replayable. You receive the result in a future instead.
>
> Have you ever dealt with async programming techniques like promises,
> futures, callbacks?
>
> Async programming does not change the fact that you get a result of your
> operation only WHERE and WHEN.
>
> Doing sync operations means the result is available in the "next line of
> code" whereas async operation means that some handler is called when the
> result is there.
>
>
>
> There are tons of articles around this in the web.
>
>
>
> 2016-11-23 11:29 GMT+01:00 Abhishek Kumar Maheshwari <Abhishek.Maheshwari@
> timesinternet.in>:
>
> But I need to do it in sync mode as per business requirement. If something
> went wrong then it should be replayle. That’s why I am using sync mode.
>
>
>
> *Thanks & Regards,*
> *Abhishek Kumar Maheshwari*
> *+91- 805591 (Mobile)*
>
> Times Internet Ltd. | A Times of India Group Company
>
> FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
>
> *P** Please do not print this email unless it is absolutely necessary.
> Spread environmental awareness.*
>
>
>
> *From:* Vladimir Yudovin [mailto:vla...@winguzone.com]
> *Sent:* Wednesday, November 23, 2016 3:47 PM
> *To:* user <user@cassandra.apache.org>
> *Subject:* RE: Cassandra Config as per server hardware for heavy write
>
>
>
> session.execute is coming from Session session = cluster.connect(); I
> guess?
>
>
>
> So actually all threads work with the same TCP connection. It's worth to
> try async API with Connection Pool.
>
>
>
> Best regards, Vladimir Yudovin,
>
> *Winguzone <https://winguzone.com?from=list> - Cloud Cassandra Hosting*
>
>
>
>
>
>  On Wed, 23 Nov 2016 04:49:18 -0500*Abhishek Kumar Maheshwari <
> <abhishek.maheshw...@timesinternet.in>abhishek.maheshw...@timesinternet.in
> <abhishek.maheshw...@timesinternet.in>**>* wrote 
>
>
>
> Hi
>
>
>
> I am submitting record to Executor service and below is my client config
> and code:
>
>
>
> cluster = Cluster.*builder*().addContactPoints(hostAddresses)
>
>.withRetryPolicy(DefaultRetryPolicy.*INSTANCE*)
>
>.withReconnectionPolicy(*new*
> ConstantReconnectionPolicy(3L))
>
>.withLoadBalancingPolicy(*new* TokenAwarePolicy(*new*
> DCAwareRoundRobinPolicy()))
>
>.build();
>
>
>
>ExecutorService service=Executors.*newFixedThreadPool*(1000);
>
> *for*(*final* AdLog adLog:li){
>
> service.submit(()->{
>
> *session*.execute(ktest.adImprLogToStatement(adLog.getAdLogType(),adLog.
> getAdImprLog()));
>
> inte.incrementAndGet();
>
>      });
>
>   }
>
>
>
>
>
>
>
> *Thanks & Regards,*
> *Abhishek Kumar Maheshwari*
> *+91- 805591 (Mobile)*
>
> Times Internet Ltd. | A Times of India Group Company
>
> FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
>
> *P** Please do not print th

RE: Cassandra Config as per server hardware for heavy write

[Vladimir Yudovin]

Try to build cluster with .withPoolingOptions



Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting






 On Wed, 23 Nov 2016 05:57:58 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 




Yes, i also try with async mode but I got max speed on 2500 request/sec per 
server.

 

   ExecutorService service=Executors.newFixedThreadPool(1000);

for(final AdLog adLog:li){

service.submit(()-{

session.executeAsync(ktest.adImprLogToStatement(adLog.getAdLogType(),adLog.getAdImprLog()));

inte.incrementAndGet();

 });

  }

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

 

From: Benjamin Roth [mailto:benjamin.r...@jaumo.com] 
 Sent: Wednesday, November 23, 2016 4:09 PM
 To: user@cassandra.apache.org
 Subject: Re: Cassandra Config as per server hardware for heavy write
 

This has nothing to do with sync/async operations. An async operation is also 
replayable. You receive the result in a future instead.

Have you ever dealt with async programming techniques like promises, futures, 
callbacks?


Async programming does not change the fact that you get a result of your 
operation only WHERE and WHEN.


Doing sync operations means the result is available in the "next line of code" 
whereas async operation means that some handler is called when the result is 
there.


 


There are tons of articles around this in the web.



 

2016-11-23 11:29 GMT+01:00 Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in:

But I need to do it in sync mode as per business requirement. If something went 
wrong then it should be replayle. That’s why I am using sync mode.

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.


 

From: Vladimir Yudovin [mailto:vla...@winguzone.com] 
 Sent: Wednesday, November 23, 2016 3:47 PM
 To: user user@cassandra.apache.org
 Subject: RE: Cassandra Config as per server hardware for heavy write


 

session.execute is coming from Session session = cluster.connect(); I guess?


 


So actually all threads work with the same TCP connection. It's worth to try 
async API with Connection Pool.


 


Best regards, Vladimir Yudovin,


Winguzone - Cloud Cassandra Hosting



 


 


 On Wed, 23 Nov 2016 04:49:18 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 



 


Hi

 

I am submitting record to Executor service and below is my client config and 
code:

 

cluster = Cluster.builder().addContactPoints(hostAddresses)

   .withRetryPolicy(DefaultRetryPolicy.INSTANCE)

   .withReconnectionPolicy(new 
ConstantReconnectionPolicy(3L))

   .withLoadBalancingPolicy(new TokenAwarePolicy(new 
DCAwareRoundRobinPolicy()))

   .build();

 

   ExecutorService service=Executors.newFixedThreadPool(1000);

for(final AdLog adLog:li){

service.submit(()-{

session.execute(ktest.adImprLogToStatement(adLog.getAdLogType(),adLog.getAdImprLog()));

inte.incrementAndGet();

 });

  }

 

 

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.


 

From: Vladimir Yudovin [mailto:vla...@winguzone.com] 
 Sent: Wednesday, November 23, 2016 3:15 PM
 To: user user@cassandra.apache.org
 Subject: RE: Cassandra Config as per server hardware for heavy write


 

I have a list with 1cr record. I am just iterating on it and executing the 
query. Also, I try with 200 thread


Do you fetch each list item and put it to separate thread to perform CQL query? 
Also how exactly do you connect to Cassandra?


If you use synchronous API so it's better to create connection pool (with 
TokenAwarePolicy each) and then pass each item to separate thread.


 


 


Best regards, Vladimir Yudovin,


Winguzone - Cloud Cassandra Hosting



 


 


 On Wed, 23 Nov 2016 04:23:13 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 



 


Hi Siddharth,

 

For me it seems Cassandra side. Because I have a list with 1cr record. I am 
just iterating on it and executing the query.

Also, I try with 200 thread but still speed doesn’t increase that much as 
expected. On grafana write latency is near about 10Ms.

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times In

RE: Cassandra Config as per server hardware for heavy write

[Abhishek Kumar Maheshwari]

Yes, i also try with async mode but I got max speed on 2500 request/sec per 
server.

   ExecutorService service=Executors.newFixedThreadPool(1000);
  for(final AdLog adLog:li){
 service.submit(()->{
  
session.executeAsync(ktest.adImprLogToStatement(adLog.getAdLogType(),adLog.getAdImprLog()));
   inte.incrementAndGet();
 });
  }

Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: Benjamin Roth [mailto:benjamin.r...@jaumo.com]
Sent: Wednesday, November 23, 2016 4:09 PM
To: user@cassandra.apache.org
Subject: Re: Cassandra Config as per server hardware for heavy write

This has nothing to do with sync/async operations. An async operation is also 
replayable. You receive the result in a future instead.
Have you ever dealt with async programming techniques like promises, futures, 
callbacks?
Async programming does not change the fact that you get a result of your 
operation only WHERE and WHEN.
Doing sync operations means the result is available in the "next line of code" 
whereas async operation means that some handler is called when the result is 
there.

There are tons of articles around this in the web.

2016-11-23 11:29 GMT+01:00 Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>:
But I need to do it in sync mode as per business requirement. If something went 
wrong then it should be replayle. That’s why I am using sync mode.

Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591<tel:%2B91-%C2%A0805591> (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: Vladimir Yudovin 
[mailto:vla...@winguzone.com<mailto:vla...@winguzone.com>]
Sent: Wednesday, November 23, 2016 3:47 PM
To: user <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: RE: Cassandra Config as per server hardware for heavy write

session.execute is coming from Session session = cluster.connect(); I guess?

So actually all threads work with the same TCP connection. It's worth to try 
async API with Connection Pool.

Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting


 On Wed, 23 Nov 2016 04:49:18 -0500Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote 

Hi

I am submitting record to Executor service and below is my client config and 
code:

cluster = Cluster.builder().addContactPoints(hostAddresses)
   .withRetryPolicy(DefaultRetryPolicy.INSTANCE)
   .withReconnectionPolicy(new 
ConstantReconnectionPolicy(3L))
   .withLoadBalancingPolicy(new TokenAwarePolicy(new 
DCAwareRoundRobinPolicy()))
   .build();

   ExecutorService service=Executors.newFixedThreadPool(1000);
for(final AdLog adLog:li){
service.submit(()->{
session.execute(ktest.adImprLogToStatement(adLog.getAdLogType(),adLog.getAdImprLog()));
inte.incrementAndGet();
 });
  }



Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591<tel:%2B91-%C2%A0805591> (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: Vladimir Yudovin 
[mailto:vla...@winguzone.com<mailto:vla...@winguzone.com>]
Sent: Wednesday, November 23, 2016 3:15 PM
To: user <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: RE: Cassandra Config as per server hardware for heavy write

>I have a list with 1cr record. I am just iterating on it and executing the 
>query. Also, I try with 200 thread
Do you fetch each list item and put it to separate thread to perform CQL query? 
Also how exactly do you connect to Cassandra?
If you use synchronous API so it's better to create connection pool (with 
TokenAwarePolicy each) and then pass each item to separate thread.


Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting


 On Wed, 23 Nov 2016 04:23:13 -0500Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote 

Hi Siddharth,

For me it seems Cassandra side. Because I have a list with 1cr record. I am 
just iterating on it and executing the query

Re: Cassandra Config as per server hardware for heavy write

[Benjamin Roth]

This has nothing to do with sync/async operations. An async operation is
also replayable. You receive the result in a future instead.
Have you ever dealt with async programming techniques like promises,
futures, callbacks?
Async programming does not change the fact that you get a result of your
operation only WHERE and WHEN.
Doing sync operations means the result is available in the "next line of
code" whereas async operation means that some handler is called when the
result is there.

There are tons of articles around this in the web.

2016-11-23 11:29 GMT+01:00 Abhishek Kumar Maheshwari <
abhishek.maheshw...@timesinternet.in>:

> But I need to do it in sync mode as per business requirement. If something
> went wrong then it should be replayle. That’s why I am using sync mode.
>
>
>
> *Thanks & Regards,*
> *Abhishek Kumar Maheshwari*
> *+91- 805591 <%2B91-%C2%A0805591> (Mobile)*
>
> Times Internet Ltd. | A Times of India Group Company
>
> FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
>
> *P** Please do not print this email unless it is absolutely necessary.
> Spread environmental awareness.*
>
>
>
> *From:* Vladimir Yudovin [mailto:vla...@winguzone.com]
> *Sent:* Wednesday, November 23, 2016 3:47 PM
> *To:* user <user@cassandra.apache.org>
> *Subject:* RE: Cassandra Config as per server hardware for heavy write
>
>
>
> session.execute is coming from Session session = cluster.connect(); I
> guess?
>
>
>
> So actually all threads work with the same TCP connection. It's worth to
> try async API with Connection Pool.
>
>
>
> Best regards, Vladimir Yudovin,
>
> *Winguzone <https://winguzone.com?from=list> - Cloud Cassandra Hosting*
>
>
>
>
>
>  On Wed, 23 Nov 2016 04:49:18 -0500*Abhishek Kumar Maheshwari
> <abhishek.maheshw...@timesinternet.in
> <abhishek.maheshw...@timesinternet.in>>* wrote 
>
>
>
> Hi
>
>
>
> I am submitting record to Executor service and below is my client config
> and code:
>
>
>
> cluster = Cluster.*builder*().addContactPoints(hostAddresses)
>
>.withRetryPolicy(DefaultRetryPolicy.*INSTANCE*)
>
>.withReconnectionPolicy(*new*
> ConstantReconnectionPolicy(3L))
>
>.withLoadBalancingPolicy(*new* TokenAwarePolicy(*new*
> DCAwareRoundRobinPolicy()))
>
>.build();
>
>
>
>ExecutorService service=Executors.*newFixedThreadPool*(1000);
>
> *for*(*final* AdLog adLog:li){
>
> service.submit(()->{
>
> *session*.execute(ktest.adImprLogToStatement(adLog.getAdLogType(),adLog.
> getAdImprLog()));
>
> inte.incrementAndGet();
>
>  });
>
>   }
>
>
>
>
>
>
>
> *Thanks & Regards,*
> *Abhishek Kumar Maheshwari*
> *+91- 805591 <%2B91-%C2%A0805591> (Mobile)*
>
> Times Internet Ltd. | A Times of India Group Company
>
> FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
>
> *P** Please do not print this email unless it is absolutely necessary.
> Spread environmental awareness.*
>
>
>
> *From:* Vladimir Yudovin [mailto:vla...@winguzone.com]
> *Sent:* Wednesday, November 23, 2016 3:15 PM
> *To:* user <user@cassandra.apache.org>
> *Subject:* RE: Cassandra Config as per server hardware for heavy write
>
>
>
> >I have a list with 1cr record. I am just iterating on it and executing
> the query. Also, I try with 200 thread
>
> Do you fetch each list item and put it to separate thread to perform CQL
> query? Also how exactly do you connect to Cassandra?
>
> If you use synchronous API so it's better to create connection pool (with
> TokenAwarePolicy each) and then pass each item to separate thread.
>
>
>
>
>
> Best regards, Vladimir Yudovin,
>
> *Winguzone <https://winguzone.com?from=list> - Cloud Cassandra Hosting*
>
>
>
>
>
>  On Wed, 23 Nov 2016 04:23:13 -0500*Abhishek Kumar Maheshwari
> <abhishek.maheshw...@timesinternet.in
> <abhishek.maheshw...@timesinternet.in>**>* wrote 
>
>
>
> Hi Siddharth,
>
>
>
> For me it seems Cassandra side. Because I have a list with 1cr record. I
> am just iterating on it and executing the query.
>
> Also, I try with 200 thread but still speed doesn’t increase that much as
> expected. On grafana write latency is near about 10Ms.
>
>
>
> *Thanks & Regards,*
> *Abhishek Kumar Maheshwari*
> *+91- 805591 <%2B91-%C2%A0805591> (Mobile)*
>
> Times Internet Ltd. | A Times of India Group Company
>
> FC - 6, Sector 16A, Film City,  Noida,  U.P. 2

RE: Cassandra Config as per server hardware for heavy write

[Abhishek Kumar Maheshwari]

But I need to do it in sync mode as per business requirement. If something went 
wrong then it should be replayle. That’s why I am using sync mode.

Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: Vladimir Yudovin [mailto:vla...@winguzone.com]
Sent: Wednesday, November 23, 2016 3:47 PM
To: user <user@cassandra.apache.org>
Subject: RE: Cassandra Config as per server hardware for heavy write

session.execute is coming from Session session = cluster.connect(); I guess?

So actually all threads work with the same TCP connection. It's worth to try 
async API with Connection Pool.

Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting


 On Wed, 23 Nov 2016 04:49:18 -0500Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote 

Hi

I am submitting record to Executor service and below is my client config and 
code:

cluster = Cluster.builder().addContactPoints(hostAddresses)
   .withRetryPolicy(DefaultRetryPolicy.INSTANCE)
   .withReconnectionPolicy(new 
ConstantReconnectionPolicy(3L))
   .withLoadBalancingPolicy(new TokenAwarePolicy(new 
DCAwareRoundRobinPolicy()))
   .build();

   ExecutorService service=Executors.newFixedThreadPool(1000);
for(final AdLog adLog:li){
service.submit(()->{
session.execute(ktest.adImprLogToStatement(adLog.getAdLogType(),adLog.getAdImprLog()));
inte.incrementAndGet();
 });
  }



Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: Vladimir Yudovin 
[mailto:vla...@winguzone.com<mailto:vla...@winguzone.com>]
Sent: Wednesday, November 23, 2016 3:15 PM
To: user <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: RE: Cassandra Config as per server hardware for heavy write

>I have a list with 1cr record. I am just iterating on it and executing the 
>query. Also, I try with 200 thread
Do you fetch each list item and put it to separate thread to perform CQL query? 
Also how exactly do you connect to Cassandra?
If you use synchronous API so it's better to create connection pool (with 
TokenAwarePolicy each) and then pass each item to separate thread.


Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting


 On Wed, 23 Nov 2016 04:23:13 -0500Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote 

Hi Siddharth,

For me it seems Cassandra side. Because I have a list with 1cr record. I am 
just iterating on it and executing the query.
Also, I try with 200 thread but still speed doesn’t increase that much as 
expected. On grafana write latency is near about 10Ms.

Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: siddharth verma 
[mailto:sidd.verma29.l...@gmail.com<mailto:sidd.verma29.l...@gmail.com>]
Sent: Wednesday, November 23, 2016 2:23 PM
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: Re: Cassandra Config as per server hardware for heavy write

Hi Abhishek,
You could check whether you are throttling on client side queries or on 
cassandra side.
You could also use grafana to monitor the cluster as well.
As you said, you are using 100 threads, it can't be sure whether you are 
throttling cassandra cluster to its max limit.

As Benjamin suggested, you could use cassandra stress tool.

Lastly, if after everything( and you are sure, that cassandra seems slow) the 
TPS comes out to be the numbers as you suggested, you could check you schema, 
many rows in one partition key, read queries, read write load, write queries 
with Batch/LWT, compactions running etc.


For checking ONLY cassandra throughput, you could use cassandra-stress with any 
schema of your choice.

Regards


On Wed, Nov 23, 2016 at 2:07 PM, Vladimir Yudovin 
<vla...@winguzone.com<mailto:vla...@winguzone.com>> wrote:
So do you see speed write saturation at this number of thread? Does doubling to 
200 bring increase?


Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting, Zero 
production

RE: Cassandra Config as per server hardware for heavy write

[Vladimir Yudovin]

session.execute is coming from Session session = cluster.connect(); I guess?



So actually all threads work with the same TCP connection. It's worth to try 
async API with Connection Pool.



Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting






 On Wed, 23 Nov 2016 04:49:18 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 




Hi

 

I am submitting record to Executor service and below is my client config and 
code:

 

cluster = Cluster.builder().addContactPoints(hostAddresses)

   .withRetryPolicy(DefaultRetryPolicy.INSTANCE)

   .withReconnectionPolicy(new 
ConstantReconnectionPolicy(3L))

   .withLoadBalancingPolicy(new TokenAwarePolicy(new 
DCAwareRoundRobinPolicy()))

   .build();

 

   ExecutorService service=Executors.newFixedThreadPool(1000);

for(final AdLog adLog:li){

service.submit(()-{

session.execute(ktest.adImprLogToStatement(adLog.getAdLogType(),adLog.getAdImprLog()));

inte.incrementAndGet();

 });

  }

 

 

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.


 

From: Vladimir Yudovin [mailto:vla...@winguzone.com] 
 Sent: Wednesday, November 23, 2016 3:15 PM
 To: user user@cassandra.apache.org
 Subject: RE: Cassandra Config as per server hardware for heavy write


 

I have a list with 1cr record. I am just iterating on it and executing the 
query. Also, I try with 200 thread


Do you fetch each list item and put it to separate thread to perform CQL query? 
Also how exactly do you connect to Cassandra?


If you use synchronous API so it's better to create connection pool (with 
TokenAwarePolicy each) and then pass each item to separate thread.


 


 


Best regards, Vladimir Yudovin,


Winguzone - Cloud Cassandra Hosting



 


 


 On Wed, 23 Nov 2016 04:23:13 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 



 


Hi Siddharth,

 

For me it seems Cassandra side. Because I have a list with 1cr record. I am 
just iterating on it and executing the query.

Also, I try with 200 thread but still speed doesn’t increase that much as 
expected. On grafana write latency is near about 10Ms.

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

 

From: siddharth verma [mailto:sidd.verma29.l...@gmail.com] 
 Sent: Wednesday, November 23, 2016 2:23 PM
 To:  user@cassandra.apache.org
 Subject: Re: Cassandra Config as per server hardware for heavy write
 

Hi Abhishek,

You could check whether you are throttling on client side queries or on 
cassandra side.


You could also use grafana to monitor the cluster as well.


As you said, you are using 100 threads, it can't be sure whether you are 
throttling cassandra cluster to its max limit.


 


As Benjamin suggested, you could use cassandra stress tool.


 


Lastly, if after everything( and you are sure, that cassandra seems slow) the 
TPS comes out to be the numbers as you suggested, you could check you schema, 
many rows in one partition key, read queries, read write load, write queries 
with Batch/LWT, compactions running etc.


 


 


For checking ONLY cassandra throughput, you could use cassandra-stress with any 
schema of your choice.


 


Regards


 



 

On Wed, Nov 23, 2016 at 2:07 PM, Vladimir Yudovin vla...@winguzone.com 
wrote:

So do you see speed write saturation at this number of thread? Does doubling to 
200 bring increase?


 


 


Best regards, Vladimir Yudovin,


Winguzone - Cloud Cassandra Hosting, Zero production time



 


 


 On Wed, 23 Nov 2016 03:31:32 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 



 


No I am using 100 threads.

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.


 

From: Vladimir Yudovin [mailto:vla...@winguzone.com] 
 Sent: Wednesday, November 23, 2016 2:00 PM
 To: user user@cassandra.apache.org
 Subject: RE: Cassandra Config as per server hardware for heavy write


 

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode.


Is your Java program single threaded?


 


Best regards, Vladimir Yudovin,


Winguzone - Cloud Cassandra Hosting, Zero production time



 


 


 On Wed, 23 Nov 2016 03:09:29 -0500Abhishek Kumar Maheshwari

RE: Cassandra Config as per server hardware for heavy write

[Abhishek Kumar Maheshwari]

Hi

I am submitting record to Executor service and below is my client config and 
code:

cluster = Cluster.builder().addContactPoints(hostAddresses)
   .withRetryPolicy(DefaultRetryPolicy.INSTANCE)
   .withReconnectionPolicy(new 
ConstantReconnectionPolicy(3L))
   .withLoadBalancingPolicy(new TokenAwarePolicy(new 
DCAwareRoundRobinPolicy()))
   .build();

   ExecutorService service=Executors.newFixedThreadPool(1000);
  for(final AdLog adLog:li){
 service.submit(()->{

session.execute(ktest.adImprLogToStatement(adLog.getAdLogType(),adLog.getAdImprLog()));
   inte.incrementAndGet();
 });
  }



Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: Vladimir Yudovin [mailto:vla...@winguzone.com]
Sent: Wednesday, November 23, 2016 3:15 PM
To: user <user@cassandra.apache.org>
Subject: RE: Cassandra Config as per server hardware for heavy write

>I have a list with 1cr record. I am just iterating on it and executing the 
>query. Also, I try with 200 thread
Do you fetch each list item and put it to separate thread to perform CQL query? 
Also how exactly do you connect to Cassandra?
If you use synchronous API so it's better to create connection pool (with 
TokenAwarePolicy each) and then pass each item to separate thread.


Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting


 On Wed, 23 Nov 2016 04:23:13 -0500Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote 

Hi Siddharth,

For me it seems Cassandra side. Because I have a list with 1cr record. I am 
just iterating on it and executing the query.
Also, I try with 200 thread but still speed doesn’t increase that much as 
expected. On grafana write latency is near about 10Ms.

Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: siddharth verma 
[mailto:sidd.verma29.l...@gmail.com<mailto:sidd.verma29.l...@gmail.com>]
Sent: Wednesday, November 23, 2016 2:23 PM
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: Re: Cassandra Config as per server hardware for heavy write

Hi Abhishek,
You could check whether you are throttling on client side queries or on 
cassandra side.
You could also use grafana to monitor the cluster as well.
As you said, you are using 100 threads, it can't be sure whether you are 
throttling cassandra cluster to its max limit.

As Benjamin suggested, you could use cassandra stress tool.

Lastly, if after everything( and you are sure, that cassandra seems slow) the 
TPS comes out to be the numbers as you suggested, you could check you schema, 
many rows in one partition key, read queries, read write load, write queries 
with Batch/LWT, compactions running etc.


For checking ONLY cassandra throughput, you could use cassandra-stress with any 
schema of your choice.

Regards


On Wed, Nov 23, 2016 at 2:07 PM, Vladimir Yudovin 
<vla...@winguzone.com<mailto:vla...@winguzone.com>> wrote:
So do you see speed write saturation at this number of thread? Does doubling to 
200 bring increase?


Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting, Zero 
production time


 On Wed, 23 Nov 2016 03:31:32 -0500Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote 

No I am using 100 threads.

Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: Vladimir Yudovin 
[mailto:vla...@winguzone.com<mailto:vla...@winguzone.com>]
Sent: Wednesday, November 23, 2016 2:00 PM
To: user <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: RE: Cassandra Config as per server hardware for heavy write

>I have 1Cr records in my Java ArrayList and yes I am writing in sync mode.
Is your Java program single threaded?

Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting, Zero 
production time


 On Wed, 23 Nov 2016 03:09:29 -0500Abhishek Kumar Maheshwari 
<abh

RE: Cassandra Config as per server hardware for heavy write

[Vladimir Yudovin]

I have a list with 1cr record. I am just iterating on it and executing the 
query. Also, I try with 200 thread

Do you fetch each list item and put it to separate thread to perform CQL query? 
Also how exactly do you connect to Cassandra?

If you use synchronous API so it's better to create connection pool (with 
TokenAwarePolicy each) and then pass each item to separate thread.





Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting






 On Wed, 23 Nov 2016 04:23:13 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 




Hi Siddharth,

 

For me it seems Cassandra side. Because I have a list with 1cr record. I am 
just iterating on it and executing the query.

Also, I try with 200 thread but still speed doesn’t increase that much as 
expected. On grafana write latency is near about 10Ms.

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

 

From: siddharth verma [mailto:sidd.verma29.l...@gmail.com] 
 Sent: Wednesday, November 23, 2016 2:23 PM
 To: user@cassandra.apache.org
 Subject: Re: Cassandra Config as per server hardware for heavy write
 

Hi Abhishek,

You could check whether you are throttling on client side queries or on 
cassandra side.


You could also use grafana to monitor the cluster as well.


As you said, you are using 100 threads, it can't be sure whether you are 
throttling cassandra cluster to its max limit.


 


As Benjamin suggested, you could use cassandra stress tool.


 


Lastly, if after everything( and you are sure, that cassandra seems slow) the 
TPS comes out to be the numbers as you suggested, you could check you schema, 
many rows in one partition key, read queries, read write load, write queries 
with Batch/LWT, compactions running etc.


 


 


For checking ONLY cassandra throughput, you could use cassandra-stress with any 
schema of your choice.


 


Regards


 



 

On Wed, Nov 23, 2016 at 2:07 PM, Vladimir Yudovin vla...@winguzone.com 
wrote:

So do you see speed write saturation at this number of thread? Does doubling to 
200 bring increase?


 


 


Best regards, Vladimir Yudovin,


Winguzone - Cloud Cassandra Hosting, Zero production time



 


 


 On Wed, 23 Nov 2016 03:31:32 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 



 


No I am using 100 threads.

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.


 

From: Vladimir Yudovin [mailto:vla...@winguzone.com] 
 Sent: Wednesday, November 23, 2016 2:00 PM
 To: user user@cassandra.apache.org
 Subject: RE: Cassandra Config as per server hardware for heavy write


 

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode.


Is your Java program single threaded?


 


Best regards, Vladimir Yudovin,


Winguzone - Cloud Cassandra Hosting, Zero production time



 


 


 On Wed, 23 Nov 2016 03:09:29 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 



 


Hi Benjamin,

 

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode. My 
table is as below:

 

CREATE TABLE XXX_YY_MMS (

date timestamp,

userid text,

time timestamp,

xid text,

addimid text,

advcid bigint,

algo bigint,

alla text,

aud text,

bmid text,

ctyid text,

bid double,

ctxid text,

devipid text,

gmid text,

ip text,

itcid bigint,

iid text,

metid bigint,

osdid text,

paid int,

position text,

pcid bigint,

refurl text,

sec text,

siid bigint,

tmpid bigint,

xforwardedfor text,

PRIMARY KEY (date, userid, time, xid)

) WITH CLUSTERING ORDER BY (userid ASC, time ASC, xid ASC)

AND bloom_filter_fp_chance = 0.01

AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'

AND comment = ''

AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}

AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}

AND dclocal_read_repair_chance = 0.1

AND default_time_to_live = 0

AND gc_grace_seconds = 864000

AND max_index_interval = 2048

AND memtable_flush_period_in_ms = 0

AND min_index_interval = 128

AND read_repair_chance = 0.0

AND speculative_retry = '99.0PERCENTILE';

 

So please let me know what I miss?

 

And for this hardware below config is fine?

 

concurrent_reads: 32

concurrent_writes: 64

concurrent_counter_writes: 

RE: Cassandra Config as per server hardware for heavy write

[Abhishek Kumar Maheshwari]

Hi Siddharth,

For me it seems Cassandra side. Because I have a list with 1cr record. I am 
just iterating on it and executing the query.
Also, I try with 200 thread but still speed doesn’t increase that much as 
expected. On grafana write latency is near about 10Ms.

Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: siddharth verma [mailto:sidd.verma29.l...@gmail.com]
Sent: Wednesday, November 23, 2016 2:23 PM
To: user@cassandra.apache.org
Subject: Re: Cassandra Config as per server hardware for heavy write

Hi Abhishek,
You could check whether you are throttling on client side queries or on 
cassandra side.
You could also use grafana to monitor the cluster as well.
As you said, you are using 100 threads, it can't be sure whether you are 
throttling cassandra cluster to its max limit.

As Benjamin suggested, you could use cassandra stress tool.

Lastly, if after everything( and you are sure, that cassandra seems slow) the 
TPS comes out to be the numbers as you suggested, you could check you schema, 
many rows in one partition key, read queries, read write load, write queries 
with Batch/LWT, compactions running etc.


For checking ONLY cassandra throughput, you could use cassandra-stress with any 
schema of your choice.

Regards


On Wed, Nov 23, 2016 at 2:07 PM, Vladimir Yudovin 
<vla...@winguzone.com<mailto:vla...@winguzone.com>> wrote:
So do you see speed write saturation at this number of thread? Does doubling to 
200 bring increase?


Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting, Zero 
production time


 On Wed, 23 Nov 2016 03:31:32 -0500Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote 

No I am using 100 threads.

Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: Vladimir Yudovin 
[mailto:vla...@winguzone.com<mailto:vla...@winguzone.com>]
Sent: Wednesday, November 23, 2016 2:00 PM
To: user <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: RE: Cassandra Config as per server hardware for heavy write

>I have 1Cr records in my Java ArrayList and yes I am writing in sync mode.
Is your Java program single threaded?

Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting, Zero 
production time


 On Wed, 23 Nov 2016 03:09:29 -0500Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote 

Hi Benjamin,

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode. My 
table is as below:

CREATE TABLE XXX_YY_MMS (
date timestamp,
userid text,
time timestamp,
xid text,
addimid text,
advcid bigint,
algo bigint,
alla text,
aud text,
bmid text,
ctyid text,
bid double,
ctxid text,
devipid text,
gmid text,
ip text,
itcid bigint,
iid text,
metid bigint,
osdid text,
paid int,
position text,
pcid bigint,
refurl text,
sec text,
siid bigint,
tmpid bigint,
xforwardedfor text,
PRIMARY KEY (date, userid, time, xid)
) WITH CLUSTERING ORDER BY (userid ASC, time ASC, xid ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io<http://org.apache.cassandra.io>.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';

So please let me know what I miss?

And for this hardware below config is fine?

concurrent_reads: 32
concurrent_writes: 64
concurrent_counter_writes: 32
compaction_throughput_mb_per_sec: 32
concurrent_compactors: 8

thanks,
Abhishek

From: Benjamin Roth 
[mailto:benjamin.r...@jaumo.com<mailto:benjamin.r...@jaumo.com>]
Sent: Wednesday, November 23, 2016 12:56 PM
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: Re: Cassandra Config as per server hardware for heavy write

This is ridiculously

RE: Cassandra Config as per server hardware for heavy write

[Abhishek Kumar Maheshwari]

Hi Vladimir,

I try the same but it doesn’t increase. also in grafan average write latency is 
near about 10Ms.

Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: Vladimir Yudovin [mailto:vla...@winguzone.com]
Sent: Wednesday, November 23, 2016 2:07 PM
To: user <user@cassandra.apache.org>
Subject: RE: Cassandra Config as per server hardware for heavy write

So do you see speed write saturation at this number of thread? Does doubling to 
200 bring increase?


Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting, Zero 
production time


 On Wed, 23 Nov 2016 03:31:32 -0500Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote 

No I am using 100 threads.

Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: Vladimir Yudovin 
[mailto:vla...@winguzone.com<mailto:vla...@winguzone.com>]
Sent: Wednesday, November 23, 2016 2:00 PM
To: user <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: RE: Cassandra Config as per server hardware for heavy write

>I have 1Cr records in my Java ArrayList and yes I am writing in sync mode.
Is your Java program single threaded?

Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting, Zero 
production time


 On Wed, 23 Nov 2016 03:09:29 -0500Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote 

Hi Benjamin,

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode. My 
table is as below:

CREATE TABLE XXX_YY_MMS (
date timestamp,
userid text,
time timestamp,
xid text,
addimid text,
advcid bigint,
algo bigint,
alla text,
aud text,
bmid text,
ctyid text,
bid double,
ctxid text,
devipid text,
gmid text,
ip text,
itcid bigint,
iid text,
metid bigint,
osdid text,
paid int,
position text,
pcid bigint,
refurl text,
sec text,
siid bigint,
tmpid bigint,
xforwardedfor text,
PRIMARY KEY (date, userid, time, xid)
) WITH CLUSTERING ORDER BY (userid ASC, time ASC, xid ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';

So please let me know what I miss?

And for this hardware below config is fine?

concurrent_reads: 32
concurrent_writes: 64
concurrent_counter_writes: 32
compaction_throughput_mb_per_sec: 32
concurrent_compactors: 8

thanks,
Abhishek

From: Benjamin Roth 
[mailto:benjamin.r...@jaumo.com<mailto:benjamin.r...@jaumo.com>]
Sent: Wednesday, November 23, 2016 12:56 PM
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: Re: Cassandra Config as per server hardware for heavy write

This is ridiculously slow for that hardware setup. Sounds like you benchmark 
with a single thread and / or sync queries or very large writes.
A setup like this should be easily able to handle tens of thousands of writes / 
s

2016-11-23 8:02 GMT+01:00 Jonathan Haddad 
<j...@jonhaddad.com<mailto:j...@jonhaddad.com>>:
How are you benchmarking that?
On Tue, Nov 22, 2016 at 9:16 PM Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote:
Hi,

I have 8 servers in my Cassandra Cluster. Each server has 64 GB ram and 40 
Cores and 8 SSD. Currently I have below config in Cassandra.yaml:

concurrent_reads: 32
concurrent_writes: 64
concurrent_counter_writes: 32
compaction_throughput_mb_per_sec: 32
concurrent_compactors: 8

With this configuration, I can write 1700 Request/Sec per server.

But our desired write performance is 3000-4000 Request/Sec per server. As per 
my Understanding Max value for these parameters can be as below:
concurrent_reads: 32
concurrent_writes: 128(

RE: Cassandra Config as per server hardware for heavy write

[Vladimir Yudovin]

So do you see speed write saturation at this number of thread? Does doubling to 
200 bring increase?





Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting, Zero production time






 On Wed, 23 Nov 2016 03:31:32 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 




No I am using 100 threads.

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.


 

From: Vladimir Yudovin [mailto:vla...@winguzone.com] 
 Sent: Wednesday, November 23, 2016 2:00 PM
 To: user user@cassandra.apache.org
 Subject: RE: Cassandra Config as per server hardware for heavy write


 

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode.


Is your Java program single threaded?


 


Best regards, Vladimir Yudovin,


Winguzone - Cloud Cassandra Hosting, Zero production time



 


 


 On Wed, 23 Nov 2016 03:09:29 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 



 


Hi Benjamin,

 

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode. My 
table is as below:

 

CREATE TABLE XXX_YY_MMS (

date timestamp,

userid text,

time timestamp,

xid text,

addimid text,

advcid bigint,

algo bigint,

alla text,

aud text,

bmid text,

ctyid text,

bid double,

ctxid text,

devipid text,

gmid text,

ip text,

itcid bigint,

iid text,

metid bigint,

osdid text,

paid int,

position text,

pcid bigint,

refurl text,

sec text,

siid bigint,

tmpid bigint,

xforwardedfor text,

PRIMARY KEY (date, userid, time, xid)

) WITH CLUSTERING ORDER BY (userid ASC, time ASC, xid ASC)

AND bloom_filter_fp_chance = 0.01

AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'

AND comment = ''

AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}

AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}

AND dclocal_read_repair_chance = 0.1

AND default_time_to_live = 0

AND gc_grace_seconds = 864000

AND max_index_interval = 2048

AND memtable_flush_period_in_ms = 0

AND min_index_interval = 128

AND read_repair_chance = 0.0

AND speculative_retry = '99.0PERCENTILE';

 

So please let me know what I miss?

 

And for this hardware below config is fine?

 

concurrent_reads: 32

concurrent_writes: 64

concurrent_counter_writes: 32

compaction_throughput_mb_per_sec: 32

concurrent_compactors: 8

 

thanks,

Abhishek

 

From: Benjamin Roth [mailto:benjamin.r...@jaumo.com] 
 Sent: Wednesday, November 23, 2016 12:56 PM
 To:  user@cassandra.apache.org
 Subject: Re: Cassandra Config as per server hardware for heavy write
 

This is ridiculously slow for that hardware setup. Sounds like you benchmark 
with a single thread and / or sync queries or very large writes.

A setup like this should be easily able to handle tens of thousands of writes / 
s



 

2016-11-23 8:02 GMT+01:00 Jonathan Haddad j...@jonhaddad.com:

How are you benchmarking that?

On Tue, Nov 22, 2016 at 9:16 PM Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote:


Hi,

 

I have 8 servers in my Cassandra Cluster. Each server has 64 GB ram and 40 
Cores and 8 SSD. Currently I have below config in Cassandra.yaml:

 

concurrent_reads: 32

concurrent_writes: 64

concurrent_counter_writes: 32

compaction_throughput_mb_per_sec: 32

concurrent_compactors: 8

 

With this configuration, I can write 1700 Request/Sec per server.

 

But our desired write performance is 3000-4000 Request/Sec per server. As per 
my Understanding Max value for these parameters can be as below:

concurrent_reads: 32

concurrent_writes: 128(8*16 Corew)

concurrent_counter_writes: 32

compaction_throughput_mb_per_sec: 128

concurrent_compactors: 8 or 16 (as I have 8 SSD and 16 core reserve for this)

 

Please let me know this is fine or I need to tune some other parameters for 
speedup write.

 

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

 


Education gets Exciting with IIM Kozhikode Executive Post Graduate Programme in 
Management - 2 years (AMBA accredited with full benefits of IIMK Alumni 
status). Brought to you by IIMK in association with TSW, an Executive Education 
initiative from The Times of India Group. Learn more: www.timestsw.com






 

 


 

 


--

 

Benjamin Roth


Prokurist


 


Jaumo GmbH

RE: Cassandra Config as per server hardware for heavy write

[Abhishek Kumar Maheshwari]

No I am using 100 threads.

Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

From: Vladimir Yudovin [mailto:vla...@winguzone.com]
Sent: Wednesday, November 23, 2016 2:00 PM
To: user <user@cassandra.apache.org>
Subject: RE: Cassandra Config as per server hardware for heavy write

>I have 1Cr records in my Java ArrayList and yes I am writing in sync mode.
Is your Java program single threaded?

Best regards, Vladimir Yudovin,
Winguzone<https://winguzone.com?from=list> - Cloud Cassandra Hosting, Zero 
production time


 On Wed, 23 Nov 2016 03:09:29 -0500Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote 

Hi Benjamin,

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode. My 
table is as below:

CREATE TABLE XXX_YY_MMS (
date timestamp,
userid text,
time timestamp,
xid text,
addimid text,
advcid bigint,
algo bigint,
alla text,
aud text,
bmid text,
ctyid text,
bid double,
ctxid text,
devipid text,
gmid text,
ip text,
itcid bigint,
iid text,
metid bigint,
osdid text,
paid int,
position text,
pcid bigint,
refurl text,
sec text,
siid bigint,
tmpid bigint,
xforwardedfor text,
PRIMARY KEY (date, userid, time, xid)
) WITH CLUSTERING ORDER BY (userid ASC, time ASC, xid ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';

So please let me know what I miss?

And for this hardware below config is fine?

concurrent_reads: 32
concurrent_writes: 64
concurrent_counter_writes: 32
compaction_throughput_mb_per_sec: 32
concurrent_compactors: 8

thanks,
Abhishek

From: Benjamin Roth 
[mailto:benjamin.r...@jaumo.com<mailto:benjamin.r...@jaumo.com>]
Sent: Wednesday, November 23, 2016 12:56 PM
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: Re: Cassandra Config as per server hardware for heavy write

This is ridiculously slow for that hardware setup. Sounds like you benchmark 
with a single thread and / or sync queries or very large writes.
A setup like this should be easily able to handle tens of thousands of writes / 
s

2016-11-23 8:02 GMT+01:00 Jonathan Haddad 
<j...@jonhaddad.com<mailto:j...@jonhaddad.com>>:
How are you benchmarking that?
On Tue, Nov 22, 2016 at 9:16 PM Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote:
Hi,

I have 8 servers in my Cassandra Cluster. Each server has 64 GB ram and 40 
Cores and 8 SSD. Currently I have below config in Cassandra.yaml:

concurrent_reads: 32
concurrent_writes: 64
concurrent_counter_writes: 32
compaction_throughput_mb_per_sec: 32
concurrent_compactors: 8

With this configuration, I can write 1700 Request/Sec per server.

But our desired write performance is 3000-4000 Request/Sec per server. As per 
my Understanding Max value for these parameters can be as below:
concurrent_reads: 32
concurrent_writes: 128(8*16 Corew)
concurrent_counter_writes: 32
compaction_throughput_mb_per_sec: 128
concurrent_compactors: 8 or 16 (as I have 8 SSD and 16 core reserve for this)

Please let me know this is fine or I need to tune some other parameters for 
speedup write.


Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

Education gets Exciting with IIM Kozhikode Executive Post Graduate Programme in 
Management - 2 years (AMBA accredited with full benefits of IIMK Alumni 
status). Brought to you by IIMK in association with TSW, an Executive Education 
initiative from The Times of India Group. Learn more: 
www.timestsw.com<http://www.timestsw.com>





--

Benjamin Roth
Prokurist

Jaumo GmbH · www.jaumo.com<http://www.jaumo.com>
Wehrstraße 46 · 73035 Göppingen · Germany
Phone +49 7161 304880-6 · Fax +49 7161 304880-1
AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

RE: Cassandra Config as per server hardware for heavy write

[Vladimir Yudovin]

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode.

Is your Java program single threaded?



Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting, Zero production time






 On Wed, 23 Nov 2016 03:09:29 -0500Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote 




Hi Benjamin,

 

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode. My 
table is as below:

 

CREATE TABLE XXX_YY_MMS (

date timestamp,

userid text,

time timestamp,

xid text,

addimid text,

advcid bigint,

algo bigint,

alla text,

aud text,

bmid text,

ctyid text,

bid double,

ctxid text,

devipid text,

gmid text,

ip text,

itcid bigint,

iid text,

metid bigint,

osdid text,

paid int,

position text,

pcid bigint,

refurl text,

sec text,

siid bigint,

tmpid bigint,

xforwardedfor text,

PRIMARY KEY (date, userid, time, xid)

) WITH CLUSTERING ORDER BY (userid ASC, time ASC, xid ASC)

AND bloom_filter_fp_chance = 0.01

AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'

AND comment = ''

AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}

AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}

AND dclocal_read_repair_chance = 0.1

AND default_time_to_live = 0

AND gc_grace_seconds = 864000

AND max_index_interval = 2048

AND memtable_flush_period_in_ms = 0

AND min_index_interval = 128

AND read_repair_chance = 0.0

AND speculative_retry = '99.0PERCENTILE';

 

So please let me know what I miss?

 

And for this hardware below config is fine?

 

concurrent_reads: 32

concurrent_writes: 64

concurrent_counter_writes: 32

compaction_throughput_mb_per_sec: 32

concurrent_compactors: 8

 

thanks,

Abhishek

 

From: Benjamin Roth [mailto:benjamin.r...@jaumo.com] 
 Sent: Wednesday, November 23, 2016 12:56 PM
 To: user@cassandra.apache.org
 Subject: Re: Cassandra Config as per server hardware for heavy write
 

This is ridiculously slow for that hardware setup. Sounds like you benchmark 
with a single thread and / or sync queries or very large writes.

A setup like this should be easily able to handle tens of thousands of writes / 
s



 

2016-11-23 8:02 GMT+01:00 Jonathan Haddad j...@jonhaddad.com:

How are you benchmarking that?

On Tue, Nov 22, 2016 at 9:16 PM Abhishek Kumar Maheshwari 
abhishek.maheshw...@timesinternet.in wrote:


Hi,

 

I have 8 servers in my Cassandra Cluster. Each server has 64 GB ram and 40 
Cores and 8 SSD. Currently I have below config in Cassandra.yaml:

 

concurrent_reads: 32

concurrent_writes: 64

concurrent_counter_writes: 32

compaction_throughput_mb_per_sec: 32

concurrent_compactors: 8

 

With this configuration, I can write 1700 Request/Sec per server.

 

But our desired write performance is 3000-4000 Request/Sec per server. As per 
my Understanding Max value for these parameters can be as below:

concurrent_reads: 32

concurrent_writes: 128(8*16 Corew)

concurrent_counter_writes: 32

compaction_throughput_mb_per_sec: 128

concurrent_compactors: 8 or 16 (as I have 8 SSD and 16 core reserve for this)

 

Please let me know this is fine or I need to tune some other parameters for 
speedup write.

 

 

Thanks  Regards,
 Abhishek Kumar Maheshwari
 +91- 805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

 


Education gets Exciting with IIM Kozhikode Executive Post Graduate Programme in 
Management - 2 years (AMBA accredited with full benefits of IIMK Alumni 
status). Brought to you by IIMK in association with TSW, an Executive Education 
initiative from The Times of India Group. Learn more:  www.timestsw.com












 


--



Benjamin Roth

Prokurist



Jaumo GmbH · www.jaumo.com

Wehrstraße 46 · 73035 Göppingen · Germany

Phone +49 7161 304880-6 · Fax +49 7161 304880-1

AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: Cassandra Config as per server hardware for heavy write

[Benjamin Roth]

There is cassandra-stress to benchmark your cluster.

See docs here:
https://docs.datastax.com/en/cassandra/3.x/cassandra/tools/toolsCStress.html?hl=stress

2016-11-23 9:09 GMT+01:00 Abhishek Kumar Maheshwari <
abhishek.maheshw...@timesinternet.in>:

> Hi Benjamin,
>
>
>
> I have 1Cr records in my Java ArrayList and yes I am writing in sync mode.
> My table is as below:
>
>
>
> CREATE TABLE XXX_YY_MMS (
>
> date timestamp,
>
> userid text,
>
> time timestamp,
>
> xid text,
>
> addimid text,
>
> advcid bigint,
>
> algo bigint,
>
> alla text,
>
> aud text,
>
> bmid text,
>
> ctyid text,
>
> bid double,
>
> ctxid text,
>
> devipid text,
>
> gmid text,
>
> ip text,
>
> itcid bigint,
>
> iid text,
>
> metid bigint,
>
> osdid text,
>
> paid int,
>
> position text,
>
> pcid bigint,
>
> refurl text,
>
> sec text,
>
> siid bigint,
>
> tmpid bigint,
>
> xforwardedfor text,
>
> PRIMARY KEY (date, userid, time, xid)
>
> ) WITH CLUSTERING ORDER BY (userid ASC, time ASC, xid ASC)
>
> AND bloom_filter_fp_chance = 0.01
>
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
>
> AND comment = ''
>
> AND compaction = {'class': 'org.apache.cassandra.db.compaction.
> SizeTieredCompactionStrategy'}
>
> AND compression = {'sstable_compression': 'org.apache.cassandra.io.
> compress.LZ4Compressor'}
>
> AND dclocal_read_repair_chance = 0.1
>
> AND default_time_to_live = 0
>
> AND gc_grace_seconds = 864000
>
> AND max_index_interval = 2048
>
> AND memtable_flush_period_in_ms = 0
>
> AND min_index_interval = 128
>
> AND read_repair_chance = 0.0
>
> AND speculative_retry = '99.0PERCENTILE';
>
>
>
> So please let me know what I miss?
>
>
>
> And for this hardware below config is fine?
>
>
>
> concurrent_reads: 32
>
> concurrent_writes: 64
>
> concurrent_counter_writes: 32
>
> compaction_throughput_mb_per_sec: 32
>
> concurrent_compactors: 8
>
>
>
> thanks,
>
> Abhishek
>
>
>
> *From:* Benjamin Roth [mailto:benjamin.r...@jaumo.com]
> *Sent:* Wednesday, November 23, 2016 12:56 PM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Cassandra Config as per server hardware for heavy write
>
>
>
> This is ridiculously slow for that hardware setup. Sounds like you
> benchmark with a single thread and / or sync queries or very large writes.
>
> A setup like this should be easily able to handle tens of thousands of
> writes / s
>
>
>
> 2016-11-23 8:02 GMT+01:00 Jonathan Haddad <j...@jonhaddad.com>:
>
> How are you benchmarking that?
>
> On Tue, Nov 22, 2016 at 9:16 PM Abhishek Kumar Maheshwari <
> abhishek.maheshw...@timesinternet.in> wrote:
>
> Hi,
>
>
>
> I have 8 servers in my Cassandra Cluster. Each server has 64 GB ram and 40
> Cores and 8 SSD. Currently I have below config in Cassandra.yaml:
>
>
>
> concurrent_reads: 32
>
> concurrent_writes: 64
>
> concurrent_counter_writes: 32
>
> compaction_throughput_mb_per_sec: 32
>
> concurrent_compactors: 8
>
>
>
> With this configuration, I can write 1700 Request/Sec per server.
>
>
>
> But our desired write performance is 3000-4000 Request/Sec per server. As
> per my Understanding Max value for these parameters can be as below:
>
> concurrent_reads: 32
>
> concurrent_writes: 128(8*16 Corew)
>
> concurrent_counter_writes: 32
>
> compaction_throughput_mb_per_sec: 128
>
> concurrent_compactors: 8 or 16 (as I have 8 SSD and 16 core reserve for
> this)
>
>
>
> Please let me know this is fine or I need to tune some other parameters
> for speedup write.
>
>
>
>
>
> *Thanks & Regards,*
> *Abhishek Kumar Maheshwari*
> *+91- 805591 <%2B91-%C2%A0805591> (Mobile)*
>
> Times Internet Ltd. | A Times of India Group Company
>
> FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
>
> *P** Please do not print this email unless it is absolutely necessary.
> Spread environmental awareness.*
>
>
>
> Education gets Exciting with IIM Kozhikode Executive Post Graduate
> Programme in Management - 2 years (AMBA accredited with full benefits of
> IIMK Alumni status). Brought to you by IIMK in association with TSW, an
> Executive Education initiative from The Times of India Group. Learn more:
> www.timestsw.com
>
>
>
>
>
> --
>
> Benjamin Roth
> Prokurist
>
> Jaumo GmbH · www.jaumo.com
> Wehrstraße 46 · 73035 Göppingen · Germany
> Phone +49 7161 304880-6 · Fax +49 7161 304880-1
> AG Ulm · HRB 731058 · Managing Director: Jens Kammerer
>



-- 
Benjamin Roth
Prokurist

Jaumo GmbH · www.jaumo.com
Wehrstraße 46 · 73035 Göppingen · Germany
Phone +49 7161 304880-6 · Fax +49 7161 304880-1
AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

RE: Cassandra Config as per server hardware for heavy write

[Abhishek Kumar Maheshwari]

Hi Benjamin,

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode. My 
table is as below:

CREATE TABLE XXX_YY_MMS (
date timestamp,
userid text,
time timestamp,
xid text,
addimid text,
advcid bigint,
algo bigint,
alla text,
aud text,
bmid text,
ctyid text,
bid double,
ctxid text,
devipid text,
gmid text,
ip text,
itcid bigint,
iid text,
metid bigint,
osdid text,
paid int,
position text,
pcid bigint,
refurl text,
sec text,
siid bigint,
tmpid bigint,
xforwardedfor text,
PRIMARY KEY (date, userid, time, xid)
) WITH CLUSTERING ORDER BY (userid ASC, time ASC, xid ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';

So please let me know what I miss?

And for this hardware below config is fine?

concurrent_reads: 32
concurrent_writes: 64
concurrent_counter_writes: 32
compaction_throughput_mb_per_sec: 32
concurrent_compactors: 8

thanks,
Abhishek

From: Benjamin Roth [mailto:benjamin.r...@jaumo.com]
Sent: Wednesday, November 23, 2016 12:56 PM
To: user@cassandra.apache.org
Subject: Re: Cassandra Config as per server hardware for heavy write

This is ridiculously slow for that hardware setup. Sounds like you benchmark 
with a single thread and / or sync queries or very large writes.
A setup like this should be easily able to handle tens of thousands of writes / 
s

2016-11-23 8:02 GMT+01:00 Jonathan Haddad 
<j...@jonhaddad.com<mailto:j...@jonhaddad.com>>:
How are you benchmarking that?
On Tue, Nov 22, 2016 at 9:16 PM Abhishek Kumar Maheshwari 
<abhishek.maheshw...@timesinternet.in<mailto:abhishek.maheshw...@timesinternet.in>>
 wrote:
Hi,

I have 8 servers in my Cassandra Cluster. Each server has 64 GB ram and 40 
Cores and 8 SSD. Currently I have below config in Cassandra.yaml:

concurrent_reads: 32
concurrent_writes: 64
concurrent_counter_writes: 32
compaction_throughput_mb_per_sec: 32
concurrent_compactors: 8

With this configuration, I can write 1700 Request/Sec per server.

But our desired write performance is 3000-4000 Request/Sec per server. As per 
my Understanding Max value for these parameters can be as below:
concurrent_reads: 32
concurrent_writes: 128(8*16 Corew)
concurrent_counter_writes: 32
compaction_throughput_mb_per_sec: 128
concurrent_compactors: 8 or 16 (as I have 8 SSD and 16 core reserve for this)

Please let me know this is fine or I need to tune some other parameters for 
speedup write.


Thanks & Regards,
Abhishek Kumar Maheshwari
+91- 805591<tel:%2B91-%C2%A0805591> (Mobile)
Times Internet Ltd. | A Times of India Group Company
FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
P Please do not print this email unless it is absolutely necessary. Spread 
environmental awareness.

Education gets Exciting with IIM Kozhikode Executive Post Graduate Programme in 
Management - 2 years (AMBA accredited with full benefits of IIMK Alumni 
status). Brought to you by IIMK in association with TSW, an Executive Education 
initiative from The Times of India Group. Learn more: 
www.timestsw.com<http://www.timestsw.com>



--
Benjamin Roth
Prokurist

Jaumo GmbH · www.jaumo.com<http://www.jaumo.com>
Wehrstraße 46 · 73035 Göppingen · Germany
Phone +49 7161 304880-6 · Fax +49 7161 304880-1
AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: Cassandra Config as per server hardware for heavy write

[Benjamin Roth]

This is ridiculously slow for that hardware setup. Sounds like you
benchmark with a single thread and / or sync queries or very large writes.
A setup like this should be easily able to handle tens of thousands of
writes / s

2016-11-23 8:02 GMT+01:00 Jonathan Haddad :

> How are you benchmarking that?
> On Tue, Nov 22, 2016 at 9:16 PM Abhishek Kumar Maheshwari <
> abhishek.maheshw...@timesinternet.in> wrote:
>
>> Hi,
>>
>>
>>
>> I have 8 servers in my Cassandra Cluster. Each server has 64 GB ram and
>> 40 Cores and 8 SSD. Currently I have below config in Cassandra.yaml:
>>
>>
>>
>> concurrent_reads: 32
>>
>> concurrent_writes: 64
>>
>> concurrent_counter_writes: 32
>>
>> compaction_throughput_mb_per_sec: 32
>>
>> concurrent_compactors: 8
>>
>>
>>
>> With this configuration, I can write 1700 Request/Sec per server.
>>
>>
>>
>> But our desired write performance is 3000-4000 Request/Sec per server. As
>> per my Understanding Max value for these parameters can be as below:
>>
>> concurrent_reads: 32
>>
>> concurrent_writes: 128(8*16 Corew)
>>
>> concurrent_counter_writes: 32
>>
>> compaction_throughput_mb_per_sec: 128
>>
>> concurrent_compactors: 8 or 16 (as I have 8 SSD and 16 core reserve for
>> this)
>>
>>
>>
>> Please let me know this is fine or I need to tune some other parameters
>> for speedup write.
>>
>>
>>
>>
>>
>> *Thanks & Regards,*
>> *Abhishek Kumar Maheshwari*
>> *+91- 805591 <%2B91-%C2%A0805591> (Mobile)*
>>
>> Times Internet Ltd. | A Times of India Group Company
>>
>> FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
>>
>> *P** Please do not print this email unless it is absolutely necessary.
>> Spread environmental awareness.*
>>
>>
>> Education gets Exciting with IIM Kozhikode Executive Post Graduate
>> Programme in Management - 2 years (AMBA accredited with full benefits of
>> IIMK Alumni status). Brought to you by IIMK in association with TSW, an
>> Executive Education initiative from The Times of India Group. Learn more:
>> www.timestsw.com
>>
>


-- 
Benjamin Roth
Prokurist

Jaumo GmbH · www.jaumo.com
Wehrstraße 46 · 73035 Göppingen · Germany
Phone +49 7161 304880-6 · Fax +49 7161 304880-1
AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: Cassandra Config as per server hardware for heavy write

[Jonathan Haddad]

How are you benchmarking that?
On Tue, Nov 22, 2016 at 9:16 PM Abhishek Kumar Maheshwari <
abhishek.maheshw...@timesinternet.in> wrote:

> Hi,
>
>
>
> I have 8 servers in my Cassandra Cluster. Each server has 64 GB ram and 40
> Cores and 8 SSD. Currently I have below config in Cassandra.yaml:
>
>
>
> concurrent_reads: 32
>
> concurrent_writes: 64
>
> concurrent_counter_writes: 32
>
> compaction_throughput_mb_per_sec: 32
>
> concurrent_compactors: 8
>
>
>
> With this configuration, I can write 1700 Request/Sec per server.
>
>
>
> But our desired write performance is 3000-4000 Request/Sec per server. As
> per my Understanding Max value for these parameters can be as below:
>
> concurrent_reads: 32
>
> concurrent_writes: 128(8*16 Corew)
>
> concurrent_counter_writes: 32
>
> compaction_throughput_mb_per_sec: 128
>
> concurrent_compactors: 8 or 16 (as I have 8 SSD and 16 core reserve for
> this)
>
>
>
> Please let me know this is fine or I need to tune some other parameters
> for speedup write.
>
>
>
>
>
> *Thanks & Regards,*
> *Abhishek Kumar Maheshwari*
> *+91- 805591 (Mobile)*
>
> Times Internet Ltd. | A Times of India Group Company
>
> FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA
>
> *P** Please do not print this email unless it is absolutely necessary.
> Spread environmental awareness.*
>
>
> Education gets Exciting with IIM Kozhikode Executive Post Graduate
> Programme in Management - 2 years (AMBA accredited with full benefits of
> IIMK Alumni status). Brought to you by IIMK in association with TSW, an
> Executive Education initiative from The Times of India Group. Learn more:
> www.timestsw.com
>