[ 
https://issues.apache.org/jira/browse/CASSANDRA-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14268221#comment-14268221
 ] 

Ariel Weisberg commented on CASSANDRA-8457:
-------------------------------------------

I wanted to account for the impact of coalescing at low concurrency. Low 
concurrency is not a recipe for great performance, but it is part of the out of 
the box experience and people do compare different databases at low concurrency.

In GCE coalescing provided a 12% increase in throughput in this specific 
message heavy high concurrency workload. The penalty is that at low concurrency 
there is an immediate loss of performance with any coalescing and a large 
window has a greater impact at low concurrency so there is tension there. The 
larger the window the better the performance bump.

Testing with 3 client threads each running on a dedicated client instance (3 
threads total). This is in GCE.

With TCP no delay on and coalescing
||Coalesce window microseconds|Throughput||
|0| 2191|
|6| 1910|
|12| 1873|
|25| 1867|
|50| 1779|
|100| 1667|
|150| 1566|
|200| 1491|

I also tried disabling coalescing when using HSHA and it didn't seem to make a 
difference. Surprising considering the impact of 25 microseconds of coalescing 
intra-cluster.

I also experimented with some other things. Binding interrupts cores 0 and 8 
and task setting C* off of those cores. I didn't see a big impact.

I did see a small positive impact using 3 clients 8 servers which means the 
measurements with 2 clients might be a little suspect. With 3 clients and 200 
microseconds of coalescing it peaked at 165k in GCE.

I also found out that banned CPUs in irqbalance is broken and has no effect and 
this has been the case for some time.

Right scale does not offer instances with enhanced networking. To find out 
whether coalescing provides real benefits in EC2/Xen or milder GCE like 
benefits I will have to get my hands on some.


> nio MessagingService
> --------------------
>
>                 Key: CASSANDRA-8457
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8457
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Ariel Weisberg
>              Labels: performance
>             Fix For: 3.0
>
>
> Thread-per-peer (actually two each incoming and outbound) is a big 
> contributor to context switching, especially for larger clusters.  Let's look 
> at switching to nio, possibly via Netty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to