[
https://issues.apache.org/jira/browse/CASSANDRA-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512597#comment-14512597
]
Benedict commented on CASSANDRA-8789:
-------------------------------------
FTR, my current perception of this is:
* it does look to me like the increased throughput of the new code is a
plausible cause of server degradation in these localhost tests, since we know
that the server has no extra shedding logic in place beyond the normal timeout.
** improved shedding should be addressed separately, e.g. CASSANDRA-8518
* that doesn't mean head of line blocking isn't a real concern, especially for
low bandwidth links
** it does seem likely already an issue in 2.0/2.1 to some greater or lesser
degree given the existing combination of gossip with read response data
** however this does change the exposure profile, and especially for smart
routed clients it might exacerbate this problem in certain cases
** i don't think the exposure profile is sufficiently different to consider
this a regression or to revert the other positive improvements delivered by
this change
* I do think we can quite easily manage this by opening a new connection,
managed by netty or raw NIO, over which we communicate only gossip messages (or
other low frequency, high urgency messages)
** this would in the typical case mean we are using no more connections than
2.1 (though with large mutations/responses we may end up using 50% more
connections), but:
*** these connections would not have significant threading impacts
*** nor would they have any impact on the improved throughput delivered by
coalescing
* CASSANDRA-9237 is IMO a good place to continue this discussion
> OutboundTcpConnectionPool should route messages to sockets by size not type
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-8789
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8789
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Ariel Weisberg
> Assignee: Ariel Weisberg
> Fix For: 3.0
>
> Attachments: 8789.diff
>
>
> I was looking at this trying to understand what messages flow over which
> connection.
> For reads the request goes out over the command connection and the response
> comes back over the ack connection.
> For writes the request goes out over the command connection and the response
> comes back over the command connection.
> Reads get a dedicated socket for responses. Mutation commands and responses
> both travel over the same socket along with read requests.
> Sockets are used uni-directional so there are actually four sockets in play
> and four threads at each node (2 inbounded, 2 outbound).
> CASSANDRA-488 doesn't leave a record of what the impact of this change was.
> If someone remembers what situations were made better it would be good to
> know.
> I am not clear on when/how this is helpful. The consumer side shouldn't be
> blocking so the only head of line blocking issue is the time it takes to
> transfer data over the wire.
> If message size is the cause of blocking issues then the current design mixes
> small messages and large messages on the same connection retaining the head
> of line blocking.
> Read requests share the same connection as write requests (which are large),
> and write acknowledgments (which are small) share the same connections as
> write requests. The only winner is read acknowledgements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
