[
https://issues.apache.org/jira/browse/CASSANDRA-13993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357555#comment-16357555
]
Jason Brown commented on CASSANDRA-13993:
-----------------------------------------
Back in the mists of time, in cassandra 1.2 we had two comments in the Verbs
enum:
- a message about [backward
compatibility|https://github.com/apache/cassandra/blob/cassandra-1.2/src/java/org/apache/cassandra/net/MessagingService.java#L119],
which appeared before the {{UNUSED_}} verbs
{code:java}
// use as padding for backwards compatability where a previous version needs
to validate a verb from the future.
{code}
- a message about [adding to new
verbs|https://github.com/apache/cassandra/blob/cassandra-1.2/src/java/org/apache/cassandra/net/MessagingService.java#L124]
the end of the list (after the UNUSED verbs)
{code:java}
// remember to add new verbs at the end, since we serialize by ordinal
{code}
The former message assumes we can receive some limited number of messages with
verb ids that are unknown, and not blow up trying to deserialize the message.
In 2.0, both of those comments were moved:
- the backward compatibility comment is [now before the newly introduced paxos
verbs|https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/net/MessagingService.java#L125]
- the new verbs comment is [*before* the UNUSED
verbs|https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/net/MessagingService.java#L130]
I think this is where things become confusing. But read on ...
The situation stayed the same until 3.0, where we deleted the backward
compatibility comment, but kept the message about adding new verbs in the same
place. This is more or less what we have in trunk. Hence, looking at trunk now,
it's not clear if the UNUSED verbs are for future proofing the deserialization
or are some sort of external party-specific messages.
Further, in this current scheme it's not guaranteed for someone to create their
custom verb and have it be safe across versions and upgrades - at least not
until CASSANDRA-13283 (committed for 4.0).
It seems that the original intent of the UNUSED verbs was to allow "verbs from
the future" to be "validated"; that is, [not throw an
ArrayIndexOutOfBoundsException|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/net/MessageIn.java#L87]
when a node sees a message with a verb id it's doen't know about (assuming
that verb ids matches one of the UNUSED verb ids. That message would
ultimiately [be thrown away in
{{MessageDeliveryTask}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/net/MessageDeliveryTask.java#L58]
as we would have no {{IVerbHandler}} for the unused verb.
Further, if we assume the UNUSED verb to be future proofing, then new verbs
should, in fact, be added *before* the UNUSED verbs.
As the ability to add new, custom verbs and be future proof from new
conflicting verbs (assuming all verbs got their id from the enum's ordinal)
didn't arrive until CASSANDRA-13283 (basically 4.0), I think it's reasonable to
assume that nobody is currently running with custom verbs (unless they have
backported CASSANDRA-13283). Thus, I think it should be safe to add new verbs
to 4.0 before the UNUSED verbs as long as the new verb ids fall into the UNUSED
verb ids that 3.0 and 3.11 have declared. I believe this is what we have done
along.
wdyt? [~aweisberg] [~JoshuaMcKenzie]
> Add optional startup delay to wait until peers are ready
> --------------------------------------------------------
>
> Key: CASSANDRA-13993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13993
> Project: Cassandra
> Issue Type: Improvement
> Components: Lifecycle
> Reporter: Jason Brown
> Assignee: Jason Brown
> Priority: Minor
> Fix For: 4.x
>
>
> When bouncing a node in a large cluster, is can take a while to recognize the
> rest of the cluster as available. This is especially true if using TLS on
> internode messaging connections. The bouncing node (and any clients connected
> to it) may see a series of Unavailable or Timeout exceptions until the node
> is 'warmed up' as connecting to the rest of the cluster is asynchronous from
> the rest of the startup process.
> There are two aspects that drive a node's ability to successfully communicate
> with a peer after a bounce:
> - marking the peer as 'alive' (state that is held in gossip). This affects
> the unavailable exceptions
> - having both open outbound and inbound connections open and ready to each
> peer. This affects timeouts.
> Details of each of these mechanisms are described in the comments below.
> This ticket proposes adding a mechanism, optional and configurable, to delay
> opening the client native protocol port until some percentage of the peers in
> the cluster is marked alive and connected to/from. Thus while we potentially
> slow down startup (delay opening the client port), we alleviate the chance
> that queries made by clients don't hit transient unavailable/timeout
> exceptions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
