[
https://issues.apache.org/jira/browse/CASSANDRA-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155592#comment-13155592
]
Peter Schuller commented on CASSANDRA-3483:
-------------------------------------------
We ended up going for the simpler rebuild patch as Chris hinted at. I'll quote
myself:
{quote}
I've been looking at this some more.
Here's the proposal so far:
We introduce a GHOST state, in which a node receives neither reads nor writes.
This allows us to bring in a group of nodes in the ring without suffering any
ill effects. It is completely invisible to reads and writes, and will never
count towards e.g. consistency level.
Once all ghosts are ready, we can start bootstrapping, taking ghost nodes into
accounts for purposes of determining which range we are responsible for, but
streaming only from non-ghost nodes. This accomplishes the goal of not
transferring more data than necessary.
In order to avoid a bootstrapping node from taking writes for more than it's
eventual share, we'd have to make the write endpoints be aware of ghost nodes.
This is dosable, but not critical since we're bisecting a range that was
previously handled by a single node anyway so the traffic would be managable.
It would just be cleaner to not have to cleanup afterwards.
Once we transition from bootstrapping to being "up", we have a bigger problem
however. If the read paths and write paths are only aware of non-ghost nodes,
the read/write paths would think that these nodes had more ownership than they
really do.
So we must really be taking into account the other nodes in the read/write path
as well - but only when determining ownership of a completely bootstrapped node
that was previously a ghost. That means we must distinguish between a "normally
up" node and one that's been bootstraped from ghost state (call it "ghost
strapped").
This suddenly gets complex. The process of group bootstrap would then be:
Add a bunch of nodes in GHOST state
Bootstrap all of them, each of them going into GHOSTSTRAPPED state
Once all are GHOSTSTRAPPED, we can safely transntion from GHOSTSTRAPPED to
normal/up
Is there a simpler solution?
{quote}
After some additional discussion we felt this was adding to much complexity and
potential edge cases/bugs that it became more cost-effective to just go with
the simple rebuild for our immediate needs, hoping to address the problem of
adding lots of nodes to a DC separately in some other way.
A patch is forthcoming soon.
> Support bringing up a new datacenter to existing cluster without repair
> -----------------------------------------------------------------------
>
> Key: CASSANDRA-3483
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3483
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 1.0.2
> Reporter: Chris Goffinet
> Assignee: Peter Schuller
>
> Was talking to Brandon in irc, and we ran into a case where we want to bring
> up a new DC to an existing cluster. He suggested from jbellis the way to do
> it currently was set strategy options of dc2:0, then add the nodes. After the
> nodes are up, change the RF of dc2, and run repair.
> I'd like to avoid a repair as it runs AES and is a bit more intense than how
> bootstrap works currently by just streaming ranges from the SSTables. Would
> it be possible to improve this functionality (adding a new DC to existing
> cluster) than the proposed method? We'd be happy to do a patch if we got some
> input on the best way to go about it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
