[
https://issues.apache.org/jira/browse/CASSANDRA-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15216190#comment-15216190
]
Thom Valley commented on CASSANDRA-7779:
----------------------------------------
This grows to an even bigger scale when you move up to 5 DCs.
SSTABLELOADER is very inefficient when trying to load multiple DCs. Most
global multi-DC implementations are bandwidth sensitive and streaming RF copies
of the data to each target DC is very expensive / can be impacted by bandwidth
limitations.
Being able to LOAD data to a single DC and have Cassandra do the replication to
the additional DCs would be much more efficient, as Cassandra does a great job
of limiting resource consumption.
I realize that's not really SSTABLELOADER as it exists today, but didn't want
to file yet another ticket for something so closely related.
> Add option to sstableloader to only stream to the local dc
> ----------------------------------------------------------
>
> Key: CASSANDRA-7779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7779
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Reporter: Nick Bailey
> Fix For: 2.1.x
>
>
> This is meant to be a potential workaround for CASSANDRA-4756. Due to that
> ticket, trying to load a cluster wide snapshot via sstableloader will
> potentially stream an enormous amount of data. In a 3 datacenter cluster with
> rf=3 in each datacenter, 81 copies of the data would be streamed. Once we
> have per range sstables we can optimize sstableloader to merge data and only
> stream one copy, but until then we need a workaround. By only streaming to
> the local datacenter we can load the data locally in each datacenter and only
> have 9 copies of the data rather than 81.
> This could potentially be achieved by the option to ignore certain nodes that
> already exists in sstableloader, but in the case of vnodes and topology
> changes in the cluster, this could require specifying every node in the
> cluster as 'ignored' on the command line which could be problematic. This is
> just a shortcut to avoid that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
