[
https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811914#comment-13811914
]
Piotr Kołaczkowski commented on CASSANDRA-6268:
-----------------------------------------------
[~alexliu68] Can you provide steps to reproduce? I did test it with another DC
configured with VNodes and didn't observe what you're saying.
> Poor performance of Hadoop if any DC is using VNodes
> ----------------------------------------------------
>
> Key: CASSANDRA-6268
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6268
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
> Reporter: Piotr Kołaczkowski
> Assignee: Piotr Kołaczkowski
> Attachments: 6268.txt
>
>
> Some customers are complaining about huge number of splits in Hadoop caused
> by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are
> generated from the results of describe_ring, which returns a huge number of
> ranges anyways, and doesn't take into account that there will be huge number
> of consecutive ranges residing on the nodes we'd like the M/R job to be run.
> The proposed fix:
> 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE -
> defaults to all Hadoop DCs)
> 2. merges consecutive ranges before generating Hadoop splits, so we don't
> have artificial range splitting caused by vnodes in the other DCs
> For non-DSE users this feature is turned off by default and doesn't change
> the old behaviour.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
