[ https://issues.apache.org/jira/browse/HDFS-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905411#comment-16905411 ]
CR Hota commented on HDFS-13123: -------------------------------- [~hemanthboyina] Thanks for the initial patch. We may need a final design doc for this task, explaining some of the below points. # How is atomicity in distcp taken into account here? If distcp fails, destination cluster may have unused files lying around unaudited. May be user can specify atomicity flag through admin. # Will all the actual work be done by common yarn queue belonging to "router" irrespective of user ? # How are multiple rebalancings going to work if executed? Should admin maintain a state of what all rebalancing is in progress and what all completed. Some basic auditing at least. # How does this rebalancing work play with overall user quota management ? # Rebalancing across secured clusters? etc. > RBF: Add a balancer tool to move data across subcluster > -------------------------------------------------------- > > Key: HDFS-13123 > URL: https://issues.apache.org/jira/browse/HDFS-13123 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Wei Yan > Assignee: hemanthboyina > Priority: Major > Attachments: HDFS Router-Based Federation Rebalancer.pdf, > HDFS-13123.patch > > > Follow the discussion in HDFS-12615. This Jira is to track effort for > building a rebalancer tool, used by router-based federation to move data > among subclusters. -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org