[ https://issues.apache.org/jira/browse/HDFS-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359032#comment-16359032 ]
Chris Douglas commented on HDFS-13123: -------------------------------------- bq. hard linking across block pools as one option and even tiered storage Yes, [~virajith] and I wrote a prototype of this with an intern for a similar project. Making it a proper transaction is complicated, but architecturally RBF is in the right place to coordinate this cleanly. We added APIs to generate and attach an FSImage for a NN subtree. Attaching an image required reallocating not only the inodeIds but also the blockIds, which were hardlinked into a contiguous range in the destination blockId space. We didn't solve all the failover and edit log cases, but these seem tractable as long as the subtree is immutable. Without that assumption (which RBF can enforce/detect) thar be dragons. > RBF: Add a balancer tool to move data across subsluter > ------------------------------------------------------- > > Key: HDFS-13123 > URL: https://issues.apache.org/jira/browse/HDFS-13123 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Wei Yan > Assignee: Wei Yan > Priority: Major > Attachments: HDFS Router-Based Federation Rebalancer.pdf > > > Follow the discussion in HDFS-12615. This Jira is to track effort for > building a rebalancer tool, used by router-based federation to move data > among subclusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org