[
https://issues.apache.org/jira/browse/HDFS-15087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134754#comment-17134754
]
Jinglun commented on HDFS-15087:
--------------------------------
Hi [Uma Maheswara Rao
G|https://xiaomi.f.mioffice.cn/jira/secure/ViewProfile.jspa?name=umamaheswararao],
thanks your comments !
{quote}Is that snapshots mandatory for this feature to work?{quote}
The old design in this Jira is abandoned. The new design in HDFS-15294 uses
distcp with snapshot diff to sync the source and the target. So yes the
snapshot is mandatory.
{quote}If it's mandatory, would it be possible to think and make it as optional
and have alternative thoughts to get the diff?{quote}
If we want to balance from HDFS to S3 or ozone then we need a new way to sync
data. May be we can improve distcp to support sync without snapshot. Something
needs to be done is to ignore the deleted files and preserve all the attributes
when they are changed. Just some initial thoughts and I think it could be a
solution.
> RBF: Balance/Rename across federation namespaces
> ------------------------------------------------
>
> Key: HDFS-15087
> URL: https://issues.apache.org/jira/browse/HDFS-15087
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Jinglun
> Assignee: Jinglun
> Priority: Major
> Attachments: HDFS-15087.initial.patch, HFR_Rename Across Federation
> Namespaces.pdf
>
>
> The Xiaomi storage team has developed a new feature called HFR(HDFS
> Federation Rename) that enables us to do balance/rename across federation
> namespaces. The idea is to first move the meta to the dst NameNode and then
> link all the replicas. It has been working in our largest production cluster
> for 2 months. We use it to balance the namespaces. It turns out HFR is fast
> and flexible. The detail could be found in the design doc.
> Looking forward to a lively discussion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
