[
https://issues.apache.org/jira/browse/HDFS-14316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796793#comment-16796793
]
Ayush Saxena commented on HDFS-14316:
-------------------------------------
Thanx [~elgoiri] for the patch.
* For the second scenario, I guess we can drop that loop as of now, as it
doesn't seem to be doing anything to me and mark a TODO comment to handle in
future, if supposidly we get an ALT. I guess retrying there won't be that high
at cost. That NS isn't serving so at NS that is no loss. Here we may loose some
milliseconds, if that failed destination lands up being the first one we try.
But I guess that shall be OK
* For the file duplication, setting fault tolerant is the router admin call. If
he is pretty sure that the client to the mount won't write duplicate files then
this shall be very good. I guess there are couple of general use case where
this can happen. Might be a line of warning or at the document, explaining this
would be fair enough
* For the files I guess this is pretty good. But if we look at the directory
level, There might be some problem with non-FolderAll orders. For FolderAll the
directories are supposed to be in all the subclusters, so while writing file
too. It shall land up at least in the correct Folder and have the correct
attributes(Like EC Policy, Storage Policy). For the non-FolderAll. For file on
non-availability if it creates the directory, then multiple Folders for the
mount entry can also come up. For mkdirs also this problem of multiple folder
can come up with these order. I guess little bit use case of the orders may get
compromised with this. Provided our attributes invocation is also based on
ORDER and they consider NonFolderAll mounts to have single destination only, So
they are sequential invocation.Here also it might be a problem. Is it possible,
We allow fault tolerance on FolderAll mounts only?
* Apart in the code there are couple of stringBuilder appends. I guess
concurrent string builders can be reused, that lands up with a lighter byte
code, as claimed by a PDM warning.(sb.append(...); sb.append(..) to
sb.append(...) .append(..))
* This may require a rebase after HDFS-14351.
> RBF: Support unavailable subclusters for mount points with multiple
> destinations
> --------------------------------------------------------------------------------
>
> Key: HDFS-14316
> URL: https://issues.apache.org/jira/browse/HDFS-14316
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Íñigo Goiri
> Assignee: Íñigo Goiri
> Priority: Major
> Attachments: HDFS-14316-HDFS-13891.000.patch,
> HDFS-14316-HDFS-13891.001.patch, HDFS-14316-HDFS-13891.002.patch,
> HDFS-14316-HDFS-13891.003.patch, HDFS-14316-HDFS-13891.004.patch,
> HDFS-14316-HDFS-13891.005.patch, HDFS-14316-HDFS-13891.006.patch,
> HDFS-14316-HDFS-13891.007.patch, HDFS-14316-HDFS-13891.008.patch
>
>
> Currently mount points with multiple destinations (e.g., HASH_ALL) fail
> writes when the destination subcluster is down. We need an option to allow
> writing in other subclusters when one is down.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
