[
https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17128305#comment-17128305
]
Jinglun edited comment on HDFS-15346 at 6/9/20, 1:33 PM:
---------------------------------------------------------
Hi [~linyiqun], thanks your great comments and valuable suggestions ! I'll
need some time to shoot all of them. So let me respond to the question first.
{quote}Here we reset permission to 0, that means no any operation is allowed?
Is this expected, why not is 400 (only allow read)? The comment said that
'cancelling the x permission of the source path.' makes me confused.
{quote}
Yes here we reset the permission to 0. Both read and write in the source path
and all its sub-paths are denied. As far as I know all the read operations need
to check its parents' execution permission. So setting to 400 can't make it
only allowing read. We still can't read its sub-paths. I think the only way to
make it 'only allowing read' is to recursively reduce each directory's
permission to 555. Reduce permission means: if the original permission is 777
then change it to 555. If the original permission is 700 then make it to 500.
Saving all the directories' permissions is very expensive. A better way may be
letting the NameNode to support 'readonly-directory'. I think we can first
using the '0 permission' way to make sure the data is consistent. Then start a
sub-task to enable the NameNode 'readonly-directory'. Finally change this to
the NameNode 'readonly-directory'.
{quote}One follow-up task I am thinking that we can have a separated config
file something named fedbalance-default.xml for fedbalance tool, like
ditcp-default.xml for distcp tool now. I don't prefer to add all tool config
settings into hdfs-default.xml.
{quote}
Agree with you ! Using a fedbalance-default.xml is much better.
{quote}The test need a little long time to execute the whole test.
{quote}
I'll try to figure it out. But it might be quite tricky as the unit tests use
both MiniDFSCluster and MiniMRYarnCluster. And there are many rounds of distcp.
Please tell me if you have any suggestions, thanks !
was (Author: lijinglun):
Hi [~linyiqun], thanks your great comments and valuable suggestions ! I'll
need some time to shoot all of them. So let me respond to the question first.
{quote}Here we reset permission to 0, that means no any operation is allowed?
Is this expected, why not is 400 (only allow read)? The comment said that
'cancelling the x permission of the source path.' makes me confused.
{quote}
Yes here we reset the permission to 0. Both read and write in the source path
and all its sub-paths are denied. As far as I know all the read operations need
to check its parents' execution permission. So setting to 400 can't make it
only allowing read. We still can't read its sub-paths. I think the only way to
make it 'only allowing read' is to recursively reduce each directory's
permission to 555. Reduce permission means: if the original permission is 777
then change it to 555. If the original permission is 700 then make it to 500.
Saving all the directories' permissions is very expensive. A better way may be
letting the NameNode to support 'readonly-directory'. I think we can first
using the '0 permission' way to make sure the data is consistent. Then start a
sub-task to enable the NameNode 'readonly-directory'. Finally change this to
the NameNode 'readonly-directory'.
> RBF: DistCpFedBalance implementation
> ------------------------------------
>
> Key: HDFS-15346
> URL: https://issues.apache.org/jira/browse/HDFS-15346
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jinglun
> Assignee: Jinglun
> Priority: Major
> Attachments: HDFS-15346.001.patch, HDFS-15346.002.patch,
> HDFS-15346.003.patch, HDFS-15346.004.patch, HDFS-15346.005.patch,
> HDFS-15346.006.patch, HDFS-15346.007.patch, HDFS-15346.008.patch
>
>
> Patch in HDFS-15294 is too big to review so we split it into 2 patches. This
> is the second one. Detail can be found at HDFS-15294.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
