[ https://issues.apache.org/jira/browse/HDFS-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13969806#comment-13969806 ]
Daryn Sharp commented on HDFS-6165: ----------------------------------- Two-part comment to avoid overly lengthy response. The FsShell logic isn't correctly. You can't simply ignore non-empty dirs. Removal of the parent dir will fail too. I think the shell rm will then return 0 (success) even though it miserably failed. Checking for RemoteException is making assumptions about hdfs semantics, and searching for a substring is very fragile. I guess I should step back and explain how posix deletes work, although maybe everyone already knows this. There is no recursive delete. A userland tool has to recurse the directory structure and remove all children. Rm prompts when it cannot traverse a directory (lack of perms). However, the kernel will treat an empty directory just like a file - no permissions are required to delete. Rm is prompting because it doesn't know if the remove will work. (Sidenote: I tested on my mac and it will allow rmdir of a non-perm empty dir, so it's rm just being fussy and not trying to delete when -f is given even though it could) > "hdfs dfs -rm -r" and "hdfs -rmdir" commands can't remove empty directory > -------------------------------------------------------------------------- > > Key: HDFS-6165 > URL: https://issues.apache.org/jira/browse/HDFS-6165 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client > Affects Versions: 2.3.0 > Reporter: Yongjun Zhang > Assignee: Yongjun Zhang > Priority: Minor > Attachments: HDFS-6165.001.patch, HDFS-6165.002.patch, > HDFS-6165.003.patch, HDFS-6165.004.patch, HDFS-6165.004.patch > > > Given a directory owned by user A with WRITE permission containing an empty > directory owned by user B, it is not possible to delete user B's empty > directory with either "hdfs dfs -rm -r" or "hdfs dfs -rmdir". Because the > current implementation requires FULL permission of the empty directory, and > throws exception. > On the other hand, on linux, "rm -r" and "rmdir" command can remove empty > directory as long as the parent directory has WRITE permission (and prefix > component of the path have EXECUTE permission), For the tested OSes, some > prompt user asking for confirmation, some don't. > Here's a reproduction: > {code} > [root@vm01 ~]# hdfs dfs -ls /user/ > Found 4 items > drwxr-xr-x - userabc users 0 2013-05-03 01:55 /user/userabc > drwxr-xr-x - hdfs supergroup 0 2013-05-03 00:28 /user/hdfs > drwxrwxrwx - mapred hadoop 0 2013-05-03 00:13 /user/history > drwxr-xr-x - hdfs supergroup 0 2013-04-14 16:46 /user/hive > [root@vm01 ~]# hdfs dfs -ls /user/userabc > Found 8 items > drwx------ - userabc users 0 2013-05-02 17:00 /user/userabc/.Trash > drwxr-xr-x - userabc users 0 2013-05-03 01:34 /user/userabc/.cm > drwx------ - userabc users 0 2013-05-03 01:06 > /user/userabc/.staging > drwxr-xr-x - userabc users 0 2013-04-14 18:31 /user/userabc/apps > drwxr-xr-x - userabc users 0 2013-04-30 18:05 /user/userabc/ds > drwxr-xr-x - hdfs users 0 2013-05-03 01:54 /user/userabc/foo > drwxr-xr-x - userabc users 0 2013-04-30 16:18 > /user/userabc/maven_source > drwxr-xr-x - hdfs users 0 2013-05-03 01:40 > /user/userabc/test-restore > [root@vm01 ~]# hdfs dfs -ls /user/userabc/foo/ > [root@vm01 ~]# sudo -u userabc hdfs dfs -rm -r -skipTrash /user/userabc/foo > rm: Permission denied: user=userabc, access=ALL, > inode="/user/userabc/foo":hdfs:users:drwxr-xr-x > {code} > The super user can delete the directory. > {code} > [root@vm01 ~]# sudo -u hdfs hdfs dfs -rm -r -skipTrash /user/userabc/foo > Deleted /user/userabc/foo > {code} > The same is not true for files, however. They have the correct behavior. > {code} > [root@vm01 ~]# sudo -u hdfs hdfs dfs -touchz /user/userabc/foo-file > [root@vm01 ~]# hdfs dfs -ls /user/userabc/ > Found 8 items > drwx------ - userabc users 0 2013-05-02 17:00 /user/userabc/.Trash > drwxr-xr-x - userabc users 0 2013-05-03 01:34 /user/userabc/.cm > drwx------ - userabc users 0 2013-05-03 01:06 > /user/userabc/.staging > drwxr-xr-x - userabc users 0 2013-04-14 18:31 /user/userabc/apps > drwxr-xr-x - userabc users 0 2013-04-30 18:05 /user/userabc/ds > -rw-r--r-- 1 hdfs users 0 2013-05-03 02:11 > /user/userabc/foo-file > drwxr-xr-x - userabc users 0 2013-04-30 16:18 > /user/userabc/maven_source > drwxr-xr-x - hdfs users 0 2013-05-03 01:40 > /user/userabc/test-restore > [root@vm01 ~]# sudo -u userabc hdfs dfs -rm -skipTrash /user/userabc/foo-file > Deleted /user/userabc/foo-file > {code} > Using "hdfs dfs -rmdir" command: > {code} > bash-4.1$ hadoop fs -lsr / > lsr: DEPRECATED: Please use 'ls -R' instead. > drwxr-xr-x - hdfs supergroup 0 2014-03-25 16:29 /user > drwxr-xr-x - hdfs supergroup 0 2014-03-25 16:28 /user/hdfs > drwxr-xr-x - usrabc users 0 2014-03-28 23:39 /user/usrabc > drwxr-xr-x - abc abc 0 2014-03-28 23:39 > /user/usrabc/foo-empty1 > [root@vm01 usrabc]# su usrabc > [usrabc@vm01 ~]$ hdfs dfs -rmdir /user/usrabc/foo-empty1 > rmdir: Permission denied: user=usrabc, access=ALL, > inode="/user/usrabc/foo-empty1":abc:abc:drwxr-xr-x > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)