[ https://issues.apache.org/jira/browse/HADOOP-9984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800031#comment-13800031 ]
Sanjay Radia commented on HADOOP-9984: -------------------------------------- bq. [sanjay] Fix all internal utilities, hive, pig, map reduce, yarn, etc to not use isDir() and understand that a directory may contain symlinks. bq. [daryn] I do not agree. This means symlinks are not transparent and not compatible with pre-2.x. Our tools *may* want to copy a symlink as-is rather than copy the file it refers to; all I am saying is that if there is a need to do that we need to fix such tools. For example, distcp needs to be symlink-aware rather than blindly copy 1PB when in reality one would have desired to copy the symlink. The main concern is other applications that are doing a listStatus + isDir; for that I have listed 2 options and my own personal opinion on what we should do. > FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by > default > ---------------------------------------------------------------------------------- > > Key: HADOOP-9984 > URL: https://issues.apache.org/jira/browse/HADOOP-9984 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs > Affects Versions: 2.1.0-beta > Reporter: Colin Patrick McCabe > Assignee: Colin Patrick McCabe > Priority: Blocker > Attachments: HADOOP-9984.001.patch, HADOOP-9984.003.patch, > HADOOP-9984.005.patch, HADOOP-9984.007.patch, HADOOP-9984.009.patch, > HADOOP-9984.010.patch, HADOOP-9984.011.patch, HADOOP-9984.012.patch, > HADOOP-9984.013.patch, HADOOP-9984.014.patch, HADOOP-9984.015.patch > > > During the process of adding symlink support to FileSystem, we realized that > many existing HDFS clients would be broken by listStatus and globStatus > returning symlinks. One example is applications that assume that > !FileStatus#isFile implies that the inode is a directory. As we discussed in > HADOOP-9972 and HADOOP-9912, we should default these APIs to returning > resolved paths. -- This message was sent by Atlassian JIRA (v6.1#6144)