[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13760573#comment-13760573 ] Andrew Wang commented on HADOOP-8040: - IIUC, Arun wants to make 2.2.0 off of branch-2.1, not branch-2, so right now this isn't slated for GA. I agree with Eli though that it would be really unfortunate if symlink support wasn't in GA, since it means punting it out until the next compat-breaking release. We need to do a lot of testing though, and resolve semantics issues like HADOOP-9912. > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Fix For: 3.0.0, 2.3.0 > > Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch, > hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch, > hadoop-8040-6.patch, hadoop-8040-7.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13758688#comment-13758688 ] Colin Patrick McCabe commented on HADOOP-8040: -- All of the subtasks are in the branch-2 upstream now. When branch-2.2 is created, I assume it will come off of this branch, so it will include all of this. As it is, we have fix version set to 2.3 since there is no JIRA entry for 2.2 (yet?). > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Fix For: 3.0.0, 2.3.0 > > Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch, > hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch, > hadoop-8040-6.patch, hadoop-8040-7.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13756803#comment-13756803 ] Eli Collins commented on HADOOP-8040: - IMO we need to get this into 2.2.0 (GA), ideally you don't want GA systems around that don't fully support symlinks. > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Fix For: 3.0.0, 2.3.0 > > Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch, > hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch, > hadoop-8040-6.patch, hadoop-8040-7.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13692998#comment-13692998 ] Alejandro Abdelnur commented on HADOOP-8040: created HDFS-4933, HDFS-4934 & HDFS-4935 to add symlinks to WebHDFS. > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch, > hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch, > hadoop-8040-6.patch, hadoop-8040-7.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13692545#comment-13692545 ] Romain Rigaux commented on HADOOP-8040: --- Is there a JIRA for symlink support in WebHdfs and HttpFs API? > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch, > hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch, > hadoop-8040-6.patch, hadoop-8040-7.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13692502#comment-13692502 ] Andrew Wang commented on HADOOP-8040: - [~harip] I don't think so, since 2.1 is already close to release. Maybe we can get it in for 2.2, but there's also some more follow-on work I want to do here (adding hadoop shell support for instance). > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch, > hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch, > hadoop-8040-6.patch, hadoop-8040-7.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13692497#comment-13692497 ] Hari Mankude commented on HADOOP-8040: -- Will this patch make it into hadoop 2.1? This port of symlinks to fs classes is very useful. > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch, > hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch, > hadoop-8040-6.patch, hadoop-8040-7.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690703#comment-13690703
]
Hadoop QA commented on HADOOP-8040:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589138/hadoop-8040-7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2692//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2692//console
This message is automatically generated.
> Add symlink support to FileSystem
> -
>
> Key: HADOOP-8040
> URL: https://issues.apache.org/jira/browse/HADOOP-8040
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha
>Reporter: Eli Collins
>Assignee: Andrew Wang
> Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch,
> hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch,
> hadoop-8040-6.patch, hadoop-8040-7.patch
>
>
> HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done
> on the client-side, and therefore requires client support. An HDFS symlink
> (created by FileContext) when accessed by FileSystem will result in an
> unhandled UnresolvedLinkException. Because not all users will migrate from
> FileSystem to FileContext in lock step, and we want users of FileSystem to be
> able to access all paths created by FileContext, we need to support symlink
> resolution in FileSystem as well, to facilitate migration to FileContext.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690682#comment-13690682 ] Colin Patrick McCabe commented on HADOOP-8040: -- +1 > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch, > hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch, > hadoop-8040-6.patch, hadoop-8040-7.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13687448#comment-13687448
]
Hadoop QA commented on HADOOP-8040:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12588466/hadoop-8040-6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2668//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2668//console
This message is automatically generated.
> Add symlink support to FileSystem
> -
>
> Key: HADOOP-8040
> URL: https://issues.apache.org/jira/browse/HADOOP-8040
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha
>Reporter: Eli Collins
>Assignee: Andrew Wang
> Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch,
> hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch,
> hadoop-8040-6.patch
>
>
> HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done
> on the client-side, and therefore requires client support. An HDFS symlink
> (created by FileContext) when accessed by FileSystem will result in an
> unhandled UnresolvedLinkException. Because not all users will migrate from
> FileSystem to FileContext in lock step, and we want users of FileSystem to be
> able to access all paths created by FileContext, we need to support symlink
> resolution in FileSystem as well, to facilitate migration to FileContext.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13687390#comment-13687390 ] Colin Patrick McCabe commented on HADOOP-8040: -- bq. I refactored these out of FileContext and decided to stick them in Path. Seems kind of weird to have a "check" instance method that throws an exception like this, so I'd mildly prefer to leave them static. I'll leave it up to you, but I really prefer instance methods in this case. bq. Semantics for all the local filesystems is kinda fuzzy, but I agree that this feels incorrect. I'd prefer to fix both of these classes at once though in a follow-on JIRA (especially if JNI is potentially involved). OK. I filed HADOOP-9652 for this. bq. Hmm, interesting comment, good eye. The issue here is that since symlinks can point to other filesystems, we have to return a FileStatus, not an HdfsFileStatus. Plain old FileStatus doesn't have fileId, so we have to leave it out. I think this is okay from a user perspective, since FileSystem methods only return FileStatus, and HdfsFileStatus isn't a subclass of FileStatus anyway. Yeah, given that it has to be copied anyway, this is probably fine. Can you add a comment? bq. I think this is as intended. The new inner class sometimes needs to wrap final parameters of the containing method. Since the params are different each time (and different per call), I think it needs to do this at runtime. OK. bq. I think the intent here was to just pick a reasonable upper bound. I doubt any real user has >32 links to links, and I don't think there's a reason to tune it down either. Looks like there was some discussion on this earlier: https://issues.apache.org/jira/browse/HDFS-245?focusedCommentId=12642970&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12642970 32 is probably fine for now. If we ever need it to be configurable, we can do that in a separate JIRA. > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch, > hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch, > hadoop-8040-6.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13687131#comment-13687131
]
Colin Patrick McCabe commented on HADOOP-8040:
--
{code}
protected static void checkNotSchemeWithRelative(final Path path)
protected static void checkNotRelative(final Path path)
public static String getPathWithoutSchemeAndAuthority(Path path)
{code}
Why are these static methods rather than instance methods? They seem to just
operate on a single path, and don't do anything smart when you pass null, so I
can't see why you'd want them to be static. I also don't see why you would
want {{protected}} here, rather than package-private (i.e. no qualifier.).
Also, a function whose name begins with getPath... should return a path. Let
the caller call toString if he wants to.
{code}
System.out.println("RLFS wd: " + workingDir);
{code}
Debug printout left in.
{code}
private FileStatus getFileLinkStatusInternal(final Path f) throws IOException
{
String target = readLink(f);
try {
FileStatus fs = getFileStatus(f);
// If f refers to a regular file or directory
if (target.isEmpty()) {
return fs;
}
// Otherwise f refers to a symlink
return new FileStatus(fs.getLen(),
false,
fs.getReplication(),
fs.getBlockSize(),
fs.getModificationTime(),
fs.getAccessTime(),
fs.getPermission(),
fs.getOwner(),
fs.getGroup(),
new Path(target),
f);
{code}
Symlinks do not necessarily have the same owner as the file they point to.
Here's any example:
{code}
cmccabe@keter:~/hadoop2> touch a
cmccabe@keter:~/hadoop2> ln -s a b
cmccabe@keter:~/hadoop2> sudo chown bob b
root's password:
cmccabe@keter:~/hadoop2> ls -l a b
-rw-r--r-- 1 bob users 0 Jun 18 12:14 a
lrwxrwxrwx 1 cmccabe users 1 Jun 18 12:14 b -> a
{code}
Symlinks have a file mode (aka permission bits) which are independent of the
files they point to. On certain UNIXes (but not Linux), symlinks can have
permissions other than 0777. I'm not sure if you can get access to lstat from
JDK6, or if you'll need JNI for this.
{code}
final public FileStatus makeQualified(URI defaultUri, Path path) {
{code}
This is basically a copy constructor that also modifies
{{HdfsFileStatus#path}}. Are you sure it wouldn't be easier to just have a
method to change the path of the object? As new fields are added to
HdfsFileStatus, it's going to be very, very easy for people to forget to change
this function to copy them. In fact, you seem to have left a field out here
yourself-- fileId. This is especially an issue when subclasses start coming
into play. All the problems that plague clone() plague this method too. I
also feel like having a mutator like setParentPath(URI defaultURI, Path parent)
would be more efficient, as well as more maintainable.
FSLinkResolver: it seems like you only need one of these per function. Since
they are stateless, there isn't any point in creating garbage that you'll have
to collect later. Just make a static FSLinkResolver and use
that in getContentSummary. etc. etc. Also, should MAX_PATH_LINKS be
configurable?
> Add symlink support to FileSystem
> -
>
> Key: HADOOP-8040
> URL: https://issues.apache.org/jira/browse/HADOOP-8040
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha
>Reporter: Eli Collins
>Assignee: Andrew Wang
> Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch,
> hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch
>
>
> HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done
> on the client-side, and therefore requires client support. An HDFS symlink
> (created by FileContext) when accessed by FileSystem will result in an
> unhandled UnresolvedLinkException. Because not all users will migrate from
> FileSystem to FileContext in lock step, and we want users of FileSystem to be
> able to access all paths created by FileContext, we need to support symlink
> resolution in FileSystem as well, to facilitate migration to FileContext.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13686074#comment-13686074
]
Hadoop QA commented on HADOOP-8040:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12588205/hadoop-8040-5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2661//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2661//console
This message is automatically generated.
> Add symlink support to FileSystem
> -
>
> Key: HADOOP-8040
> URL: https://issues.apache.org/jira/browse/HADOOP-8040
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha
>Reporter: Eli Collins
>Assignee: Andrew Wang
> Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch,
> hadoop-8040-3.patch, hadoop-8040-4.patch, hadoop-8040-5.patch
>
>
> HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done
> on the client-side, and therefore requires client support. An HDFS symlink
> (created by FileContext) when accessed by FileSystem will result in an
> unhandled UnresolvedLinkException. Because not all users will migrate from
> FileSystem to FileContext in lock step, and we want users of FileSystem to be
> able to access all paths created by FileContext, we need to support symlink
> resolution in FileSystem as well, to facilitate migration to FileContext.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684035#comment-13684035
]
Hadoop QA commented on HADOOP-8040:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12587934/hadoop-8040-4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1153 javac
compiler warnings (more than the trunk's current 1152 warnings).
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.TestSymlinkLocalFSFileContext
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2651//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2651//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2651//console
This message is automatically generated.
> Add symlink support to FileSystem
> -
>
> Key: HADOOP-8040
> URL: https://issues.apache.org/jira/browse/HADOOP-8040
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha
>Reporter: Eli Collins
>Assignee: Andrew Wang
> Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch,
> hadoop-8040-3.patch, hadoop-8040-4.patch
>
>
> HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done
> on the client-side, and therefore requires client support. An HDFS symlink
> (created by FileContext) when accessed by FileSystem will result in an
> unhandled UnresolvedLinkException. Because not all users will migrate from
> FileSystem to FileContext in lock step, and we want users of FileSystem to be
> able to access all paths created by FileContext, we need to support symlink
> resolution in FileSystem as well, to facilitate migration to FileContext.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622694#comment-13622694 ] Andrew Wang commented on HADOOP-8040: - Hey folks, just a status update: I'm waiting on a more substantive review before rolling up the javac warning and Brock's nit. Thanks for your comments thus far! > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch, > hadoop-8040-3.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13616278#comment-13616278 ] Brock Noland commented on HADOOP-8040: -- I took a quick look at this and I did notice there is at least one sysout which is not in a test. > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch, > hadoop-8040-3.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608383#comment-13608383
]
Hadoop QA commented on HADOOP-8040:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574640/hadoop-8040-3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:red}-1 javac{color}. The applied patch generated 1364 javac
compiler warnings (more than the trunk's current 1363 warnings).
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2346//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2346//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2346//console
This message is automatically generated.
> Add symlink support to FileSystem
> -
>
> Key: HADOOP-8040
> URL: https://issues.apache.org/jira/browse/HADOOP-8040
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha
>Reporter: Eli Collins
>Assignee: Andrew Wang
> Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch,
> hadoop-8040-3.patch
>
>
> HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done
> on the client-side, and therefore requires client support. An HDFS symlink
> (created by FileContext) when accessed by FileSystem will result in an
> unhandled UnresolvedLinkException. Because not all users will migrate from
> FileSystem to FileContext in lock step, and we want users of FileSystem to be
> able to access all paths created by FileContext, we need to support symlink
> resolution in FileSystem as well, to facilitate migration to FileContext.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608259#comment-13608259
]
Daryn Sharp commented on HADOOP-8040:
-
I need to find test to look at this more, but I would expect if the base
{{FileSystem}} implementation is sufficiently generic, {{ViewFileSystem}} would
"just work" because it would resolve the paths relative to the viewfs path. It
should only be a problem if delegating the resolution into nested filesystems,
ala {{ViewFileSystem}} mount points, and through a {{FilterFileSystem}}.
> Add symlink support to FileSystem
> -
>
> Key: HADOOP-8040
> URL: https://issues.apache.org/jira/browse/HADOOP-8040
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha
>Reporter: Eli Collins
>Assignee: Andrew Wang
> Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch,
> hadoop-8040-3.patch
>
>
> HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done
> on the client-side, and therefore requires client support. An HDFS symlink
> (created by FileContext) when accessed by FileSystem will result in an
> unhandled UnresolvedLinkException. Because not all users will migrate from
> FileSystem to FileContext in lock step, and we want users of FileSystem to be
> able to access all paths created by FileContext, we need to support symlink
> resolution in FileSystem as well, to facilitate migration to FileContext.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607949#comment-13607949
]
Andrew Wang commented on HADOOP-8040:
-
Hi Daryn, thanks for taking a look.
This patch only adds symlink support to {{DistributedFileSystem}} and
{{LocalFileSystem}}. Other filesystems will throw
{{UnsupportedOperationException}}s. It's going to be tricky to add support to
{{ViewFileSystem}} and {{ChRootedFileSystem}} without a internal/external
interface split like in FileContext/AbstractFileSystem, since just calling the
underlying filesystem's methods will lead to internal resolution. I think this
could be fixed by checking paths first with {{FileSystem#resolveLink}}, or
substituting in a viewfs-specific {{FSLinkResolver}} in the underlying FSs,
which, instead of qualifying against the underlying FS, qualify against the
viewfs.
Perhaps we can revisit that in another JIRA, I think this is pretty useful with
just {{DFS}} support.
Relative symlinks are tested in {{SymlinkBaseTest}}, and I believe handled
correctly. Symlink targets are saved as specified at creation. Resolution
happens in {{FSLinkResolver#resolve}}: we fully qualify relative link targets
against their containing filesystem, then try the call again with the new
fully-qualified {{Path}}.
> Add symlink support to FileSystem
> -
>
> Key: HADOOP-8040
> URL: https://issues.apache.org/jira/browse/HADOOP-8040
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha
>Reporter: Eli Collins
>Assignee: Andrew Wang
> Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch
>
>
> HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done
> on the client-side, and therefore requires client support. An HDFS symlink
> (created by FileContext) when accessed by FileSystem will result in an
> unhandled UnresolvedLinkException. Because not all users will migrate from
> FileSystem to FileContext in lock step, and we want users of FileSystem to be
> able to access all paths created by FileContext, we need to support symlink
> resolution in FileSystem as well, to facilitate migration to FileContext.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607621#comment-13607621
]
Daryn Sharp commented on HADOOP-8040:
-
I've only done a cursory scan of the patch, but does this work correctly with
{{ViewFileSystem}} and {{ChrootedFileSystem}}? Ie. will a relative or absolute
symlink path be resolved relative to the underlying filesystem (incorrect), or
to the mount table/chroot (correct)? Also, does this support relative
symlinks? It doesn't look like it does?
> Add symlink support to FileSystem
> -
>
> Key: HADOOP-8040
> URL: https://issues.apache.org/jira/browse/HADOOP-8040
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha
>Reporter: Eli Collins
>Assignee: Andrew Wang
> Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch
>
>
> HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done
> on the client-side, and therefore requires client support. An HDFS symlink
> (created by FileContext) when accessed by FileSystem will result in an
> unhandled UnresolvedLinkException. Because not all users will migrate from
> FileSystem to FileContext in lock step, and we want users of FileSystem to be
> able to access all paths created by FileContext, we need to support symlink
> resolution in FileSystem as well, to facilitate migration to FileContext.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607140#comment-13607140
]
Hadoop QA commented on HADOOP-8040:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574439/hadoop-8040-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:red}-1 javac{color}. The applied patch generated 1364 javac
compiler warnings (more than the trunk's current 1363 warnings).
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.TestSymlinkHdfsFileSystem
org.apache.hadoop.fs.TestSymlinkHdfsFileContext
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2344//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2344//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2344//console
This message is automatically generated.
> Add symlink support to FileSystem
> -
>
> Key: HADOOP-8040
> URL: https://issues.apache.org/jira/browse/HADOOP-8040
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha
>Reporter: Eli Collins
>Assignee: Andrew Wang
> Attachments: hadoop-8040-1.patch, hadoop-8040-2.patch
>
>
> HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done
> on the client-side, and therefore requires client support. An HDFS symlink
> (created by FileContext) when accessed by FileSystem will result in an
> unhandled UnresolvedLinkException. Because not all users will migrate from
> FileSystem to FileContext in lock step, and we want users of FileSystem to be
> able to access all paths created by FileContext, we need to support symlink
> resolution in FileSystem as well, to facilitate migration to FileContext.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13606732#comment-13606732 ] Andrew Wang commented on HADOOP-8040: - Working on the test patch failures. I think the test and FSLinkResolver related patches can still be reviewed in the meantime, since the test failures are related to the DistributedFileSystem and LocalFileSystem implementations. > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Eli Collins >Assignee: Andrew Wang > Attachments: hadoop-8040-1.patch > > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13606105#comment-13606105
]
Hadoop QA commented on HADOOP-8040:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574315/hadoop-8040-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 11 new
or modified test files.
{color:red}-1 one of tests included doesn't have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
71 warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 2
release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.TestFilterFileSystem
org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem
org.apache.hadoop.fs.viewfs.TestViewFsTrash
org.apache.hadoop.hdfs.TestFileCreation
org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
org.apache.hadoop.hdfs.TestDistributedFileSystem
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2341//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2341//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2341//console
This message is automatically generated.
> Add symlink support to FileSystem
> -
>
> Key: HADOOP-8040
> URL: https://issues.apache.org/jira/browse/HADOOP-8040
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha
>Reporter: Eli Collins
>Assignee: Andrew Wang
> Attachments: hadoop-8040-1.patch
>
>
> HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done
> on the client-side, and therefore requires client support. An HDFS symlink
> (created by FileContext) when accessed by FileSystem will result in an
> unhandled UnresolvedLinkException. Because not all users will migrate from
> FileSystem to FileContext in lock step, and we want users of FileSystem to be
> able to access all paths created by FileContext, we need to support symlink
> resolution in FileSystem as well, to facilitate migration to FileContext.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13596576#comment-13596576 ] Andrew Wang commented on HADOOP-8040: - Yea, there is. I took this up somewhat recently, I'm hoping to get a patch up in the next week or two. > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0 >Reporter: Eli Collins >Assignee: Andrew Wang > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8040) Add symlink support to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13596564#comment-13596564 ] Konstantin Shvachko commented on HADOOP-8040: - This is one year old jira. Andrew, is there a plan to move forward with it? > Add symlink support to FileSystem > - > > Key: HADOOP-8040 > URL: https://issues.apache.org/jira/browse/HADOOP-8040 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 0.23.0 >Reporter: Eli Collins >Assignee: Andrew Wang > > HADOOP-6421 added symbolic links to FileContext. Resolving symlinks is done > on the client-side, and therefore requires client support. An HDFS symlink > (created by FileContext) when accessed by FileSystem will result in an > unhandled UnresolvedLinkException. Because not all users will migrate from > FileSystem to FileContext in lock step, and we want users of FileSystem to be > able to access all paths created by FileContext, we need to support symlink > resolution in FileSystem as well, to facilitate migration to FileContext. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
