[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475614#comment-13475614 ] Hudson commented on HADOOP-8906: Integrated in Hadoop-Mapreduce-trunk #1225 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1225/]) HADOOP-8906. paths with multiple globs are unreliable. Contributed by Daryn Sharp. (Revision 1397704) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397704 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestGlobPaths.java > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Fix For: 2.0.3-alpha, 0.23.5 > > Attachments: HADOOP-8906-branch_0.23.patch, > HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch, HADOOP-8906.patch, > HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475601#comment-13475601 ] Hudson commented on HADOOP-8906: Integrated in Hadoop-Hdfs-trunk #1194 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1194/]) HADOOP-8906. paths with multiple globs are unreliable. Contributed by Daryn Sharp. (Revision 1397704) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397704 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestGlobPaths.java > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Fix For: 2.0.3-alpha, 0.23.5 > > Attachments: HADOOP-8906-branch_0.23.patch, > HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch, HADOOP-8906.patch, > HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475597#comment-13475597 ] Hudson commented on HADOOP-8906: Integrated in Hadoop-Hdfs-0.23-Build #403 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/403/]) HADOOP-8906. paths with multiple globs are unreliable. Contributed by Daryn Sharp. (Revision 1397709) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397709 Files : * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestGlobPaths.java > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Fix For: 2.0.3-alpha, 0.23.5 > > Attachments: HADOOP-8906-branch_0.23.patch, > HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch, HADOOP-8906.patch, > HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475349#comment-13475349 ] Hudson commented on HADOOP-8906: Integrated in Hadoop-Mapreduce-trunk-Commit #2881 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2881/]) HADOOP-8906. paths with multiple globs are unreliable. Contributed by Daryn Sharp. (Revision 1397704) Result = FAILURE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397704 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestGlobPaths.java > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Fix For: 2.0.3-alpha, 0.23.5 > > Attachments: HADOOP-8906-branch_0.23.patch, > HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch, HADOOP-8906.patch, > HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475296#comment-13475296 ] Hudson commented on HADOOP-8906: Integrated in Hadoop-Common-trunk-Commit #2858 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2858/]) HADOOP-8906. paths with multiple globs are unreliable. Contributed by Daryn Sharp. (Revision 1397704) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397704 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestGlobPaths.java > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HADOOP-8906-branch_0.23.patch, > HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch, HADOOP-8906.patch, > HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475294#comment-13475294 ] Hudson commented on HADOOP-8906: Integrated in Hadoop-Hdfs-trunk-Commit #2920 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2920/]) HADOOP-8906. paths with multiple globs are unreliable. Contributed by Daryn Sharp. (Revision 1397704) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397704 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestGlobPaths.java > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HADOOP-8906-branch_0.23.patch, > HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch, HADOOP-8906.patch, > HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475270#comment-13475270 ] Jason Lowe commented on HADOOP-8906: +1, thanks Daryn. I'll commit this shortly. > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HADOOP-8906-branch_0.23.patch, > HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch, HADOOP-8906.patch, > HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475210#comment-13475210
]
Hadoop QA commented on HADOOP-8906:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548911/HADOOP-8906.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1622//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1622//console
This message is automatically generated.
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-8906-branch_0.23.patch,
> HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch, HADOOP-8906.patch,
> HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch
>
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474297#comment-13474297
]
Hadoop QA commented on HADOOP-8906:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548745/HADOOP-8906.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1611//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1611//console
This message is automatically generated.
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch,
> HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch
>
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474267#comment-13474267 ] Jason Lowe commented on HADOOP-8906: bq. In essence, perhaps a user filter means the query is always a glob? I can see it going either way. Yes, I thought about that as well. Maybe it would be more consistent to return empty instead of null in that case, but I was erring on the side of caution to maintain compatibility with the previous version's behavior. It all comes down to what a result of null really means. If it's being used to check for globs in the path then arguably we should continue to return null because someone could be using/abusing globStatus(path, falseFilter) to check for globs in a path even if the path exists in the filesystem. > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch, > HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474256#comment-13474256 ] Daryn Sharp commented on HADOOP-8906: - Yes, I had that case but somehow accidentally removed in the final patch. The case of a non-glob path with a user-supplied filter is an interesting one. "null" means the path isn't a glob AND doesn't exist. When an existing non-glob path is removed by the filter, then arguably maybe it should return empty array since it's not that the path doesn't exist but that the filter had no matches. In essence, perhaps a user filter means the query is always a glob? I can see it going either way. > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch, > HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474251#comment-13474251 ] Jason Lowe commented on HADOOP-8906: The corner case is now handled, but the new tests added don't test for it. There should be a test for a non-globbed path for an existing file with the false filter, and all the false filter tests check for either a globbed path or non-existent files. I tried adding a test locally with the false filter for "/" and noticed that it didn't return null. Instead it returned "/" because the filter isn't applied in the special cases of "/" and "", which seems wrong. It turns out that the existing code also had this bug, so I suppose it's at least consistent with the previous version's behavior. > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch, > HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474218#comment-13474218
]
Hadoop QA commented on HADOOP-8906:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548747/HADOOP-8906-branch_0.23.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1612//console
This message is automatically generated.
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-8906-branch_0.23.patch, HADOOP-8906.patch,
> HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch
>
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473865#comment-13473865
]
Hadoop QA commented on HADOOP-8906:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548681/HADOOP-8906.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1607//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1607//console
This message is automatically generated.
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch,
> HADOOP-8906.patch
>
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473802#comment-13473802 ] Jason Lowe commented on HADOOP-8906: Thanks for the updates, Daryn. I'm eager to see the Jenkins results, although it seems the Jenkins build is stuck right now. After a closer look, I'm wondering if there's one more subtle difference between the old and new versions, this time when the user specifies a filter. I think the old version will return null in the case of an non-globbed path that found a file but it didn't pass the specified filter. It applies the filter as it searches before it checks for the empty-array-should-return-null case. In the new version, it applies the specified filter *after* it checks for whether there are matches, and in this scenario there will be a match since the filter hasn't been applied yet. I'm wondering if we should apply the filter before checking for an empty match array to align with the old behavior. Granted it's odd to provide a non-globbed path that wouldn't pass the filter, so it's a corner-case. Otherwise everything looks great, pending Jenkins since it found some things in the last run that we missed. > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch, > HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473714#comment-13473714
]
Hadoop QA commented on HADOOP-8906:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548634/HADOOP-8906.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.TestFsShellCopy
org.apache.hadoop.hdfs.TestDFSShell
org.apache.hadoop.cli.TestHDFSCLI
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1603//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1603//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1603//console
This message is automatically generated.
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch
>
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473709#comment-13473709 ] Suresh Srinivas commented on HADOOP-8906: - BTW the findbug seems to be related to this patch. > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473653#comment-13473653 ] Suresh Srinivas commented on HADOOP-8906: - Not sure if Jason's comment is addressed in the new patch. +1 with that addressed. > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473594#comment-13473594
]
Hadoop QA commented on HADOOP-8906:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548616/HADOOP-8906.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.TestFsShellCopy
org.apache.hadoop.hdfs.TestDFSShell
org.apache.hadoop.cli.TestHDFSCLI
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1601//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1601//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1601//console
This message is automatically generated.
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-8906.patch, HADOOP-8906.patch, HADOOP-8906.patch
>
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473542#comment-13473542
]
Jason Lowe commented on HADOOP-8906:
The changes look good, but I think there's a lingering bug wrt. how
GlobExpander's results are handled. For example, I would expect the following
test to pass:
{noformat}
assertEquals(0, fs.globStatus(new
Path("/{nonexistent1/a,nonexistent2/b}")).length);
{noformat}
However it fails because globStatus is returning null instead of an empty
array. FsShell relies on the fact that globStatus will never return null for
paths that contain pattern characters, and we could end up creating paths with
pattern characters. I checked trunk, and it turns out this test crashes
globStatus internally. Not crashing is nice, but we shouldn't be reporting
null for paths with pattern characters. Fortunately the fix is pretty easy.
We just need to check if GlobExpander returned more than one result which
implies patterns were used and we should convert a result of null into an empty
array in that case.
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-8906.patch, HADOOP-8906.patch
>
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473481#comment-13473481
]
Daryn Sharp commented on HADOOP-8906:
-
Yes, {{FsShell}}'s ls is working as expected, but it's output format leaves
quite a bit to be desired. It's a hybrid of "ls" and "find" commands. One of
the many things I intend to do is add a "-l" flag that mimics the real ls, but
never seem to find the time in part because of nasty bugs like this. :)
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-8906.patch, HADOOP-8906.patch
>
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473461#comment-13473461
]
Colin Patrick McCabe commented on HADOOP-8906:
--
The UNIX ls command has a syntax that's more familiar:
{code}
ls -l /*
/usr:
total 276
drwxr-xr-x 2 root root 77824 Oct 9 11:14 bin
drwxr-xr-x 92 root root 20480 Sep 25 10:49 lib
drwxr-xr-x 165 root root 118784 Oct 9 11:14 lib64
...
/var:
total 56
...
{code}
But basically, you're right... this behavior is fine and expected (unlike the
behavior described in this JIRA.)
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-8906.patch
>
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473446#comment-13473446
]
Jason Lowe commented on HADOOP-8906:
bq. The current behavior does seem kind of... odd.
Agreed it does some odd things, but actually the example you gave is exactly
what I would expect since UN*X shells do the same thing. The wildcard expands
to match 'a' and then the 'a' directory is passed as an argument to the 'ls'
command.
{noformat}
$ ls ./*
ls: cannot access ./*: No such file or directory
$ mkdir -p a/b/c
$ ls ./*
b
{noformat}
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-8906.patch
>
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473412#comment-13473412
]
Colin Patrick McCabe commented on HADOOP-8906:
--
The current behavior does seem kind of... odd.
{code}
cmccabe@keter:/h> ./bin/hadoop fs -mkdir -p /a/b/c
cmccabe@keter:/h> ./bin/hadoop fs -ls '/*'
Found 1 items
drwxr-xr-x - cmccabe supergroup 0 2012-10-10 10:43 /a/b
{code}
I think we need a good unit test for this more than anything else.
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-8906.patch
>
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472818#comment-13472818 ] Daryn Sharp commented on HADOOP-8906: - It doesn't fix all cases though... There's another place. > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HADOOP-8906.patch > > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472752#comment-13472752
]
Daryn Sharp commented on HADOOP-8906:
-
This appears to be the unfortunate fallout of the change to make {{listStatus}}
throw an exception when the path doesn't exist.
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Priority: Critical
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[
https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472709#comment-13472709
]
Daryn Sharp commented on HADOOP-8906:
-
There's more multi-glob bugginess. Given {{/file}}, and {{/dir/file}}:
# {{/\*/file}} will fail with a permission denied because it tries to treat
{{/file}} as a directory since it matches the first "*".
# Worse yet, {{/\*/file\*}} will return BOTH {{/file}} and {{/dir/file}}!
> paths with multiple globs are unreliable
>
>
> Key: HADOOP-8906
> URL: https://issues.apache.org/jira/browse/HADOOP-8906
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Priority: Critical
>
> Let's say we have have a structure of "$date/$user/stuff/file". Multiple
> globs are unreliable unless every directory in the structure exists.
> These work:
> date*/user
> date*/user/stuff
> date*/user/stuff/file
> These fail:
> date*/user/*
> date*/user/*/*
> date*/user/stu*
> date*/user/stu*/*
> date*/user/stu*/file
> date*/user/stuff/*
> date*/user/stuff/f*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8906) paths with multiple globs are unreliable
[ https://issues.apache.org/jira/browse/HADOOP-8906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472696#comment-13472696 ] Daryn Sharp commented on HADOOP-8906: - It appears that after a glob, all the intervening path components before the next glob _must_ exist. > paths with multiple globs are unreliable > > > Key: HADOOP-8906 > URL: https://issues.apache.org/jira/browse/HADOOP-8906 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Priority: Critical > > Let's say we have have a structure of "$date/$user/stuff/file". Multiple > globs are unreliable unless every directory in the structure exists. > These work: > date*/user > date*/user/stuff > date*/user/stuff/file > These fail: > date*/user/* > date*/user/*/* > date*/user/stu* > date*/user/stu*/* > date*/user/stu*/file > date*/user/stuff/* > date*/user/stuff/f* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
