[jira] [Commented] (HDFS-13233) RBF：getMountPoint doesn't return the correct mount point of the mount table
[ https://issues.apache.org/jira/browse/HDFS-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389618#comment-16389618 ] wangzhiyuan commented on HDFS-13233: [~elgoiri] Thanks for your information, I think I got the issue, upload new patch > RBF：getMountPoint doesn't return the correct mount point of the mount table > --- > > Key: HDFS-13233 > URL: https://issues.apache.org/jira/browse/HDFS-13233 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Affects Versions: 3.2.0 >Reporter: wangzhiyuan >Priority: Major > Attachments: HDFS-13233.001.patch, HDFS-13233.002.patch, > HDFS-13233.003.patch > > > Method MountTableResolver->getMountPoint will traverse mount table and return > the first mount point which the input path starts with, but the condition is > not sufficient. > Suppose the mount point table contains: "/user" "/user/test" "/user/test1", > if the input path is "/user/test111", the return mount point is > "/user/test1", but the correct one should be "/user". -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13233) RBF：getMountPoint doesn't return the correct mount point of the mount table
[
https://issues.apache.org/jira/browse/HDFS-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388126#comment-16388126
]
Íñigo Goiri commented on HDFS-13233:
Thanks [~striver.wang] for [^HDFS-13233.002.patch].
It looks good, I would add a javadoc to {{isParentEntry()}} giving a high level
idea of the behavior and probably a couple examples there.
There are a couple failed unit tests that seem related:
* TestRouterQuota
* TestMountTableResolver
Do you mind taking a look?
> RBF：getMountPoint doesn't return the correct mount point of the mount table
> ---
>
> Key: HDFS-13233
> URL: https://issues.apache.org/jira/browse/HDFS-13233
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 3.2.0
>Reporter: wangzhiyuan
>Priority: Major
> Attachments: HDFS-13233.001.patch, HDFS-13233.002.patch
>
>
> Method MountTableResolver->getMountPoint will traverse mount table and return
> the first mount point which the input path starts with, but the condition is
> not sufficient.
> Suppose the mount point table contains: "/user" "/user/test" "/user/test1",
> if the input path is "/user/test111", the return mount point is
> "/user/test1", but the correct one should be "/user".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13233) RBF：getMountPoint doesn't return the correct mount point of the mount table
[
https://issues.apache.org/jira/browse/HDFS-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387510#comment-16387510
]
genericqa commented on HDFS-13233:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 31s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 35s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}146m 22s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}193m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure |
| | hadoop.hdfs.server.federation.router.TestRouterQuota |
| | hadoop.hdfs.server.federation.resolver.TestMountTableResolver |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-13233 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913158/HDFS-13233.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 2f431a0c5706 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 745190e |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/23316/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/23316/testReport/ |
| Max. process+thread count | 3505 (vs. ulimit of 1) |
| modules | C:
[jira] [Commented] (HDFS-13233) RBF：getMountPoint doesn't return the correct mount point of the mount table
[ https://issues.apache.org/jira/browse/HDFS-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387345#comment-16387345 ] maobaolong commented on HDFS-13233: --- [~striver.wang] make sense. > RBF：getMountPoint doesn't return the correct mount point of the mount table > --- > > Key: HDFS-13233 > URL: https://issues.apache.org/jira/browse/HDFS-13233 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Affects Versions: 3.2.0 >Reporter: wangzhiyuan >Priority: Major > Attachments: HDFS-13233.001.patch, HDFS-13233.002.patch > > > Method MountTableResolver->getMountPoint will traverse mount table and return > the first mount point which the input path starts with, but the condition is > not sufficient. > Suppose the mount point table contains: "/user" "/user/test" "/user/test1", > if the input path is "/user/test111", the return mount point is > "/user/test1", but the correct one should be "/user". -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13233) RBF：getMountPoint doesn't return the correct mount point of the mount table
[
https://issues.apache.org/jira/browse/HDFS-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387342#comment-16387342
]
wangzhiyuan commented on HDFS-13233:
[~elgoiri] New patch has been uploaded.
Actually I think below function is OK:
private boolean isParentEntry(final String path, final String parent) { if
(!path.startsWith(parent)) {return false;
} if (path.equals(parent)) {return true;
} return path.charAt(parent.length()) == '/';
}
path.startsWith(parent) could guarantee path.length() >= parent.length()
!path.equals(parent) could guarantee path.length() != parent.length()
So path.charAt(parent.length()) is safe here.
> RBF：getMountPoint doesn't return the correct mount point of the mount table
> ---
>
> Key: HDFS-13233
> URL: https://issues.apache.org/jira/browse/HDFS-13233
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 3.2.0
>Reporter: wangzhiyuan
>Priority: Major
> Attachments: HDFS-13233.001.patch, HDFS-13233.002.patch
>
>
> Method MountTableResolver->getMountPoint will traverse mount table and return
> the first mount point which the input path starts with, but the condition is
> not sufficient.
> Suppose the mount point table contains: "/user" "/user/test" "/user/test1",
> if the input path is "/user/test111", the return mount point is
> "/user/test1", but the correct one should be "/user".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13233) RBF：getMountPoint doesn't return the correct mount point of the mount table
[
https://issues.apache.org/jira/browse/HDFS-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387322#comment-16387322
]
genericqa commented on HDFS-13233:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 44s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 2s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}129m 39s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}178m 14s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.server.federation.router.TestRouterQuota |
| | hadoop.hdfs.TestCrcCorruption |
| | hadoop.hdfs.server.federation.resolver.TestMountTableResolver |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-13233 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913137/HDFS-13233.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 8c4ef5344e7b 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 745190e |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/23310/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
[jira] [Commented] (HDFS-13233) RBF：getMountPoint doesn't return the correct mount point of the mount table
[ https://issues.apache.org/jira/browse/HDFS-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387221#comment-16387221 ] wangzhiyuan commented on HDFS-13233: [~elgoiri] Thanks for your quick response, will fix it as your require. > RBF：getMountPoint doesn't return the correct mount point of the mount table > --- > > Key: HDFS-13233 > URL: https://issues.apache.org/jira/browse/HDFS-13233 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Affects Versions: 3.2.0 >Reporter: wangzhiyuan >Priority: Major > Attachments: HDFS-13233.001.patch > > > Method MountTableResolver->getMountPoint will traverse mount table and return > the first mount point which the input path starts with, but the condition is > not sufficient. > Suppose the mount point table contains: "/user" "/user/test" "/user/test1", > if the input path is "/user/test111", the return mount point is > "/user/test1", but the correct one should be "/user". -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13233) RBF：getMountPoint doesn't return the correct mount point of the mount table
[
https://issues.apache.org/jira/browse/HDFS-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387202#comment-16387202
]
Íñigo Goiri commented on HDFS-13233:
Thanks [~striver.wang] for the report and the fix.
I would probably make the full condition a private function for readability.
Please double check as I haven't gone through it carefully but I would do
something like:
{code}
while (entry != null && isParentEntry(path, entry.getKey())) {
...
private boolean isParentEntry(final String path, final String parent) {
if (!path.startsWith(parent)) {
return false;
}
if (path.equals(parent)) {
return true;
}
return path.charAt(entry.getKey().length()) == '/';
}
{code}
I would also add checks to avoid having the catch for
{{StringIndexOutOfBoundsException}}.
> RBF：getMountPoint doesn't return the correct mount point of the mount table
> ---
>
> Key: HDFS-13233
> URL: https://issues.apache.org/jira/browse/HDFS-13233
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 3.2.0
>Reporter: wangzhiyuan
>Priority: Major
> Attachments: HDFS-13233.001.patch
>
>
> Method MountTableResolver->getMountPoint will traverse mount table and return
> the first mount point which the input path starts with, but the condition is
> not sufficient.
> Suppose the mount point table contains: "/user" "/user/test" "/user/test1",
> if the input path is "/user/test111", the return mount point is
> "/user/test1", but the correct one should be "/user".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
