[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=769821=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769821
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 12/May/22 18:59
Start Date: 12/May/22 18:59
Worklog Time Spent: 10m
Work Description: virajith commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1125324697
@omalley - looks like this was merged but PR is marked as closed. I believe
the convention is to "squash merge" the PR and the PR will be marked as
"Merged". Otherwise, it's confusing if the PR has been rejected or not.
Issue Time Tracking
---
Worklog Id: (was: 769821)
Time Spent: 7h 40m (was: 7.5h)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=769421=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769421
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 12/May/22 05:06
Start Date: 12/May/22 05:06
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1124533860
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 4s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 11s | | trunk passed |
| +1 :green_heart: | compile | 23m 13s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 44s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 28s | | trunk passed |
| -1 :x: | javadoc | 1m 51s |
[/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/19/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-common in trunk failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 4m 23s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 11s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 31s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 27s | | the patch passed |
| +1 :green_heart: | compile | 22m 38s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 22m 38s | | the patch passed |
| +1 :green_heart: | compile | 20m 44s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 44s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 19s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/19/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 71 new + 159 unchanged - 3 fixed = 230 total (was
162) |
| +1 :green_heart: | mvnsite | 4m 28s | | the patch passed |
| -1 :x: | javadoc | 1m 38s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/19/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-common in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 4m 35s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 37s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 39s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 451m 11s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/19/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 59s | | The patch does not
generate ASF License warnings. |
| | | 702m 34s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
| |
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=769419=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769419
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 12/May/22 04:42
Start Date: 12/May/22 04:42
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1124523817
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 55s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 42s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 15s | | trunk passed |
| +1 :green_heart: | compile | 25m 7s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 21m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 57s | | trunk passed |
| -1 :x: | javadoc | 1m 36s |
[/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/20/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-common in trunk failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 4m 4s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 50s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 28s | | the patch passed |
| +1 :green_heart: | compile | 24m 15s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 24m 15s | | the patch passed |
| +1 :green_heart: | compile | 21m 47s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 47s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 21s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/20/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 66 new + 159 unchanged - 3 fixed = 225 total (was
162) |
| +1 :green_heart: | mvnsite | 3m 57s | | the patch passed |
| -1 :x: | javadoc | 1m 28s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/20/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-common in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 4m 1s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 27m 2s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 20s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 357m 44s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/20/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 33s | | The patch does not
generate ASF License warnings. |
| | | 616m 58s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestClientProtocolForPipelineRecovery |
| Subsystem | Report/Notes |
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=769363=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769363
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 12/May/22 00:01
Start Date: 12/May/22 00:01
Worklog Time Spent: 10m
Work Description: omalley closed pull request #4181: HADOOP-18193:Support
nested mount points in INodeTree
URL: https://github.com/apache/hadoop/pull/4181
Issue Time Tracking
---
Worklog Id: (was: 769363)
Time Spent: 7h 10m (was: 7h)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=769313=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769313
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 21:20
Start Date: 11/May/22 21:20
Worklog Time Spent: 10m
Work Description: li-leyang commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1124306105
> This is looking good, thanks, Lei.
>
> Can you add a test case that looks like: /a /a/b/c/d /a/b/c/d/e/f/g
>
> I'd like a test that makes sure that multiple levels work correctly.
I have added multi-level nested mount point like below in the
TestNestedMountPoint#setup section so it should be covered, unless you mean
something else.
https://github.com/apache/hadoop/pull/4181/files#diff-bf87c782e721db09e85aa8a87b0c0ed6ea05bca2a279abb628d218c63353650aR73
/a/b
/a/b/c/d
/a/b/c/d/e
Issue Time Tracking
---
Worklog Id: (was: 769313)
Time Spent: 7h (was: 6h 50m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 7h
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=769246=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769246
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 19:06
Start Date: 11/May/22 19:06
Worklog Time Spent: 10m
Work Description: li-leyang commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1124187886
> I'd suggest adding getLink to Inode. See
https://github.com/omalley/hadoop/tree/hadoop-18193 or more precisely
[omalley@6733b57](https://github.com/omalley/hadoop/commit/6733b5785252bc40eed1d323a91144253b7e8fe4)
Changed per your suggestion
Issue Time Tracking
---
Worklog Id: (was: 769246)
Time Spent: 6h 50m (was: 6h 40m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=769209=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769209
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 17:26
Start Date: 11/May/22 17:26
Worklog Time Spent: 10m
Work Description: li-leyang commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r870572457
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -682,6 +748,32 @@ protected InodeTree(final Configuration config, final
String viewName,
}
}
+ /**
Review Comment:
i have changed this to always sort
Issue Time Tracking
---
Worklog Id: (was: 769209)
Time Spent: 6h 40m (was: 6.5h)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=769208=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769208
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 17:23
Start Date: 11/May/22 17:23
Worklog Time Spent: 10m
Work Description: li-leyang commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r870570022
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -133,6 +138,11 @@ public INode(String pathToNode, UserGroupInformation aUgi)
{
// and is read only.
abstract boolean isInternalDir();
+/**
+ * INode representing a INodeDir which also contains a INodeLink(nested
mount point)
+ */
+abstract boolean isDirAndLink();
+
// INode linking to another filesystem. Represented
// via mount table link config entries.
boolean isLink() {
Review Comment:
Changed.
Issue Time Tracking
---
Worklog Id: (was: 769208)
Time Spent: 6.5h (was: 6h 20m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=769198=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769198
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 16:59
Start Date: 11/May/22 16:59
Worklog Time Spent: 10m
Work Description: omalley commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r870549228
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -133,6 +138,11 @@ public INode(String pathToNode, UserGroupInformation aUgi)
{
// and is read only.
abstract boolean isInternalDir();
+/**
+ * INode representing a INodeDir which also contains a INodeLink(nested
mount point)
+ */
+abstract boolean isDirAndLink();
+
// INode linking to another filesystem. Represented
// via mount table link config entries.
boolean isLink() {
Review Comment:
Basically, it looks like half of the 6 callers to isLink should change to
!isInternalDir.
Issue Time Tracking
---
Worklog Id: (was: 769198)
Time Spent: 6h 20m (was: 6h 10m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=769196=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769196
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 16:54
Start Date: 11/May/22 16:54
Worklog Time Spent: 10m
Work Description: omalley commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r870545469
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -133,6 +138,11 @@ public INode(String pathToNode, UserGroupInformation aUgi)
{
// and is read only.
abstract boolean isInternalDir();
+/**
+ * INode representing a INodeDir which also contains a INodeLink(nested
mount point)
+ */
+abstract boolean isDirAndLink();
+
// INode linking to another filesystem. Represented
// via mount table link config entries.
boolean isLink() {
Review Comment:
You already have whether it is an internal node. Change the traversal code
to use that rather than isLink. I understand that the new method lets you
functionally accomplish your goal, but the abstraction is wrong. You could get
there by renaming the methods, but that isn't as satisfying as correcting this
patch to define isLink correctly.
Issue Time Tracking
---
Worklog Id: (was: 769196)
Time Spent: 6h 10m (was: 6h)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=769191=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769191
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 16:51
Start Date: 11/May/22 16:51
Worklog Time Spent: 10m
Work Description: omalley commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r870542414
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -682,6 +748,32 @@ protected InodeTree(final Configuration config, final
String viewName,
}
}
+ /**
Review Comment:
I would argue that the simplicity of always sorting is better than the
O(mount*log(mount)) cost that happens once when the file system is created. In
fact, you at that point could move the flag into the constructor.
Issue Time Tracking
---
Worklog Id: (was: 769191)
Time Spent: 6h (was: 5h 50m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 6h
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768959=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768959
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 09:57
Start Date: 11/May/22 09:57
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1123477914
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 20s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 31s | | trunk passed |
| +1 :green_heart: | compile | 24m 47s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 21m 33s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 3m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 6s | | trunk passed |
| -1 :x: | javadoc | 1m 36s |
[/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/18/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-common in trunk failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 4m 28s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 49s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 1m 1s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 30s | | the patch passed |
| +1 :green_heart: | compile | 22m 35s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 22m 35s | | the patch passed |
| +1 :green_heart: | compile | 20m 50s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 50s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 15s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/18/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 73 new + 159 unchanged - 3 fixed = 232 total (was
162) |
| +1 :green_heart: | mvnsite | 4m 23s | | the patch passed |
| -1 :x: | javadoc | 1m 44s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/18/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-common in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 4m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 51s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 36s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 464m 42s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/18/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 45s | | The patch does not
generate ASF License warnings. |
| | | 717m 46s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
| Subsystem | Report/Notes |
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768799=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768799
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 01:17
Start Date: 11/May/22 01:17
Worklog Time Spent: 10m
Work Description: li-leyang commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869785802
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -133,6 +138,11 @@ public INode(String pathToNode, UserGroupInformation aUgi)
{
// and is read only.
abstract boolean isInternalDir();
+/**
+ * INode representing a INodeDir which also contains a INodeLink(nested
mount point)
+ */
+abstract boolean isDirAndLink();
+
// INode linking to another filesystem. Represented
// via mount table link config entries.
boolean isLink() {
Review Comment:
> In particular, I'd say that isLink should return true on all links
(internal & leaf)
Yeah, I've considered this but this involves refactoring in
traversal/resolve logic and also change the semantics of isLink() which returns
true for an internal node. (I am more concerned of this from traversal
perspective but i agree it is contradictory from definition perspective) So I
decided to create a new prop to identify internal node with mount point. But
the downside is I have to add additional check whenever x.isLink() is invoked.
Issue Time Tracking
---
Worklog Id: (was: 768799)
Time Spent: 5h 40m (was: 5.5h)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768798=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768798
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 01:17
Start Date: 11/May/22 01:17
Worklog Time Spent: 10m
Work Description: li-leyang commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869790694
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ConfigUtil.java:
##
@@ -247,4 +247,22 @@ public static String getDefaultMountTableName(final
Configuration conf) {
return conf.get(Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE_NAME_KEY,
Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE);
}
+
+ /**
+ * Check the bool config whether nested mount point is supported.
+ * @param conf - from this conf
+ * @return whether nested mount point is supported
+ */
+ public static boolean isNestedMountPointSupported(final Configuration conf) {
+return conf.getBoolean(Constants.CONFIG_NESTED_MOUNT_POINT_SUPPORTED,
false);
Review Comment:
this is fair
Issue Time Tracking
---
Worklog Id: (was: 768798)
Time Spent: 5.5h (was: 5h 20m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768795=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768795
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 01:02
Start Date: 11/May/22 01:02
Worklog Time Spent: 10m
Work Description: li-leyang commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869785802
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -133,6 +138,11 @@ public INode(String pathToNode, UserGroupInformation aUgi)
{
// and is read only.
abstract boolean isInternalDir();
+/**
+ * INode representing a INodeDir which also contains a INodeLink(nested
mount point)
+ */
+abstract boolean isDirAndLink();
+
// INode linking to another filesystem. Represented
// via mount table link config entries.
boolean isLink() {
Review Comment:
> In particular, I'd say that isLink should return true on all links
(internal & leaf)
Yeah, I've considered this but this involves refactoring in
traversal/resolve logic and also change the semantics of isLink() which returns
true for an internal node. (I am more concerned of this from traversal
perspective but it might be contradictory from definition perspective) So I
decided to create a new prop to identify internal node with mount point. But
the downside is I have to add additional check whenever x.isLink() is invoked.
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -133,6 +138,11 @@ public INode(String pathToNode, UserGroupInformation aUgi)
{
// and is read only.
abstract boolean isInternalDir();
+/**
+ * INode representing a INodeDir which also contains a INodeLink(nested
mount point)
+ */
+abstract boolean isDirAndLink();
+
// INode linking to another filesystem. Represented
// via mount table link config entries.
boolean isLink() {
Review Comment:
> In particular, I'd say that isLink should return true on all links
(internal & leaf)
Yeah, I've considered this but this involves refactoring in
traversal/resolve logic and also change the semantics of isLink() which returns
true for an internal node. (I am more concerned of this from traversal
perspective but it might be contradictory from definition perspective) So I
decided to create a new prop to identify internal node with mount point. But
the downside is I have to add additional check whenever x.isLink() is invoked.
Issue Time Tracking
---
Worklog Id: (was: 768795)
Time Spent: 5h 20m (was: 5h 10m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768794=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768794
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 01:01
Start Date: 11/May/22 01:01
Worklog Time Spent: 10m
Work Description: li-leyang commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869785759
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -682,6 +748,32 @@ protected InodeTree(final Configuration config, final
String viewName,
}
}
+ /**
Review Comment:
Because sorting changes the existing implementation if nested mount point is
disabled(current state) and is also not required with more overhead.
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -133,6 +138,11 @@ public INode(String pathToNode, UserGroupInformation aUgi)
{
// and is read only.
abstract boolean isInternalDir();
+/**
+ * INode representing a INodeDir which also contains a INodeLink(nested
mount point)
+ */
+abstract boolean isDirAndLink();
+
// INode linking to another filesystem. Represented
// via mount table link config entries.
boolean isLink() {
Review Comment:
In particular, I'd say that isLink should return true on all links
(internal & leaf)
> Yeah, I've considered this but this involves refactoring in
traversal/resolve logic and also change the semantics of isLink() which returns
true for an internal node. (I am more concerned of this from traversal
perspective but it might be contradictory from definition perspective) So I
decided to create a new prop to identify internal node with mount point. But
the downside is I have to add additional check whenever x.isLink() is invoked.
Issue Time Tracking
---
Worklog Id: (was: 768794)
Time Spent: 5h 10m (was: 5h)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768791=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768791
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 11/May/22 00:24
Start Date: 11/May/22 00:24
Worklog Time Spent: 10m
Work Description: omalley commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869773149
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -682,6 +748,32 @@ protected InodeTree(final Configuration config, final
String viewName,
}
}
+ /**
Review Comment:
Why wouldn't you always sort the mount points? It seems like a useful
invariant.
Issue Time Tracking
---
Worklog Id: (was: 768791)
Time Spent: 5h (was: 4h 50m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768774=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768774
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 10/May/22 23:52
Start Date: 10/May/22 23:52
Worklog Time Spent: 10m
Work Description: omalley commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869761867
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -159,6 +169,11 @@ boolean isInternalDir() {
return true;
}
+@Override
Review Comment:
Given the other suggestions, I don't think you'd need this.
Issue Time Tracking
---
Worklog Id: (was: 768774)
Time Spent: 4h 50m (was: 4h 40m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768773=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768773
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 10/May/22 23:52
Start Date: 10/May/22 23:52
Worklog Time Spent: 10m
Work Description: omalley commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869761670
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -133,6 +138,11 @@ public INode(String pathToNode, UserGroupInformation aUgi)
{
// and is read only.
abstract boolean isInternalDir();
+/**
+ * INode representing a INodeDir which also contains a INodeLink(nested
mount point)
+ */
+abstract boolean isDirAndLink();
+
// INode linking to another filesystem. Represented
// via mount table link config entries.
boolean isLink() {
Review Comment:
This feels wrong. In particular, I'd say that isLink should return true on
all links (internal & leaf). This shows as you almost always call
x.isDirAndLink() || x.isLink(). Note that the definition would need to change
from the current !isInternalDir(). Obviously isInternalDir can be used to
determine if it is an internal node versus a leaf.
Issue Time Tracking
---
Worklog Id: (was: 768773)
Time Spent: 4h 40m (was: 4.5h)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768772=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768772
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 10/May/22 23:44
Start Date: 10/May/22 23:44
Worklog Time Spent: 10m
Work Description: omalley commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869734237
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ConfigUtil.java:
##
@@ -247,4 +247,22 @@ public static String getDefaultMountTableName(final
Configuration conf) {
return conf.get(Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE_NAME_KEY,
Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE);
}
+
+ /**
+ * Check the bool config whether nested mount point is supported.
+ * @param conf - from this conf
+ * @return whether nested mount point is supported
+ */
+ public static boolean isNestedMountPointSupported(final Configuration conf) {
+return conf.getBoolean(Constants.CONFIG_NESTED_MOUNT_POINT_SUPPORTED,
false);
Review Comment:
I think that the default should be true here, because this feature should be
enabled by default.
Issue Time Tracking
---
Worklog Id: (was: 768772)
Time Spent: 4.5h (was: 4h 20m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768742=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768742
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 10/May/22 22:47
Start Date: 10/May/22 22:47
Worklog Time Spent: 10m
Work Description: omalley commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869734237
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ConfigUtil.java:
##
@@ -247,4 +247,22 @@ public static String getDefaultMountTableName(final
Configuration conf) {
return conf.get(Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE_NAME_KEY,
Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE);
}
+
+ /**
+ * Check the bool config whether nested mount point is supported.
+ * @param conf - from this conf
+ * @return whether nested mount point is supported
+ */
+ public static boolean isNestedMountPointSupported(final Configuration conf) {
+return conf.getBoolean(Constants.CONFIG_NESTED_MOUNT_POINT_SUPPORTED,
false);
Review Comment:
I think that the default should be true here.
Issue Time Tracking
---
Worklog Id: (was: 768742)
Time Spent: 4h 20m (was: 4h 10m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768740=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768740
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 10/May/22 22:01
Start Date: 10/May/22 22:01
Worklog Time Spent: 10m
Work Description: li-leyang commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869712470
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java:
##
@@ -167,8 +167,21 @@ void setupMountPoints() {
new Path(targetTestRoot, "missingTarget").toUri());
ConfigUtil.addLink(conf, "/linkToAFile",
new Path(targetTestRoot, "aFile").toUri());
+
+// Enable nested mount point, ViewFilesystem should support both
non-nested and nested mount points
+ConfigUtil.setIsNestedMountPointSupported(conf, true);
+ConfigUtil.addLink(conf, "/user/userA",
+new Path(targetTestRoot, "user").toUri());
+ConfigUtil.addLink(conf, "/user/userB",
+new Path(targetTestRoot, "userB").toUri());
+ConfigUtil.addLink(conf, "/data/dataA",
+new Path(targetTestRoot, "dataA").toUri());
+ConfigUtil.addLink(conf, "/data/dataB",
+new Path(targetTestRoot, "user").toUri());
+ConfigUtil.addLink(conf, "/internalDir/linkToDir2/linkToDir2",
+new Path(targetTestRoot,"linkToDir2").toUri());
Review Comment:
I have moved nested mount point tests to ViewFileSystemBaseTest
Issue Time Tracking
---
Worklog Id: (was: 768740)
Time Spent: 4h 10m (was: 4h)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768637=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768637
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 10/May/22 18:31
Start Date: 10/May/22 18:31
Worklog Time Spent: 10m
Work Description: li-leyang commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869561085
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -682,6 +736,25 @@ protected InodeTree(final Configuration config, final
String viewName,
}
}
+ private Collection getLinkEntries(List linkEntries) {
+if (isNestedMountPointSupported) {
+ Set sortedLinkEntries = new TreeSet<>((o1, o2) -> {
+if (o1 == null) {
+ return -1;
+}
+if (o2 == null) {
+ return 1;
+}
+String src1 = o1.getSrc().toLowerCase();
Review Comment:
Good catch, changed.
Issue Time Tracking
---
Worklog Id: (was: 768637)
Time Spent: 4h (was: 3h 50m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 4h
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768628=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768628
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 10/May/22 18:08
Start Date: 10/May/22 18:08
Worklog Time Spent: 10m
Work Description: li-leyang commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869542966
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java:
##
@@ -167,8 +167,21 @@ void setupMountPoints() {
new Path(targetTestRoot, "missingTarget").toUri());
ConfigUtil.addLink(conf, "/linkToAFile",
new Path(targetTestRoot, "aFile").toUri());
+
+// Enable nested mount point, ViewFilesystem should support both
non-nested and nested mount points
+ConfigUtil.setIsNestedMountPointSupported(conf, true);
+ConfigUtil.addLink(conf, "/user/userA",
+new Path(targetTestRoot, "user").toUri());
+ConfigUtil.addLink(conf, "/user/userB",
+new Path(targetTestRoot, "userB").toUri());
+ConfigUtil.addLink(conf, "/data/dataA",
+new Path(targetTestRoot, "dataA").toUri());
+ConfigUtil.addLink(conf, "/data/dataB",
+new Path(targetTestRoot, "user").toUri());
+ConfigUtil.addLink(conf, "/internalDir/linkToDir2/linkToDir2",
+new Path(targetTestRoot,"linkToDir2").toUri());
Review Comment:
test
Issue Time Tracking
---
Worklog Id: (was: 768628)
Time Spent: 3h 50m (was: 3h 40m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768627=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768627
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 10/May/22 18:07
Start Date: 10/May/22 18:07
Worklog Time Spent: 10m
Work Description: li-leyang commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r852343467
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestNestedMountPoint.java:
##
@@ -0,0 +1,345 @@
+package org.apache.hadoop.fs.viewfs;
+
+import java.net.URI;
+import java.util.List;
+import java.util.function.Function;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.FsConstants;
+import org.apache.hadoop.fs.Path;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+
+public class TestNestedMountPoint {
+ private InodeTree inodeTree;
+ private Configuration conf;
+ private String mtName;
+ private URI fsUri;
+
+ static class TestNestMountPointFileSystem {
+public URI getUri() {
+ return uri;
+}
+
+private URI uri;
+
+TestNestMountPointFileSystem(URI uri) {
+ this.uri = uri;
+}
+ }
+
+ static class TestNestMountPointInternalFileSystem extends
TestNestMountPointFileSystem {
+TestNestMountPointInternalFileSystem(URI uri) {
+ super(uri);
+}
+ }
+
+ private static final URI LINKFALLBACK_TARGET = URI.create("hdfs://nn00");
+ private static final URI NN1_TARGET = URI.create("hdfs://nn01/a/b");
+ private static final URI NN2_TARGET = URI.create("hdfs://nn02/a/b/e");
+ private static final URI NN3_TARGET = URI.create("hdfs://nn03/a/b/c/d");
+ private static final URI NN4_TARGET = URI.create("hdfs://nn04/a/b/c/d/e");
+ private static final URI NN5_TARGET = URI.create("hdfs://nn05/b/c/d/e");
+ private static final URI NN6_TARGET = URI.create("hdfs://nn06/b/c/d/e/f");
+
+ @Before
+ public void setUp() throws Exception {
+conf = new Configuration();
+mtName = TestNestedMountPoint.class.getName();
+ConfigUtil.setIsNestedMountPointSupported(conf, true);
+ConfigUtil.addLink(conf, mtName, "/a/b", NN1_TARGET);
+ConfigUtil.addLink(conf, mtName, "/a/b/e", NN2_TARGET);
Review Comment:
Not sure what test case you are referring to
/a/b is nested in /a/b/c/d which is nested in /a/b/c/d/e.
Do you mean to add the test case like below:
/a/b -> /a/b/c/d ->/a/b/c/d/e
/a/b -> /a/b/e -> /a/b/e/f
(/a/b is a dirlink has has two nested mount points)
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ConfigUtil.java:
##
@@ -247,4 +247,22 @@ public static String getDefaultMountTableName(final
Configuration conf) {
return conf.get(Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE_NAME_KEY,
Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE);
}
+
+ /**
+ * Get the bool config whether nested mount point is supported.
Review Comment:
this is to get config value from conf object. I will change it to "check".
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -133,6 +138,9 @@ public INode(String pathToNode, UserGroupInformation aUgi) {
// and is read only.
abstract boolean isInternalDir();
+// Identity some type of inodes(nested mount point).
Review Comment:
yeah, will expand this comment
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -212,6 +225,33 @@ void addLink(final String pathComponent, final
INodeLink link)
}
children.put(pathComponent, link);
}
+
+void addDirLink(final String pathComponent, final INodeDirLink dirLink)
{
+ children.put(pathComponent, dirLink);
+}
+ }
+
+ /**
+ * Internal class to represent nested mount points in INodeTree. Nested
mount points are non-leaf nodes in INode tree.
Review Comment:
I have added in the comments.
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -682,6 +736,25 @@ protected InodeTree(final Configuration config, final
String viewName,
}
}
+ private Collection getLinkEntries(List linkEntries) {
+if (isNestedMountPointSupported) {
Review Comment:
The main reason for sorting is to group nested mp together and create
INodeDirLink for nested mp:
Lets say we have 3 mount points sorted:
/foo, /foo/bar and /foo/bar/baz
When creating inode tree:
1st pass: /foo -> create INodeLink for /foo
2nd pass: /foo/bar -> /foo is already a INodeLink so /foo is nested,
updating /foo as INodeDirLink, creating /foo/bar as INodeLink
3nd pass:
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768624=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768624
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 10/May/22 18:05
Start Date: 10/May/22 18:05
Worklog Time Spent: 10m
Work Description: virajith commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r869540140
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java:
##
@@ -167,8 +167,21 @@ void setupMountPoints() {
new Path(targetTestRoot, "missingTarget").toUri());
ConfigUtil.addLink(conf, "/linkToAFile",
new Path(targetTestRoot, "aFile").toUri());
+
+// Enable nested mount point, ViewFilesystem should support both
non-nested and nested mount points
+ConfigUtil.setIsNestedMountPointSupported(conf, true);
+ConfigUtil.addLink(conf, "/user/userA",
+new Path(targetTestRoot, "user").toUri());
+ConfigUtil.addLink(conf, "/user/userB",
+new Path(targetTestRoot, "userB").toUri());
+ConfigUtil.addLink(conf, "/data/dataA",
+new Path(targetTestRoot, "dataA").toUri());
+ConfigUtil.addLink(conf, "/data/dataB",
+new Path(targetTestRoot, "user").toUri());
+ConfigUtil.addLink(conf, "/internalDir/linkToDir2/linkToDir2",
+new Path(targetTestRoot,"linkToDir2").toUri());
Review Comment:
test
Issue Time Tracking
---
Worklog Id: (was: 768624)
Time Spent: 3.5h (was: 3h 20m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: Nested Mount Point in ViewFs.pdf
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
>
> {code:java}
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
> {code}
> INodeTree has 2 methods that need change to support nested mount points.
> {code:java}
> createLink(): build INodeTree during fs init.
> resolve(): resolve path in INodeTree with viewfs apis.
> {code}
> ViewFileSystem and ViewFs maintains an INodeTree instance(fsState) in both
> classes and call fsState.resolve(..) to resolve path to specific mount point.
> INodeTree.resolve encapsulates the logic of nested mount point resolving. So
> no changes are expected in both classes.
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
>
> Spec:
> Please review attached pdf for spec about this feature.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=768328=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768328
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 10/May/22 06:55
Start Date: 10/May/22 06:55
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1122000626
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 3s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 7s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 6s | | trunk passed |
| +1 :green_heart: | compile | 23m 16s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 49s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 27s | | trunk passed |
| -1 :x: | javadoc | 1m 53s |
[/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/17/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-common in trunk failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 4m 28s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 11s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 37s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 31s | | the patch passed |
| +1 :green_heart: | compile | 22m 27s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 22m 27s | | the patch passed |
| +1 :green_heart: | compile | 20m 39s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 39s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 13s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/17/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 66 new + 108 unchanged - 3 fixed = 174 total (was
111) |
| +1 :green_heart: | mvnsite | 4m 24s | | the patch passed |
| -1 :x: | javadoc | 1m 36s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/17/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-common in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 4m 33s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 22s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 50s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 32s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 470m 13s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/17/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 43s | | The patch does not
generate ASF License warnings. |
| | | 721m 32s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
| | hadoop.hdfs.TestReplaceDatanodeFailureReplication
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=767029=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-767029
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 06/May/22 06:15
Start Date: 06/May/22 06:15
Worklog Time Spent: 10m
Work Description: virajith commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r866488045
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -212,6 +227,43 @@ void addLink(final String pathComponent, final
INodeLink link)
}
children.put(pathComponent, link);
}
+
+void addDirLink(final String pathComponent, final INodeDirLink dirLink)
{
+ children.put(pathComponent, dirLink);
+}
+ }
+
+ /**
+ * Internal class to represent a INodeDir which also contains a INodeLink.
This is used to support nested mount point
Review Comment:
nit:
"support nested mount point where a INode is internalDir but would also
point to a mount link" ->
"support nested mount points where an INode is an internalDir and points to
a mount link."
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -682,6 +748,32 @@ protected InodeTree(final Configuration config, final
String viewName,
}
}
+ /**
+ * Get collection of linkEntry. If nested mount point is supported, sort
mount point src path based on alphabetical order.
+ * The purpose is to group nested path(shortest path always comes first)
during INodeTree creation.
Review Comment:
nit: "nested path(shortest" -> "nested paths (shortest"
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -682,6 +748,32 @@ protected InodeTree(final Configuration config, final
String viewName,
}
}
+ /**
+ * Get collection of linkEntry. If nested mount point is supported, sort
mount point src path based on alphabetical order.
Review Comment:
nit: "sort mount point src path based on alphabetical order." -> "sort mount
points based on alphabetical order of the src paths"
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java:
##
@@ -460,30 +502,116 @@ public void testRenameAcrossMounts4() throws IOException
{
.assertIsFile(fsTarget, new Path(targetTestRoot, "data/fooBar"));
}
+ // rename across nested mount points that point to same target also fail
+ @Test
+ public void testRenameAcrossNestedMountPointSameTarget() throws IOException {
+fileSystemTestHelper.createFile(fsView, "/user/foo");
+try {
+ // Nested mount points point to the same target should fail
+ // /user -> /user
+ // /user/userA -> /user
+ // Rename strategy: SAME_MOUNTPOINT
+ fsView.rename(new Path("/user/foo"), new Path("/user/userA/foo"));
+ ContractTestUtils.fail("IOException is not thrown on rename operation");
+} catch (IOException e) {
+ GenericTestUtils
+ .assertExceptionContains("Renames across Mount points not supported",
+ e);
+}
+ }
+
+
+ // rename across nested mount points fail if the mount link targets are
different
+ // even if the targets are part of the same target FS
+ @Test
+ public void testRenameAcrossMountPointDifferentTarget() throws IOException {
+fileSystemTestHelper.createFile(fsView, "/data/foo");
+// /data -> /data
+// /data/dataA -> /dataA
+// Rename strategy: SAME_MOUNTPOINT
+try {
+ fsView.rename(new Path("/data/foo"), new Path("/data/dataA/fooBar"));
+ ContractTestUtils.fail("IOException is not thrown on rename operation");
+} catch (IOException e) {
+ GenericTestUtils
+ .assertExceptionContains("Renames across Mount points not supported",
+ e);
+}
+ }
+
+ // RenameStrategy SAME_TARGET_URI_ACROSS_MOUNTPOINT enabled
+ // to rename across nested mount points that point to same target URI
+ @Test
+ public void testRenameAcrossNestedMountPointSameTargetUriAcrossMountPoint()
throws IOException {
+// /user/foo -> /user
+// /user/userA/fooBarBar -> /user
+// Rename strategy: SAME_TARGET_URI_ACROSS_MOUNTPOINT
+Configuration conf2 = new Configuration(conf);
+conf2.set(Constants.CONFIG_VIEWFS_RENAME_STRATEGY,
+ViewFileSystem.RenameStrategy.SAME_TARGET_URI_ACROSS_MOUNTPOINT
+.toString());
+FileSystem fsView2 = FileSystem.newInstance(FsConstants.VIEWFS_URI, conf2);
+fileSystemTestHelper.createFile(fsView2, "/user/foo");
+fsView2.rename(new Path("/user/foo"), new Path("/user/userA/fooBarBar"));
+ContractTestUtils.assertPathDoesNotExist(fsView2, "src should not exist
after rename",
+new
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=765749=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-765749
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 03/May/22 23:55
Start Date: 03/May/22 23:55
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1116769957
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 53s | | trunk passed |
| +1 :green_heart: | compile | 23m 27s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 48s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 13s | | trunk passed |
| -1 :x: | javadoc | 1m 53s |
[/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/15/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-common in trunk failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 2m 14s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 6s | | the patch passed |
| +1 :green_heart: | compile | 22m 40s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 22m 40s | |
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 1794 unchanged - 19
fixed = 1794 total (was 1813) |
| +1 :green_heart: | compile | 20m 44s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 44s | |
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 1671 unchanged -
19 fixed = 1671 total (was 1690) |
| -1 :x: | blanks | 0m 1s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/15/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 42s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/15/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 73 new + 150
unchanged - 12 fixed = 223 total (was 162) |
| +1 :green_heart: | mvnsite | 2m 9s | | the patch passed |
| -1 :x: | javadoc | 1m 42s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/15/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-common in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 2m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 33s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 42s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 36s | | The patch does not
generate ASF License warnings. |
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=759682=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759682
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 21/Apr/22 00:57
Start Date: 21/Apr/22 00:57
Worklog Time Spent: 10m
Work Description: virajith commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r854653058
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ConfigUtil.java:
##
@@ -247,4 +247,22 @@ public static String getDefaultMountTableName(final
Configuration conf) {
return conf.get(Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE_NAME_KEY,
Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE);
}
+
+ /**
+ * Get the bool config whether nested mount point is supported.
Review Comment:
Nit - "Get the bool config" -> "Check" - you aren't really getting a config
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -212,6 +225,33 @@ void addLink(final String pathComponent, final
INodeLink link)
}
children.put(pathComponent, link);
}
+
+void addDirLink(final String pathComponent, final INodeDirLink dirLink)
{
+ children.put(pathComponent, dirLink);
+}
+ }
+
+ /**
+ * Internal class to represent nested mount points in INodeTree. Nested
mount points are non-leaf nodes in INode tree.
Review Comment:
Can you define "nested mount points" clearly here? The statement assumes a
certain definition - be more concrete
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -403,24 +455,24 @@ private void createLink(final String src, final String
target,
final String fullPath = curInode.fullPath + (curInode == root ? "" : "/")
+ iPath;
switch (linkType) {
-case SINGLE:
- newLink = new INodeLink(fullPath, aUgi,
- initAndGetTargetFs(), target);
- break;
-case SINGLE_FALLBACK:
-case MERGE_SLASH:
- // Link fallback and link merge slash configuration
- // are handled specially at InodeTree.
- throw new IllegalArgumentException("Unexpected linkType: " + linkType);
-case MERGE:
-case NFLY:
- final String[] targetUris = StringUtils.getStrings(target);
- newLink = new INodeLink(fullPath, aUgi,
- getTargetFileSystem(settings, StringUtils.stringToURI(targetUris)),
- targetUris);
- break;
-default:
- throw new IllegalArgumentException(linkType + ": Infeasible linkType");
+ case SINGLE:
Review Comment:
avoid this change - not required
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -682,6 +736,25 @@ protected InodeTree(final Configuration config, final
String viewName,
}
}
+ private Collection getLinkEntries(List linkEntries) {
+if (isNestedMountPointSupported) {
Review Comment:
add a comment on why you are sorting them with this change?
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -682,6 +736,25 @@ protected InodeTree(final Configuration config, final
String viewName,
}
}
+ private Collection getLinkEntries(List linkEntries) {
+if (isNestedMountPointSupported) {
+ Set sortedLinkEntries = new TreeSet<>((o1, o2) -> {
+if (o1 == null) {
+ return -1;
+}
+if (o2 == null) {
+ return 1;
+}
+String src1 = o1.getSrc().toLowerCase();
Review Comment:
Also, is lower casing the right way to do this? shouldn't you just compare
the input Strings as /Foo/Bar and /foo/bar are different paths?
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -133,6 +138,9 @@ public INode(String pathToNode, UserGroupInformation aUgi) {
// and is read only.
abstract boolean isInternalDir();
+// Identity some type of inodes(nested mount point).
Review Comment:
Nit: Use multi-line comment. The other methods here seem to be off.
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java:
##
@@ -167,8 +169,21 @@ void setupMountPoints() {
new Path(targetTestRoot, "missingTarget").toUri());
ConfigUtil.addLink(conf, "/linkToAFile",
new Path(targetTestRoot, "aFile").toUri());
+
+// Enable nested mount point, ViewFilesystem should support both
non-nested and nested mount points
+ConfigUtil.setIsNestedMountPointSupported(conf, true);
Review Comment:
+1 to this. it would be good to have tests specific to mount points.
##
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=758208=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758208
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 19/Apr/22 02:04
Start Date: 19/Apr/22 02:04
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1101919927
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 38s | | trunk passed |
| +1 :green_heart: | compile | 22m 46s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 19m 51s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 17s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 47s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 44s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 50s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 56s | | the patch passed |
| +1 :green_heart: | compile | 22m 0s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 22m 0s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/11/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 4 new + 1811 unchanged - 0
fixed = 1815 total (was 1811) |
| +1 :green_heart: | compile | 19m 55s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 19m 55s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/11/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 1688
unchanged - 0 fixed = 1692 total (was 1688) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/11/artifact/out/blanks-eol.txt)
| The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 6s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/11/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 70 new + 121
unchanged - 0 fixed = 191 total (was 121) |
| +1 :green_heart: | mvnsite | 1m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 8s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 39s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 38s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 43s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not
generate ASF License warnings. |
| | | 204m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=758206=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758206
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 19/Apr/22 01:26
Start Date: 19/Apr/22 01:26
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1101903774
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 17m 16s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 56s | | trunk passed |
| +1 :green_heart: | compile | 31m 57s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 27m 22s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 3s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 31s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 31m 6s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 31m 6s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/8/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 5 new + 1805 unchanged - 1
fixed = 1810 total (was 1806) |
| +1 :green_heart: | compile | 27m 7s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 27m 7s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/8/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 5 new + 1684
unchanged - 1 fixed = 1689 total (was 1685) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/8/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 27s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/8/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 72 new + 121
unchanged - 0 fixed = 193 total (was 121) |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 28s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 3s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 23s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 49s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 50s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 14s | | The patch does not
generate ASF License warnings. |
| | | 274m 25s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=758205=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758205
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 19/Apr/22 01:25
Start Date: 19/Apr/22 01:25
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1101903128
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 18m 12s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 10s | | trunk passed |
| +1 :green_heart: | compile | 32m 12s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 27m 25s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 1s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 33s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 4s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 29s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 31m 19s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 31m 19s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/7/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 4 new + 1811 unchanged - 0
fixed = 1815 total (was 1811) |
| +1 :green_heart: | compile | 27m 9s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 27m 9s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/7/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 1688
unchanged - 0 fixed = 1692 total (was 1688) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/7/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 27s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/7/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 72 new + 121
unchanged - 0 fixed = 193 total (was 121) |
| +1 :green_heart: | mvnsite | 2m 0s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 5s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 18s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 53s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 49s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 15s | | The patch does not
generate ASF License warnings. |
| | | 273m 33s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=758187=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758187
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 19/Apr/22 00:55
Start Date: 19/Apr/22 00:55
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1101890478
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 16s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 54s | | trunk passed |
| +1 :green_heart: | compile | 30m 21s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 25m 53s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 35s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 7s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 40s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 7s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 26s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 29m 35s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 29m 35s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/6/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 4 new + 1812 unchanged - 0
fixed = 1816 total (was 1812) |
| +1 :green_heart: | compile | 26m 6s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 26m 6s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/6/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 1688
unchanged - 0 fixed = 1692 total (was 1688) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/6/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 29s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/6/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 75 new + 120
unchanged - 1 fixed = 195 total (was 121) |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 34s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 6s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 24s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 39s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 15s | | The patch does not
generate ASF License warnings. |
| | | 252m 10s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=758186=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758186
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 19/Apr/22 00:53
Start Date: 19/Apr/22 00:53
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1101889716
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 49s | | trunk passed |
| +1 :green_heart: | compile | 30m 16s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 26m 2s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 11s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 27s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 6s | | the patch passed |
| +1 :green_heart: | compile | 29m 51s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 29m 51s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/5/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 5 new + 1805 unchanged - 1
fixed = 1810 total (was 1806) |
| +1 :green_heart: | compile | 26m 14s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 26m 14s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/5/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 5 new + 1681
unchanged - 1 fixed = 1686 total (was 1682) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/5/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 28s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/5/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 75 new + 120
unchanged - 1 fixed = 195 total (was 121) |
| +1 :green_heart: | mvnsite | 2m 5s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 34s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 8s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 23s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 32s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 2s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 17s | | The patch does not
generate ASF License warnings. |
| | | 252m 14s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=758184=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758184
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 19/Apr/22 00:52
Start Date: 19/Apr/22 00:52
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1101889494
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 26s | | trunk passed |
| +1 :green_heart: | compile | 25m 51s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 22m 16s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 45s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 3s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 57s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 6s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 5s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 4s | | the patch passed |
| +1 :green_heart: | compile | 24m 57s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 24m 57s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/10/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 4 new + 1813 unchanged - 0
fixed = 1817 total (was 1813) |
| +1 :green_heart: | compile | 22m 25s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 22m 25s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/10/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 1689
unchanged - 0 fixed = 1693 total (was 1689) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/10/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 41s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/10/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 72 new + 121
unchanged - 0 fixed = 193 total (was 121) |
| +1 :green_heart: | mvnsite | 2m 6s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 59s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 57s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 1s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 38s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 34s | | The patch does not
generate ASF License warnings. |
| | | 227m 37s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=758182=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758182
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 19/Apr/22 00:41
Start Date: 19/Apr/22 00:41
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1101884696
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 4s | | trunk passed |
| +1 :green_heart: | compile | 25m 2s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 22m 39s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 36s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 5s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 21s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 9s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 2s | | the patch passed |
| +1 :green_heart: | compile | 24m 28s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 24m 28s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/9/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 5 new + 1805 unchanged - 1
fixed = 1810 total (was 1806) |
| +1 :green_heart: | compile | 22m 34s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 22m 34s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/9/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 5 new + 1681
unchanged - 1 fixed = 1686 total (was 1682) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/9/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 29s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/9/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 75 new + 120
unchanged - 1 fixed = 195 total (was 121) |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 6s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 22s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 26s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 29s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 20s | | The patch does not
generate ASF License warnings. |
| | | 226m 47s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=758175=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758175
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 19/Apr/22 00:14
Start Date: 19/Apr/22 00:14
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1101873007
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 10s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 2s | | trunk passed |
| +1 :green_heart: | compile | 23m 17s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 20m 34s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 46s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 12s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 4s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 45s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 5s | | the patch passed |
| +1 :green_heart: | compile | 22m 33s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 22m 33s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/4/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 4 new + 1811 unchanged - 0
fixed = 1815 total (was 1811) |
| +1 :green_heart: | compile | 20m 51s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 20m 51s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/4/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 1688
unchanged - 0 fixed = 1692 total (was 1688) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 41s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/4/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 75 new + 120
unchanged - 1 fixed = 195 total (was 121) |
| +1 :green_heart: | mvnsite | 2m 11s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 43s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 14s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 36s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 40s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 33s | | The patch does not
generate ASF License warnings. |
| | | 219m 34s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4181 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=758068=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758068
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 18/Apr/22 19:05
Start Date: 18/Apr/22 19:05
Worklog Time Spent: 10m
Work Description: li-leyang commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r852340403
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -81,6 +88,19 @@ enum ResultKind {
private List> regexMountPointList =
new ArrayList>();
+ private final boolean isNestedMountPointSupported;
+ private Set sortedLinkEntries = new TreeSet<>((o1, o2) -> {
Review Comment:
I agree we can get rid of a a member. I originally used it as a member but
later return in a function and didn't remove it.
Issue Time Tracking
---
Worklog Id: (was: 758068)
Time Spent: 1h 20m (was: 1h 10m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Assignee: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
>
> INodeTree has 2 methods that need change to support nested mount points.
> createLink(..): build INodeTree during fs init.
> resolve(..): resolve path in INodeTree with viewfs apis.
>
> ViewFileSystem and ViewFs referes INodeTree.resolve(..) to resolve path to
> specific mount point. No changes are expected in both classes. However, we
> need to support existing use cases and make sure no regression are caused.
>
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=757543=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757543
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 16/Apr/22 00:56
Start Date: 16/Apr/22 00:56
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1100491152
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 24s | | trunk passed |
| +1 :green_heart: | compile | 30m 40s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 26m 22s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 50s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 50s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 11s | | the patch passed |
| +1 :green_heart: | compile | 30m 37s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 30m 37s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/3/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 5 new + 1812 unchanged - 0
fixed = 1817 total (was 1812) |
| +1 :green_heart: | compile | 26m 24s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 26m 24s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/3/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 5 new + 1688
unchanged - 0 fixed = 1693 total (was 1688) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 20s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/3/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 77 new + 120
unchanged - 1 fixed = 197 total (was 121) |
| +1 :green_heart: | mvnsite | 1m 47s | | the patch passed |
| -1 :x: | javadoc | 0m 45s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/3/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| hadoop-common in the patch failed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| +1 :green_heart: | javadoc | 1m 44s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 21s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 54s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 0m 52s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/3/artifact/out/results-asflicense.txt)
| The patch generated 1 ASF License
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=757542=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757542
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 16/Apr/22 00:56
Start Date: 16/Apr/22 00:56
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1100491064
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 24s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 36s | | trunk passed |
| +1 :green_heart: | compile | 30m 57s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 26m 29s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 13s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 19s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 30m 36s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 30m 36s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/2/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 6 new + 1805 unchanged - 1
fixed = 1811 total (was 1806) |
| +1 :green_heart: | compile | 26m 34s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 26m 34s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/2/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 6 new + 1681
unchanged - 1 fixed = 1687 total (was 1682) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 15s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/2/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 77 new + 120
unchanged - 1 fixed = 197 total (was 121) |
| +1 :green_heart: | mvnsite | 1m 48s | | the patch passed |
| -1 :x: | javadoc | 0m 45s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/2/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| hadoop-common in the patch failed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| +1 :green_heart: | javadoc | 1m 43s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 6s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 44s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 0m 53s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/2/artifact/out/results-asflicense.txt)
| The patch generated 1 ASF License
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=757535=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757535
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 16/Apr/22 00:16
Start Date: 16/Apr/22 00:16
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#issuecomment-1100479705
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 55s | | trunk passed |
| +1 :green_heart: | compile | 22m 53s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 20m 3s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 12s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 16s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 48s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 14s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 58s | | the patch passed |
| +1 :green_heart: | compile | 22m 16s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 22m 16s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/1/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 5 new + 1811 unchanged - 0
fixed = 1816 total (was 1811) |
| +1 :green_heart: | compile | 20m 0s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 20m 0s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/1/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 5 new + 1688
unchanged - 0 fixed = 1693 total (was 1688) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 6s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/1/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 77 new + 120
unchanged - 1 fixed = 197 total (was 121) |
| +1 :green_heart: | mvnsite | 1m 39s | | the patch passed |
| -1 :x: | javadoc | 0m 53s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/1/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| hadoop-common in the patch failed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| +1 :green_heart: | javadoc | 1m 46s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 41s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 2s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 59s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 1m 2s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4181/1/artifact/out/results-asflicense.txt)
| The patch generated 1 ASF License
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=757524=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757524
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 15/Apr/22 22:35
Start Date: 15/Apr/22 22:35
Worklog Time Spent: 10m
Work Description: mccormickt12 commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r851542051
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java:
##
@@ -81,6 +88,19 @@ enum ResultKind {
private List> regexMountPointList =
new ArrayList>();
+ private final boolean isNestedMountPointSupported;
+ private Set sortedLinkEntries = new TreeSet<>((o1, o2) -> {
Review Comment:
its a little weird that `sortedLinkEntries` is a private member, but only
used in `getLinkEntries` which just does an `addAll` and return. I think we
should probably get rid of this as a member.
otherwise we need to consider clearing the set. It also just seems like an
unnecessary memory overhead.
Issue Time Tracking
---
Worklog Id: (was: 757524)
Time Spent: 40m (was: 0.5h)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
>
> INodeTree has 2 methods that need change to support nested mount points.
> createLink(..): build INodeTree during fs init.
> resolve(..): resolve path in INodeTree with viewfs apis.
>
> ViewFileSystem and ViewFs referes INodeTree.resolve(..) to resolve path to
> specific mount point. No changes are expected in both classes. However, we
> need to support existing use cases and make sure no regression are caused.
>
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=757521=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757521
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 15/Apr/22 22:07
Start Date: 15/Apr/22 22:07
Worklog Time Spent: 10m
Work Description: mccormickt12 commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r851533168
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java:
##
@@ -167,8 +169,21 @@ void setupMountPoints() {
new Path(targetTestRoot, "missingTarget").toUri());
ConfigUtil.addLink(conf, "/linkToAFile",
new Path(targetTestRoot, "aFile").toUri());
+
+// Enable nested mount point, ViewFilesystem should support both
non-nested and nested mount points
+ConfigUtil.setIsNestedMountPointSupported(conf, true);
Review Comment:
this is going to affect all the other tests here right?
We might want to keep the testing of nested mount points separate
Issue Time Tracking
---
Worklog Id: (was: 757521)
Time Spent: 0.5h (was: 20m)
> Support nested mount points in INodeTree
>
>
> Key: HADOOP-18193
> URL: https://issues.apache.org/jira/browse/HADOOP-18193
> Project: Hadoop Common
> Issue Type: Improvement
> Components: viewfs
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Defining following client mount table config is not supported in INodeTree
> and will throw FileAlreadyExistsException
> fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar
> fs.viewfs.mounttable.link./foo=hdfs://nn02/foo
>
> INodeTree has 2 methods that need change to support nested mount points.
> createLink(..): build INodeTree during fs init.
> resolve(..): resolve path in INodeTree with viewfs apis.
>
> ViewFileSystem and ViewFs referes INodeTree.resolve(..) to resolve path to
> specific mount point. No changes are expected in both classes. However, we
> need to support existing use cases and make sure no regression are caused.
>
> AC:
> # INodeTree.createlink should support creating nested mount
> points.(INodeTree is constructed during fs init)
> # INodeTree.resolve should support resolve path based on nested mount
> points. (INodeTree.resolve is used in viewfs apis)
> # No regression in existing ViewFileSystem and ViewFs apis.
> # Ensure some important apis are not broken with nested mount points.
> (Rename, getContentSummary, listStatus...)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[
https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=757520=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757520
]
ASF GitHub Bot logged work on HADOOP-18193:
---
Author: ASF GitHub Bot
Created on: 15/Apr/22 22:05
Start Date: 15/Apr/22 22:05
Worklog Time Spent: 10m
Work Description: mccormickt12 commented on code in PR #4181:
URL: https://github.com/apache/hadoop/pull/4181#discussion_r851530313
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestNestedMountPoint.java:
##
@@ -0,0 +1,345 @@
+package org.apache.hadoop.fs.viewfs;
+
+import java.net.URI;
+import java.util.List;
+import java.util.function.Function;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.FsConstants;
+import org.apache.hadoop.fs.Path;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+
+public class TestNestedMountPoint {
+ private InodeTree inodeTree;
+ private Configuration conf;
+ private String mtName;
+ private URI fsUri;
+
+ static class TestNestMountPointFileSystem {
+public URI getUri() {
+ return uri;
+}
+
+private URI uri;
+
+TestNestMountPointFileSystem(URI uri) {
+ this.uri = uri;
+}
+ }
+
+ static class TestNestMountPointInternalFileSystem extends
TestNestMountPointFileSystem {
+TestNestMountPointInternalFileSystem(URI uri) {
+ super(uri);
+}
+ }
+
+ private static final URI LINKFALLBACK_TARGET = URI.create("hdfs://nn00");
+ private static final URI NN1_TARGET = URI.create("hdfs://nn01/a/b");
+ private static final URI NN2_TARGET = URI.create("hdfs://nn02/a/b/e");
+ private static final URI NN3_TARGET = URI.create("hdfs://nn03/a/b/c/d");
+ private static final URI NN4_TARGET = URI.create("hdfs://nn04/a/b/c/d/e");
+ private static final URI NN5_TARGET = URI.create("hdfs://nn05/b/c/d/e");
+ private static final URI NN6_TARGET = URI.create("hdfs://nn06/b/c/d/e/f");
+
+ @Before
+ public void setUp() throws Exception {
+conf = new Configuration();
+mtName = TestNestedMountPoint.class.getName();
+ConfigUtil.setIsNestedMountPointSupported(conf, true);
+ConfigUtil.addLink(conf, mtName, "/a/b", NN1_TARGET);
+ConfigUtil.addLink(conf, mtName, "/a/b/e", NN2_TARGET);
Review Comment:
each dirlink, only has one nested mount point. Could could add one dirlink
that has multiple nested mount points under it.
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestNestedMountPoint.java:
##
@@ -0,0 +1,345 @@
+package org.apache.hadoop.fs.viewfs;
+
+import java.net.URI;
+import java.util.List;
+import java.util.function.Function;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.FsConstants;
+import org.apache.hadoop.fs.Path;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+
+public class TestNestedMountPoint {
+ private InodeTree inodeTree;
+ private Configuration conf;
+ private String mtName;
+ private URI fsUri;
+
+ static class TestNestMountPointFileSystem {
+public URI getUri() {
+ return uri;
+}
+
+private URI uri;
+
+TestNestMountPointFileSystem(URI uri) {
+ this.uri = uri;
+}
+ }
+
+ static class TestNestMountPointInternalFileSystem extends
TestNestMountPointFileSystem {
+TestNestMountPointInternalFileSystem(URI uri) {
+ super(uri);
+}
+ }
+
+ private static final URI LINKFALLBACK_TARGET = URI.create("hdfs://nn00");
+ private static final URI NN1_TARGET = URI.create("hdfs://nn01/a/b");
+ private static final URI NN2_TARGET = URI.create("hdfs://nn02/a/b/e");
+ private static final URI NN3_TARGET = URI.create("hdfs://nn03/a/b/c/d");
+ private static final URI NN4_TARGET = URI.create("hdfs://nn04/a/b/c/d/e");
+ private static final URI NN5_TARGET = URI.create("hdfs://nn05/b/c/d/e");
+ private static final URI NN6_TARGET = URI.create("hdfs://nn06/b/c/d/e/f");
+
+ @Before
+ public void setUp() throws Exception {
+conf = new Configuration();
+mtName = TestNestedMountPoint.class.getName();
+ConfigUtil.setIsNestedMountPointSupported(conf, true);
+ConfigUtil.addLink(conf, mtName, "/a/b", NN1_TARGET);
+ConfigUtil.addLink(conf, mtName, "/a/b/e", NN2_TARGET);
+ConfigUtil.addLink(conf, mtName, "/a/b/c/d", NN3_TARGET);
+ConfigUtil.addLink(conf, mtName, "/a/b/c/d/e", NN4_TARGET);
+ConfigUtil.addLink(conf, mtName, "/b/c/d/e", NN5_TARGET);
+ConfigUtil.addLink(conf, mtName, "/b/c/d/e/f", NN6_TARGET);
+ConfigUtil.addLinkFallback(conf, mtName, LINKFALLBACK_TARGET);
+
+fsUri = new URI(FsConstants.VIEWFS_SCHEME, mtName, "/", null, null);
+
+inodeTree = new
InodeTree(conf,
+
[jira] [Work logged] (HADOOP-18193) Support nested mount points in INodeTree
[ https://issues.apache.org/jira/browse/HADOOP-18193?focusedWorklogId=757505=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757505 ] ASF GitHub Bot logged work on HADOOP-18193: --- Author: ASF GitHub Bot Created on: 15/Apr/22 20:48 Start Date: 15/Apr/22 20:48 Worklog Time Spent: 10m Work Description: li-leyang opened a new pull request, #4181: URL: https://github.com/apache/hadoop/pull/4181 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? Issue Time Tracking --- Worklog Id: (was: 757505) Remaining Estimate: 0h Time Spent: 10m > Support nested mount points in INodeTree > > > Key: HADOOP-18193 > URL: https://issues.apache.org/jira/browse/HADOOP-18193 > Project: Hadoop Common > Issue Type: Improvement > Components: viewfs >Affects Versions: 2.10.0 >Reporter: Lei Yang >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Defining following client mount table config is not supported in INodeTree > and will throw FileAlreadyExistsException > fs.viewfs.mounttable.link./foo/bar=hdfs://nn1/foo/bar > fs.viewfs.mounttable.link./foo=hdfs://nn02/foo > > INodeTree has 2 methods that need change to support nested mount points. > createLink(..): build INodeTree during fs init. > resolve(..): resolve path in INodeTree with viewfs apis. > > ViewFileSystem and ViewFs referes INodeTree.resolve(..) to resolve path to > specific mount point. No changes are expected in both classes. However, we > need to support existing use cases and make sure no regression. > > AC: > # INodeTree.createlink should support creating nested mount > points.(INodeTree is constructed during fs init) > # INodeTree.resolve should support resolve path based on nested mount > points. (INodeTree.resolve is used in viewfs apis) > # No regression in existing ViewFileSystem and ViewFs apis. > # Ensure some important apis are not broken with nested mount points. > (Rename, getContentSummary, listStatus...) -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
