[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550274#comment-14550274 ] Hudson commented on HDFS-8345: -- FAILURE: Integrated in Hadoop-Yarn-trunk #932 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/932/]) HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Fix For: 2.8.0 Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch, HDFS-8345.07.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550305#comment-14550305 ] Hudson commented on HDFS-8345: -- SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #201 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/201/]) HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Fix For: 2.8.0 Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch, HDFS-8345.07.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550506#comment-14550506 ] Hudson commented on HDFS-8345: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2130 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2130/]) HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Fix For: 2.8.0 Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch, HDFS-8345.07.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550536#comment-14550536 ] Hudson commented on HDFS-8345: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #190 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/190/]) HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Fix For: 2.8.0 Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch, HDFS-8345.07.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550632#comment-14550632 ] Hudson commented on HDFS-8345: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2148 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2148/]) HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Fix For: 2.8.0 Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch, HDFS-8345.07.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550595#comment-14550595 ] Hudson commented on HDFS-8345: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #200 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/200/]) HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Fix For: 2.8.0 Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch, HDFS-8345.07.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14548479#comment-14548479 ] Arpit Agarwal commented on HDFS-8345: - Hi Sean, it's returned via DistributedFileSystem (it's in the Jira description). Applications are using the storage policy APIs so I don't see the point of making a breaking change unless necessary. Reclaiming the interface name is not enough reason. The checkstyle issues are not related to the patch. I will commit this shortly. Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch, HDFS-8345.07.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14548631#comment-14548631 ] Hudson commented on HDFS-8345: -- FAILURE: Integrated in Hadoop-trunk-Commit #7855 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7855/]) HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Fix For: 2.8.0 Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch, HDFS-8345.07.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547033#comment-14547033
]
Hadoop QA commented on HDFS-8345:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 43s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 31s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 54s | The applied patch generated 2
new checkstyle issues (total was 286, now 286). |
| {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 5m 26s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | common tests | 23m 42s | Tests passed in
hadoop-common. |
| {color:green}+1{color} | hdfs tests | 164m 26s | Tests passed in hadoop-hdfs.
|
| {color:green}+1{color} | hdfs tests | 0m 16s | Tests passed in
hadoop-hdfs-client. |
| | | 230m 58s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12733254/HDFS-8345.07.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / e453989 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/diffcheckstylehadoop-common.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11017/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11017/console |
This message was automatically generated.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch,
HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch,
HDFS-8345.06.patch, HDFS-8345.07.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545531#comment-14545531
]
Sean Busbey commented on HDFS-8345:
---
It would be nice if the returned values from getAllStoragePolicies were usable
in a setStoragePolicy method.
Could we update the FileSystem specification documents?
{code}
+ * Set the storage policy for a given file or directory.
+ *
+ * @param path file or directory path.
+ * @param policyName the name of the target storage policy. The list
+ * of supported Storage policies can be retrieved
+ * via {@link #getStoragePolicyNames}.
+ */
+ public void setStoragePolicy(final Path path, final String policyName)
+ throws IOException {
{code}
Note that it's an optional operation and some implementations will throw
UnsupportOperationException.
{code}
+ /**
+ * Retrieve all the storage policies supported by this file system.
+ *
+ * @return all storage policies supported by this filesystem.
+ * @throws IOException
+ */
+ public List? extends BlockStoragePolicySpi getAllStoragePolicies()
+ throws IOException {
{code}
Note that it's an optional operation and some implementations will throw
UnsupportOperationException.
{code}
+ * @param policyName the name of the target storage policy. The list
+ * of supported Storage policies can be retrieved
+ * via {@link #getStoragePolicyNames}.
+ */
{code}
The method getStoragePolicyNames didn't make it into the patch. Update to
point at new method?
{code}
+ public List? extends BlockStoragePolicySpi getAllStoragePolicies()
+ throws IOException {
{code}
Should this return type be Collection? extends BlockStoragePolicySpi ? does
the ordering matter? Maybe it should be a Set?
{code}
+public interface BlockStoragePolicySpi {
+
{code}
Could we switch to calling the interface BlockStoragePolicy and make the one in
HDFS DFSBlockStoragePolicy or the like?
Spi is a term of art that's going to be less accessible for folks.
{code}
+ StorageType[] getStorageTypes();
{code}
Should the various return types in BlockStoragePolicySpi be returning List or
Set or SortedSet?
{code}
return fs.createNonRecursive(f, permission, flags, bufferSize,
replication, blockSize,
-progress);
+ progress);
}
{code}
nit: this whitespace fix doesn't appear related to the rest of the changes in
the file (but I'm not sure what Hadoop's norm is for 'related')
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch,
HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546233#comment-14546233
]
Sean Busbey commented on HDFS-8345:
---
{quote}
bq. Can you point me to the exposure? BlockStoragePolicy is IA.Private, and I
would expect we can use covariant return types to make a source-compatible
update.
It was tagged IA.Private but returned via a LimitedPrivate interface,
unfortunate as that is. Changing the name, package or public interface of
BlockStoragePolicy would technically be incompatible. However if you suggest a
better name for the new interface I'd be happy to change it.
{quote}
Which LimitedPrivate though? I'd like to make sure we can't do the change in a
compatible way.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch,
HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch,
HDFS-8345.06.patch, HDFS-8345.07.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546354#comment-14546354
]
Hadoop QA commented on HDFS-8345:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 17s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 44s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 0s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 26s | The applied patch generated 3
new checkstyle issues (total was 286, now 287). |
| {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 2m 5s | The patch appears to cause
Findbugs (version 2.0.3) to fail. |
| {color:green}+1{color} | common tests | 23m 5s | Tests passed in
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 0m 30s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 15s | Tests passed in
hadoop-hdfs-client. |
| | | 64m 25s | |
\\
\\
|| Reason || Tests ||
| Failed build | hadoop-hdfs |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12733254/HDFS-8345.07.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f7e051c |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/diffcheckstylehadoop-common.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11014/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11014/console |
This message was automatically generated.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch,
HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch,
HDFS-8345.06.patch, HDFS-8345.07.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546528#comment-14546528
]
Hadoop QA commented on HDFS-8345:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 34s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 30s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 42s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 24s | The applied patch generated 2
new checkstyle issues (total was 286, now 286). |
| {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 5m 22s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | common tests | 22m 37s | Tests passed in
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 168m 35s | Tests failed in hadoop-hdfs. |
| {color:red}-1{color} | hdfs tests | 0m 16s | Tests failed in
hadoop-hdfs-client. |
| | | 234m 3s | |
\\
\\
|| Reason || Tests ||
| Failed build | hadoop-hdfs |
| | hadoop-hdfs-client |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12733254/HDFS-8345.07.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f7e051c |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/diffcheckstylehadoop-common.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11016/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11016/console |
This message was automatically generated.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch,
HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch,
HDFS-8345.06.patch, HDFS-8345.07.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545804#comment-14545804 ] Arpit Agarwal commented on HDFS-8345: - Thank you for the detailed review [~busbey]. bq. It would be nice if the returned values from getAllStoragePolicies were usable in a setStoragePolicy method. getAllStoragePolicies returns a collection, you can use BlockStoragePolicySpi#getName on a collection element as the parameter to setStoragePolicy. bq. Could we update the FileSystem specification documents? Which specification docs are you referring to? bq. Note that it's an optional operation and some implementations will throw UnsupportOperationException. Let's do that separately for all FileSystem API calls that fit the same pattern. bq. The method getStoragePolicyNames didn't make it into the patch. Update to point at new method? Fixed. bq. Should this return type be Collection? extends BlockStoragePolicySpi ? does the ordering matter? Maybe it should be a Set? Fixed to return a Collection. bq. Could we switch to calling the interface BlockStoragePolicy and make the one in HDFS DFSBlockStoragePolicy or the like? Spi is a term of art that's going to be less accessible for folks. bq. Should the various return types in BlockStoragePolicySpi be returning List or Set or SortedSet? I agree on both points. However it's likely incompatible since BlockStoragePolicy was exposed via a limited private interface in 2.6. bq. nit: this whitespace fix doesn't appear related to the rest of the changes in the file (but I'm not sure what Hadoop's norm is for 'related') Good catch, IntelliJ slipped that in. Fixed. Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546117#comment-14546117
]
Sean Busbey commented on HDFS-8345:
---
{quote}
bq. It would be nice if the returned values from getAllStoragePolicies were
usable in a setStoragePolicy method.
getAllStoragePolicies returns a collection, you can use
BlockStoragePolicySpi#getName on a collection element as the parameter to
setStoragePolicy.
{quote}
That's not so bad. Is there a reason not to include a convenience method that
just takes a BlockStoragePolicySpi object directly?
{quote}
bq. Could we update the FileSystem specification documents?
Which specification docs are you referring to?
{quote}
The one in the docs:
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/index.html
{quote}
bq. Could we switch to calling the interface BlockStoragePolicy and make the
one in HDFS DFSBlockStoragePolicy or the like? Spi is a term of art that's
going to be less accessible for folks.
bq. Should the various return types in BlockStoragePolicySpi be returning List
or Set or SortedSet?
I agree on both points. However it's likely incompatible since
BlockStoragePolicy was exposed via a limited private interface in 2.6.
{quote}
Can you point me to the exposure? BlockStoragePolicy is IA.Private, and I would
expect we can use covariant return types to make a source-compatible update.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch,
HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546170#comment-14546170
]
Hadoop QA commented on HDFS-8345:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 31s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 28s | There were no new javac warning
messages. |
| {color:red}-1{color} | javadoc | 9m 32s | The applied patch generated 2
additional warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 23s | The applied patch generated 2
new checkstyle issues (total was 286, now 286). |
| {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 5m 25s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | common tests | 23m 36s | Tests passed in
hadoop-common. |
| {color:green}+1{color} | hdfs tests | 168m 7s | Tests passed in hadoop-hdfs.
|
| {color:green}+1{color} | hdfs tests | 0m 16s | Tests passed in
hadoop-hdfs-client. |
| | | 234m 22s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12733187/HDFS-8345.06.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 03a293a |
| javadoc |
https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/diffJavadocWarnings.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/diffcheckstylehadoop-common.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11006/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11006/console |
This message was automatically generated.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch,
HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546173#comment-14546173 ] Arpit Agarwal commented on HDFS-8345: - bq. That's not so bad. Is there a reason not to include a convenience method that just takes a BlockStoragePolicySpi object directly? That doesn't sound useful enough to introduce one more FileSystem API. Can you think of an example where the extra API call would simplify application code? bq. Can you point me to the exposure? BlockStoragePolicy is IA.Private, and I would expect we can use covariant return types to make a source-compatible update. It was tagged IA.Private but returned via a LimitedPrivate interface, unfortunate as that is. Changing the name, package or public interface of BlockStoragePolicy would technically be incompatible. However if you suggest a better name for the new interface I'd be happy to change it. bq. The one in the docs: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/index.html Okay let me file a separate Jira for that. Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546187#comment-14546187 ] Jing Zhao commented on HDFS-8345: - bq. Is there a reason not to include a convenience method that just takes a BlockStoragePolicySpi object directly Currently we still expect admins to set up policies first, and the users later use names of existing policies to config files/directories. Directly using a policy object in the API will allow users to pass self-created policies and also bring in extra complexity for server side verification. +1 on the 07 patch. Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch, HDFS-8345.06.patch, HDFS-8345.07.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544525#comment-14544525
]
Hadoop QA commented on HDFS-8345:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12732978/HDFS-8345.04.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 09fe16f |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10989/console |
This message was automatically generated.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch,
HDFS-8345.03.patch, HDFS-8345.04.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544933#comment-14544933 ] Jing Zhao commented on HDFS-8345: - Thanks for updating the patch, Arpit! The latest patch looks pretty good to me. But looks like still need to fix the javac/javadoc warnings. Apart from that +1. Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch, HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544887#comment-14544887
]
Hadoop QA commented on HDFS-8345:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 11s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:red}-1{color} | javac | 7m 39s | The applied patch generated 4
additional warning messages. |
| {color:red}-1{color} | javadoc | 9m 50s | The applied patch generated 3
additional warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 30s | The applied patch generated 3
new checkstyle issues (total was 286, now 287). |
| {color:red}-1{color} | whitespace | 0m 2s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 39s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 5m 31s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | common tests | 23m 15s | Tests passed in
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 167m 59s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 16s | Tests passed in
hadoop-hdfs-client. |
| | | 235m 19s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.tools.TestHdfsConfigFields |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12732993/HDFS-8345.05.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 9a2a955 |
| javac |
https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/diffJavacWarnings.txt
|
| javadoc |
https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/diffJavadocWarnings.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/diffcheckstylehadoop-common.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/whitespace.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10993/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10993/console |
This message was automatically generated.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch,
HDFS-8345.03.patch, HDFS-8345.04.patch, HDFS-8345.05.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542528#comment-14542528
]
Arpit Agarwal commented on HDFS-8345:
-
Also I added a new call {{FileSystem#getAllStoragePolicies}}.
I left {{DistributedFileSystem#getStoragePolicies}} as it is and tagged it
{{@Deprecated}}.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch,
HDFS-8345.03.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542883#comment-14542883
]
Hadoop QA commented on HDFS-8345:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 58s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:red}-1{color} | javac | 7m 41s | The applied patch generated 4
additional warning messages. |
| {color:red}-1{color} | javadoc | 9m 52s | The applied patch generated 3
additional warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 27s | The applied patch generated 3
new checkstyle issues (total was 269, now 270). |
| {color:red}-1{color} | whitespace | 0m 2s | The patch has 2 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 38s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 5m 26s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | common tests | 22m 38s | Tests failed in
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 140m 40s | Tests failed in hadoop-hdfs. |
| {color:red}-1{color} | hdfs tests | 0m 14s | Tests failed in
hadoop-hdfs-client. |
| | | 207m 0s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.fs.TestFilterFileSystem |
| | hadoop.fs.TestHarFileSystem |
| Timed out tests |
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles |
| Failed build | hadoop-hdfs-client |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12732656/HDFS-8345.03.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 93b770f |
| javac |
https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/diffJavacWarnings.txt
|
| javadoc |
https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/diffJavadocWarnings.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/diffcheckstylehadoop-common.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/whitespace.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10956/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10956/console |
This message was automatically generated.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch,
HDFS-8345.03.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542226#comment-14542226
]
Arpit Agarwal commented on HDFS-8345:
-
Thanks for the review [~jingzhao]. I wanted to do so but wasn't sure we want to
expose {{BlockStoragePolicy}} publicly. There are too many public functions
that should be internal to HDFS. What do you think?
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542305#comment-14542305
]
Jing Zhao commented on HDFS-8345:
-
Looks like most of the public functions in {{BlockStoragePolicy}} are
generically choosing {{StorageType}} for replicas, and replication is a general
concept and is already included in {{FileStatus}}. If you think some functions
should be internal to HDFS, maybe we can also have a new subclass/wrapper of
BlockStoragePolicy in HDFS?
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542344#comment-14542344
]
Arpit Agarwal commented on HDFS-8345:
-
On further thought I'll have the base class return a list of
{{BlockStoragePolicySpi}} which exposes the policy details.
BlockStoragePolicy can implement BlockStoragePolicySpi. Will post an updated
patch shortly.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541080#comment-14541080
]
Jing Zhao commented on HDFS-8345:
-
Thanks for working on this, Arpit! The patch looks good to me overall. My only
concern is that only exposing {{setStoragePolicy}} and
{{getStoragePoliciyNames}} may be incomplete, since the user will only know the
names of the policies. Since {{StorageType}} is already in
{{org.apache.hadoop.fs}}, how about also moving {{BlockStoragePolicy}} into
{{org.apache.hadoop.fs}} and expose it as public? Then we can expose both
{{setStoragePolicy}} and {{getStoragePolicies}} in {{FileSystem}}.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540859#comment-14540859 ] Xiaoyu Yao commented on HDFS-8345: -- The patch looks good to me. I just have few comments: 1. NIT: Missing @throws IOException for Javadoc of FileSystem#setStoragePolicy(). 2. Do we miss the ViewFS#getStoragePolicyNames() override since ViewFS#setStoragePolicy is override. Or we expect it fallback to FileSystem#getStoragePoliciyNames that throws UnsupportedOperationException? 3. Can you add a unit test for the new API getStoragePolicyNames()? Storage policy APIs must be exposed via the FileSystem interface Key: HDFS-8345 URL: https://issues.apache.org/jira/browse/HDFS-8345 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.7.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Labels: BB2015-05-TBR Attachments: HDFS-8345.01.patch The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface
[
https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541274#comment-14541274
]
Hadoop QA commented on HDFS-8345:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 37s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 29s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 31s | The applied patch generated 3
new checkstyle issues (total was 270, now 271). |
| {color:red}-1{color} | whitespace | 0m 1s | The patch has 4 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 38s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 43s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | common tests | 22m 25s | Tests failed in
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 165m 44s | Tests failed in hadoop-hdfs. |
| | | 229m 8s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.fs.TestFilterFileSystem |
| | hadoop.fs.TestHarFileSystem |
| | hadoop.tools.TestHdfsConfigFields |
| | hadoop.tracing.TestTraceAdmin |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12732420/HDFS-8345.02.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f24452d |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/diffcheckstylehadoop-common.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/whitespace.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10941/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10941/console |
This message was automatically generated.
Storage policy APIs must be exposed via the FileSystem interface
Key: HDFS-8345
URL: https://issues.apache.org/jira/browse/HDFS-8345
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.7.0
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Labels: BB2015-05-TBR
Attachments: HDFS-8345.01.patch, HDFS-8345.02.patch
The storage policy APIs are not exposed via FileSystem. Since
DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs
through FileSystem for use by other applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
