[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381638#comment-14381638 ] Hadoop QA commented on HDFS-7824: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12707437/HDFS-7824.06.patch against trunk revision b4b4fe9. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 4 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.tracing.TestTracing org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10074//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10074//console This message is automatically generated. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch, HDFS-7824.04.patch, HDFS-7824.05.patch, HDFS-7824.06.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382265#comment-14382265 ] Arpit Agarwal commented on HDFS-7824: - +1 for the patch. Thanks for your patience with getting this patch ready. I will commit it shortly. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch, HDFS-7824.04.patch, HDFS-7824.05.patch, HDFS-7824.06.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382117#comment-14382117 ] Xiaoyu Yao commented on HDFS-7824: -- TestTracing#testWriteTraceHooks has known issue tracked by HDFS-7965. TestLazyPersistFiles#testDnRestartWithSavedReplicas failure should not related to this patch. I'm checking the test log and find lots of FsVolumeImpl.java:printReferenceTraceInfo, which may affect the timing of the verification. I will file separate JIRA for it if needed. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch, HDFS-7824.04.patch, HDFS-7824.05.patch, HDFS-7824.06.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381470#comment-14381470 ] Xiaoyu Yao commented on HDFS-7824: -- BlockManager#convertLastBlockToUnderConstruction need to pass BlockStoragePolicySuite when calling BlockCollection#computeContentSummary so that the ContentSummaryComputationContext object is constructed with a valid BlockStoragePolicy to compute quota and usage. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch, HDFS-7824.04.patch, HDFS-7824.05.patch, HDFS-7824.06.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381418#comment-14381418 ] Arpit Agarwal commented on HDFS-7824: - bq. Update patch that fixes the unit test failure inTestTruncateQuotaUpdate. Hi [~xyao], what was the fix? GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch, HDFS-7824.04.patch, HDFS-7824.05.patch, HDFS-7824.06.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380692#comment-14380692 ] Arpit Agarwal commented on HDFS-7824: - +1 for just suppressing the warning for this test. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380688#comment-14380688 ] Xiaoyu Yao commented on HDFS-7824: -- The javac warning is from TestCount.MockContentSummary which extends ContentSummary with default constructor that has been deprecated by the new ContentSummary.Builder class. {code} [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestCount.java:[318,32] [deprecation] ContentSummary() in ContentSummary has been deprecated {code} There are two approaches that we can fix this: 1) Suppress the warning for TestCount.MockContentSummary#MockContentSumary() {code} static class MockContentSummary extends ContentSummary { @SuppressWarnings(deprecation) // suppress warning on the usage of deprecated ContentSummary constructor public MockContentSummary() { } {code} 2) Change MockContentSummary from inheritance to composition and provide mocks for all the ContentSummary methods. Considering this is for the single test only and amount of changes for 2), I would prefer 1). Let me know if you have other ideas on fixing this. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380822#comment-14380822 ] Hadoop QA commented on HDFS-7824: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12707319/HDFS-7824.04.patch against trunk revision fc1031a. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified test files. {color:red}-1 javac{color:red}. The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10068//console This message is automatically generated. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch, HDFS-7824.04.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381270#comment-14381270 ] Hadoop QA commented on HDFS-7824: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12707353/HDFS-7824.05.patch against trunk revision 44809b8. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.tracing.TestTracing org.apache.hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys org.apache.hadoop.hdfs.server.namenode.TestTruncateQuotaUpdate Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10071//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10071//console This message is automatically generated. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch, HDFS-7824.04.patch, HDFS-7824.05.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381317#comment-14381317 ] Xiaoyu Yao commented on HDFS-7824: -- org.apache.hadoop.hdfs.server.namenode.TestTruncateQuotaUpdate failure seems related. I will update the patch with the fix. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch, HDFS-7824.04.patch, HDFS-7824.05.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376506#comment-14376506 ] Hadoop QA commented on HDFS-7824: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705760/HDFS-7824.03.patch against trunk revision 82eda77. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:red}-1 javac{color}. The applied patch generated 1152 javac compiler warnings (more than the trunk's current 1151 warnings). {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.hdfs.server.namenode.TestFileTruncate org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs org.apache.hadoop.hdfs.server.namenode.TestTruncateQuotaUpdate org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength org.apache.hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend org.apache.hadoop.hdfs.server.namenode.TestAddBlock org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing The test build failed in hadoop-hdfs-project/hadoop-hdfs-httpfs Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10032//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/10032//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10032//console This message is automatically generated. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376839#comment-14376839 ] Xiaoyu Yao commented on HDFS-7824: -- Thanks [~arpitagarwal] or the help!. All failures are directly or indirectly caused by missing the new Builder class {code}org.apache.hadoop.fs.ContentSummary$Builder {code} added from this patch. Looks like the jars build for this patch is overwritten by other Jenkins run. I can't repro the failure on my local machine either. {code} The failures are caused by missing Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.ContentSummary$Builder at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) {code} GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376822#comment-14376822 ] Arpit Agarwal commented on HDFS-7824: - The tests pass for me locally but Jenkins flagged some exceptions in getContentSummary. {code} org/apache/hadoop/fs/ContentSummary$Builder at org.apache.hadoop.hdfs.server.namenode.INode.computeAndConvertContentSummary(INode.java:447) at org.apache.hadoop.hdfs.server.namenode.INode.computeContentSummary(INode.java:436) at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.convertLastBlockToUnderConstruction(BlockManager.java:747) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.prepareFileForAppend(FSNamesystem.java:2698) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInternal(FSNamesystem.java:2661) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInt(FSNamesystem.java:2943) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFile(FSNamesystem.java:2914) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.append(NameNodeRpcServer.java:656) {code} GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376829#comment-14376829 ] Arpit Agarwal commented on HDFS-7824: - Also +1 on the v3 patch once Jenkins issues are resolved. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376998#comment-14376998 ] Hadoop QA commented on HDFS-7824: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705760/HDFS-7824.03.patch against trunk revision 2bc097c. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:red}-1 javac{color}. The applied patch generated 1152 javac compiler warnings (more than the trunk's current 1151 warnings). {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.hdfs.server.namenode.TestFileTruncate org.apache.hadoop.hdfs.TestReadWhileWriting org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestRbwSpaceReservation org.apache.hadoop.hdfs.server.namenode.ha.TestQuotasWithHA org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat org.apache.hadoop.fs.viewfs.TestViewFsDefaultValue org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery org.apache.hadoop.hdfs.TestEncryptedTransfer org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser org.apache.hadoop.tracing.TestTracing org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA org.apache.hadoop.hdfs.TestBlockReaderLocalLegacy org.apache.hadoop.hdfs.TestPersistBlocks org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport org.apache.hadoop.hdfs.server.namenode.TestINodeFile org.apache.hadoop.hdfs.server.namenode.TestQuotaByStorageType org.apache.hadoop.hdfs.server.namenode.TestNamenodeRetryCache org.apache.hadoop.hdfs.TestAppendDifferentChecksum org.apache.hadoop.hdfs.TestFileAppend2 org.apache.hadoop.hdfs.TestFileAppend3 org.apache.hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot org.apache.hadoop.hdfs.TestQuota org.apache.hadoop.hdfs.server.namenode.TestSnapshotPathINodes org.apache.hadoop.hdfs.TestRollingUpgrade org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs org.apache.hadoop.hdfs.server.namenode.TestTruncateQuotaUpdate org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength org.apache.hadoop.fs.permission.TestStickyBit org.apache.hadoop.hdfs.TestFileCreation org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode org.apache.hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate org.apache.hadoop.hdfs.server.namenode.TestFSImageWithSnapshot org.apache.hadoop.hdfs.TestPipelines org.apache.hadoop.fs.TestHDFSFileContextMainOperations org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure org.apache.hadoop.hdfs.server.namenode.TestAddBlock org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing The following test timeouts occurred in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshot The test build failed in hadoop-hdfs-project/hadoop-hdfs-httpfs Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10035//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/10035//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10035//console This message is automatically generated. GetContentSummary API and its namenode
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377158#comment-14377158 ] Hadoop QA commented on HDFS-7824: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705760/HDFS-7824.03.patch against trunk revision 972f1f1. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:red}-1 javac{color}. The applied patch generated 1152 javac compiler warnings (more than the trunk's current 1151 warnings). {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.ipc.TestRPCWaitForProxy org.apache.hadoop.tracing.TestTracing org.apache.hadoop.hdfs.server.namenode.TestTruncateQuotaUpdate The test build failed in hadoop-hdfs-project/hadoop-hdfs-httpfs Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10038//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/10038//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10038//console This message is automatically generated. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370536#comment-14370536 ] Hadoop QA commented on HDFS-7824: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705760/HDFS-7824.03.patch against trunk revision 91baca1. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 6 new or modified test files. {color:red}-1 javac{color}. The applied patch generated 1164 javac compiler warnings (more than the trunk's current 1152 warnings). {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.api.TestPBImplRecords org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler org.apache.hadoop.yarn.server.resourcemanager.TestClientRMTokens org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9989//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/9989//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9989//console This message is automatically generated. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch, HDFS-7824.03.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369785#comment-14369785 ] Arpit Agarwal commented on HDFS-7824: - Hi [~xyao], could you please rebase the patch for Jenkins? I am taking a look at it meanwhile. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1437#comment-1437 ] Arpit Agarwal commented on HDFS-7824: - The patch looks great to me. Just nitpicks, +1 otherwise and pending Jenkins. # ContentCounts#types - looks like this is space used by the corresponding storage types. Could you please add a comment? # EnumCounters#asImmutableArray - can we just call it asArray and document it is a shallow copy. Immutable means something else. # ContentSummaryProto#typeQuotaInfos - missing space before equals sign. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369779#comment-14369779 ] Hadoop QA commented on HDFS-7824: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12700363/HDFS-7824.02.patch against trunk revision 61a4c7f. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9981//console This message is automatically generated. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14334769#comment-14334769 ] Hadoop QA commented on HDFS-7824: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12700363/HDFS-7824.02.patch against trunk revision b610c68. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:red}-1 javac{color}. The applied patch generated 1156 javac compiler warnings (more than the trunk's current 1155 warnings). {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat org.apache.hadoop.fs.viewfs.TestViewFsDefaultValue org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery org.apache.hadoop.hdfs.TestEncryptedTransfer org.apache.hadoop.hdfs.TestPersistBlocks org.apache.hadoop.fs.permission.TestStickyBit org.apache.hadoop.hdfs.server.namenode.TestFSImageWithSnapshot org.apache.hadoop.hdfs.TestPipelines org.apache.hadoop.fs.TestHDFSFileContextMainOperations org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure The following test timeouts occurred in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs The test build failed in hadoop-hdfs-project/hadoop-hdfs-httpfs Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9654//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/9654//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9654//console This message is automatically generated. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Fix For: 2.7.0 Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch, HDFS-7824.02.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14333983#comment-14333983 ] Hadoop QA commented on HDFS-7824: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12700226/HDFS-7824.00.patch against trunk revision fe7a302. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:red}-1 javac{color}. The applied patch generated 1156 javac compiler warnings (more than the trunk's current 1155 warnings). {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.fs.TestContentSummary The following test timeouts occurred in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.hdfs.server.namenode.TestNamenodeRetryCache Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9644//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/9644//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9644//console This message is automatically generated. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Fix For: 2.7.0 Attachments: HDFS-7824.00.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7824) GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage
[ https://issues.apache.org/jira/browse/HDFS-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14334354#comment-14334354 ] Hadoop QA commented on HDFS-7824: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12700294/HDFS-7824.01.patch against trunk revision 769507b. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:red}-1 javac{color}. The applied patch generated 1156 javac compiler warnings (more than the trunk's current 1155 warnings). {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.hdfs.server.namenode.TestFileTruncate org.apache.hadoop.fs.http.client.TestHttpFSFileSystemLocalFileSystem Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9646//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/9646//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9646//console This message is automatically generated. GetContentSummary API and its namenode implementaion for Storage Type Quota/Usage - Key: HDFS-7824 URL: https://issues.apache.org/jira/browse/HDFS-7824 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao Fix For: 2.7.0 Attachments: HDFS-7824.00.patch, HDFS-7824.01.patch This JIRA is opened to provide API support of GetContentSummary with storage type quota and usage information. It includes namenode implementation, client namenode RPC protocol and Content.Counts refactoring. It is required by HDFS-7701 (CLI to display storage type quota and usage). -- This message was sent by Atlassian JIRA (v6.3.4#6332)