[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[ https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039861#comment-14039861 ] Hudson commented on HDFS-6535: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1808 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1808/]) HDFS-6535. HDFS quota update is wrong when file is appended. Contributed by George Wong. (jing9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604226) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java > HDFS quota update is wrong when file is appended > > > Key: HDFS-6535 > URL: https://issues.apache.org/jira/browse/HDFS-6535 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.4.0 >Reporter: George Wong >Assignee: George Wong > Fix For: 2.5.0 > > Attachments: HDFS-6535.patch, HDFS-6535_v1.patch, TestHDFSQuota.java > > > when a file in the directory with Quota feature is appended, the cached disk > consumption should be updated. > But currently, the update is wrong. > Use the uploaded UT to reproduce this bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[ https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039837#comment-14039837 ] Hudson commented on HDFS-6535: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1781 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1781/]) HDFS-6535. HDFS quota update is wrong when file is appended. Contributed by George Wong. (jing9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604226) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java > HDFS quota update is wrong when file is appended > > > Key: HDFS-6535 > URL: https://issues.apache.org/jira/browse/HDFS-6535 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.4.0 >Reporter: George Wong >Assignee: George Wong > Fix For: 2.5.0 > > Attachments: HDFS-6535.patch, HDFS-6535_v1.patch, TestHDFSQuota.java > > > when a file in the directory with Quota feature is appended, the cached disk > consumption should be updated. > But currently, the update is wrong. > Use the uploaded UT to reproduce this bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[ https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039777#comment-14039777 ] Hudson commented on HDFS-6535: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #590 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/590/]) HDFS-6535. HDFS quota update is wrong when file is appended. Contributed by George Wong. (jing9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604226) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java > HDFS quota update is wrong when file is appended > > > Key: HDFS-6535 > URL: https://issues.apache.org/jira/browse/HDFS-6535 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.4.0 >Reporter: George Wong >Assignee: George Wong > Fix For: 2.5.0 > > Attachments: HDFS-6535.patch, HDFS-6535_v1.patch, TestHDFSQuota.java > > > when a file in the directory with Quota feature is appended, the cached disk > consumption should be updated. > But currently, the update is wrong. > Use the uploaded UT to reproduce this bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[ https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039158#comment-14039158 ] Hudson commented on HDFS-6535: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5743 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5743/]) HDFS-6535. HDFS quota update is wrong when file is appended. Contributed by George Wong. (jing9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604226) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java > HDFS quota update is wrong when file is appended > > > Key: HDFS-6535 > URL: https://issues.apache.org/jira/browse/HDFS-6535 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.4.0 >Reporter: George Wong >Assignee: George Wong > Fix For: 2.5.0 > > Attachments: HDFS-6535.patch, HDFS-6535_v1.patch, TestHDFSQuota.java > > > when a file in the directory with Quota feature is appended, the cached disk > consumption should be updated. > But currently, the update is wrong. > Use the uploaded UT to reproduce this bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[ https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039134#comment-14039134 ] Jing Zhao commented on HDFS-6535: - The patch looks good to me. +1. I will commit it shortly. > HDFS quota update is wrong when file is appended > > > Key: HDFS-6535 > URL: https://issues.apache.org/jira/browse/HDFS-6535 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.4.0 >Reporter: George Wong >Assignee: George Wong > Attachments: HDFS-6535.patch, HDFS-6535_v1.patch, TestHDFSQuota.java > > > when a file in the directory with Quota feature is appended, the cached disk > consumption should be updated. > But currently, the update is wrong. > Use the uploaded UT to reproduce this bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[
https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038754#comment-14038754
]
Hadoop QA commented on HDFS-6535:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12651642/HDFS-6535_v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7188//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7188//console
This message is automatically generated.
> HDFS quota update is wrong when file is appended
>
>
> Key: HDFS-6535
> URL: https://issues.apache.org/jira/browse/HDFS-6535
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.4.0
>Reporter: George Wong
>Assignee: George Wong
> Attachments: HDFS-6535.patch, HDFS-6535_v1.patch, TestHDFSQuota.java
>
>
> when a file in the directory with Quota feature is appended, the cached disk
> consumption should be updated.
> But currently, the update is wrong.
> Use the uploaded UT to reproduce this bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[ https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038641#comment-14038641 ] George Wong commented on HDFS-6535: --- [~jingzhao], Could you please help me to review the updated patch? Thanks a lot. > HDFS quota update is wrong when file is appended > > > Key: HDFS-6535 > URL: https://issues.apache.org/jira/browse/HDFS-6535 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.4.0 >Reporter: George Wong >Assignee: George Wong > Attachments: HDFS-6535.patch, HDFS-6535_v1.patch, TestHDFSQuota.java > > > when a file in the directory with Quota feature is appended, the cached disk > consumption should be updated. > But currently, the update is wrong. > Use the uploaded UT to reproduce this bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[ https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038640#comment-14038640 ] George Wong commented on HDFS-6535: --- [~jingzhao], when I am working on this issue, I find another issue about Quota update. Here is the filed jira https://issues.apache.org/jira/browse/HDFS-6575 I think I may give a patch later. > HDFS quota update is wrong when file is appended > > > Key: HDFS-6535 > URL: https://issues.apache.org/jira/browse/HDFS-6535 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.4.0 >Reporter: George Wong >Assignee: George Wong > Attachments: HDFS-6535.patch, HDFS-6535_v1.patch, TestHDFSQuota.java > > > when a file in the directory with Quota feature is appended, the cached disk > consumption should be updated. > But currently, the update is wrong. > Use the uploaded UT to reproduce this bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[ https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037193#comment-14037193 ] George Wong commented on HDFS-6535: --- Thanks Jing for your reply. OK. I will add more UT to address your comment. > HDFS quota update is wrong when file is appended > > > Key: HDFS-6535 > URL: https://issues.apache.org/jira/browse/HDFS-6535 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.4.0 >Reporter: George Wong >Assignee: George Wong > Attachments: HDFS-6535.patch, TestHDFSQuota.java > > > when a file in the directory with Quota feature is appended, the cached disk > consumption should be updated. > But currently, the update is wrong. > Use the uploaded UT to reproduce this bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[ https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14036000#comment-14036000 ] Jing Zhao commented on HDFS-6535: - Hi [~GWong], thanks for working on this. This is actually a bug from HDFS-6423. Your patch looks good to me. The only comment is that maybe we can move all the new unit tests into TestDiskspaceQuotaUpdate.java, where we should also change the number of DN and the replication factor to some number greater than 1. > HDFS quota update is wrong when file is appended > > > Key: HDFS-6535 > URL: https://issues.apache.org/jira/browse/HDFS-6535 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.4.0 >Reporter: George Wong > Attachments: HDFS-6535.patch, TestHDFSQuota.java > > > when a file in the directory with Quota feature is appended, the cached disk > consumption should be updated. > But currently, the update is wrong. > Use the uploaded UT to reproduce this bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[ https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14035513#comment-14035513 ] George Wong commented on HDFS-6535: --- Hi [~brandonli], would you mind to assign this jira to me ? I am happy to follow up the fix of this. And could you review my patch? It is quite simple. Looking forward to your help. Thanks a lot. > HDFS quota update is wrong when file is appended > > > Key: HDFS-6535 > URL: https://issues.apache.org/jira/browse/HDFS-6535 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.4.0 >Reporter: George Wong > Attachments: HDFS-6535.patch, TestHDFSQuota.java > > > when a file in the directory with Quota feature is appended, the cached disk > consumption should be updated. > But currently, the update is wrong. > Use the uploaded UT to reproduce this bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6535) HDFS quota update is wrong when file is appended
[ https://issues.apache.org/jira/browse/HDFS-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031785#comment-14031785 ] George Wong commented on HDFS-6535: --- When appending a file, namenode does not update the disk consumption for the last block in the file. So, the cached disk consumption is less than the actual disk consumption. I am going to format a patch later. I have a fixed in my local branch currently. > HDFS quota update is wrong when file is appended > > > Key: HDFS-6535 > URL: https://issues.apache.org/jira/browse/HDFS-6535 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.4.0 >Reporter: George Wong > Attachments: TestHDFSQuota.java > > > when a file in the directory with Quota feature is appended, the cached disk > consumption should be updated. > But currently, the update is wrong. > Use the uploaded UT to reproduce this bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
