[
https://issues.apache.org/jira/browse/HDFS-14633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923449#comment-16923449
]
He Xiaoqiao commented on HDFS-14633:
------------------------------------
hi [~xyao],[~LiJinglun]. Sorry for late response, just check
TestQuota#testClrQuotaOnRoot failed in branch-2 cause it doesn't support
traditional binary prefix, And I try to dig commit source, and meet the
following commit info.
{code:java}
commit a524608d1e0a276fd822e9574f0a4a6d51298b13
Author: Xiaoyu Yao <x...@apache.org>
Date: Fri Aug 30 16:46:04 2019 -0700
HDFS-14633. The StorageType quota and consume in QuotaFeature is not
handled for rename. Contributed by Jinglun.
(cherry picked from commit 62d71fbac3789c7d484bc76ced9ec7fa6ff94de1)
{code}
I am confused that patch v007 source code is very different with commit for
branch-2. is there anything I missed? or commit message is not match with code.
Another way I do not trace back to `62d71fbac3789c7d484bc76ced9ec7fa6ff94de1`
which is mark as cherrypick source. where is it?:)
After check carefully, I think we could fix failed unittest
`TestQuota#testClrQuotaOnRoot` with cherrypick HDFS-12009. Any suggestions?
Thanks.
> The StorageType quota and consume in QuotaFeature is not handled for rename
> ---------------------------------------------------------------------------
>
> Key: HDFS-14633
> URL: https://issues.apache.org/jira/browse/HDFS-14633
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Jinglun
> Assignee: Jinglun
> Priority: Major
> Fix For: 2.10.0, 3.0.4, 3.3.0, 2.8.6, 3.2.1, 2.9.3, 3.1.3
>
> Attachments: HDFS-14633-testcases-explanation, HDFS-14633.002.patch,
> HDFS-14633.003.patch, HDFS-14633.004.patch, HDFS-14633.005.patch,
> HDFS-14633.006.patch, HDFS-14633.007.patch
>
>
> The NameNode manages the global state of the cluster. We should always take
> NameNode's records as the sole criterion because no matter what inconsistent
> is the NameNode should finally make everything right based on it's records.
> Let's call it rule NSC(NameNode is the Sole Criterion). That means when we do
> all quota related rpcs, we do the quota check according to NameNode's records
> regardless of any inconsistent situation, such as the replicas doesn't match
> the storage policy of the file, or the replicas count doesn't match the
> file's set replica.
> The work SPS deals with the wrongly palced replicas. There is a thought
> about putting off the consume update of the DirectoryQuota until all replicas
> are re-placed by SPS. I can't agree that because if we do so we will abandon
> letting the NameNode's records to be the sole criterion. The block
> replication is a good example of the rule NSC. When we count the consume of a
> file(CONTIGUOUS), we multiply the replication factor with the file's length,
> no matter the blocks are under replicated or excessed. We should do the same
> thing for the storage type quota.
> Another concern is the change will let setStoragePolicy throw
> QuotaByStorageTypeExceededException which it doesn't before. I don't think
> it's a big problem since the setStoragePolicy already throws IOException. Or
> we can wrap the QuotaByStorageTypeExceededException in an IOException, but I
> won't recommend that because it's ugly.
> To make storage type consume follow the rule NSC, we need change
> rename(moving a file with storage policy inherited from it's parent) and
> setStoragePolicy.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
