[ https://issues.apache.org/jira/browse/HDFS-14633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919988#comment-16919988 ]
Xiaoyu Yao commented on HDFS-14633: ----------------------------------- Thanks [~LiJinglun] for the update. The patch v7 LGTM, +1. I will commit it during the weekend if there is no other comments. > The StorageType quota and consume in QuotaFeature is not handled for rename > --------------------------------------------------------------------------- > > Key: HDFS-14633 > URL: https://issues.apache.org/jira/browse/HDFS-14633 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Jinglun > Assignee: Jinglun > Priority: Major > Attachments: HDFS-14633-testcases-explanation, HDFS-14633.002.patch, > HDFS-14633.003.patch, HDFS-14633.004.patch, HDFS-14633.005.patch, > HDFS-14633.006.patch, HDFS-14633.007.patch > > > The NameNode manages the global state of the cluster. We should always take > NameNode's records as the sole criterion because no matter what inconsistent > is the NameNode should finally make everything right based on it's records. > Let's call it rule NSC(NameNode is the Sole Criterion). That means when we do > all quota related rpcs, we do the quota check according to NameNode's records > regardless of any inconsistent situation, such as the replicas doesn't match > the storage policy of the file, or the replicas count doesn't match the > file's set replica. > The work SPS deals with the wrongly palced replicas. There is a thought > about putting off the consume update of the DirectoryQuota until all replicas > are re-placed by SPS. I can't agree that because if we do so we will abandon > letting the NameNode's records to be the sole criterion. The block > replication is a good example of the rule NSC. When we count the consume of a > file(CONTIGUOUS), we multiply the replication factor with the file's length, > no matter the blocks are under replicated or excessed. We should do the same > thing for the storage type quota. > Another concern is the change will let setStoragePolicy throw > QuotaByStorageTypeExceededException which it doesn't before. I don't think > it's a big problem since the setStoragePolicy already throws IOException. Or > we can wrap the QuotaByStorageTypeExceededException in an IOException, but I > won't recommend that because it's ugly. > To make storage type consume follow the rule NSC, we need change > rename(moving a file with storage policy inherited from it's parent) and > setStoragePolicy. -- This message was sent by Atlassian Jira (v8.3.2#803003) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org