ZanderXu created HDFS-17473: ------------------------------- Summary: [FGL] Make quota related operations thread-safe Key: HDFS-17473 URL: https://issues.apache.org/jira/browse/HDFS-17473 Project: Hadoop HDFS Issue Type: Sub-task Reporter: ZanderXu Assignee: ZanderXu
Concurrent operations on directory tree may cause Quota updates and verification to not be thread-safe. For example: # Supposing there is directory _/a/b_ and quota is on inode _a_ and _b_ # There are some directories and files under {_}/a/b{_}, such as: {_}/a/b/c/d1{_}, _/a/b/d/f1.txt_ # Supposing there is a create operation under _/a/b/c/d1_ and there is a addBlock operation on _/a/b/d/f1.txt_ # These two operations can be handled concurrently by namenode # They will update the quota on inode a concurrently since these operations just hold the read lock of the inode _a_ and {_}b{_}. # so we should make quota-related thread safe. There are two solutions to make quota-related thread safe。 Solution one: Hold the write lock of the first iNode with Quota set when resolvePath * Directly hold the write lock of iNode _a_ so that all operations involving subtree _/a_ can be handled safety. * Due to lower concurrency, maximum improvements cannot be achieved. * But the implementation is simple and straightforward. Solution two: Lock all QuotaFeatures during quota verification or update * Still hold the read lock of iNode a and b * Lock all QuotaFeatures involved in this operations, when validating or updating quotas. * Maximum improvements can be achieved. * But the implementation is a little complex ** Add a lock for each QuotaFeature ** Acquire locks for all involving QuotaFeature -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org