Stanislav Matytsin created HDFS-17379:
-----------------------------------------
Summary: setQuota/setSpaceQuota command doesn't fail if the
directory would immediately exceed the new quota
Key: HDFS-17379
URL: https://issues.apache.org/jira/browse/HDFS-17379
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs
Affects Versions: 3.1.3
Reporter: Stanislav Matytsin
With regards to documentation -
[https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html#Space_Quotas]
* hdfs dfsadmin -setSpaceQuota <N> <directory>...<directory>
Set the space quota to be N bytes for each directory. This is a hard limit on
total size of all the files under the directory tree. The space quota takes
replication also into account, i.e. one GB of data with replication of 3
consumes 3GB of quota. N can also be specified with a binary prefix for
convenience, for e.g. 50g for 50 gigabytes and 2t for 2 terabytes etc. Best
effort for each directory, with faults reported if N is neither zero nor a
positive integer, the directory does not exist or it is a file, or the
directory would immediately exceed the new quota.
So, command to set Quota have to fail if quota already exceeded
Actual result – command finished with exit code 0
Steps to reproduce:
- put file 1GB into hdfs folder
- try to set quota of 512MB
hdfs dfs -ls /hdfs_dir
Exit code: 0
Found 1 items
- 3 user hadoop 1073741824 2024-01-17 18:57 /hdfs_dir/File
hdfs dfsadmin -setSpaceQuota 512m /hdfs_dir
result:
Exit code: 0
Quota report shows negative value for SPACE_REM_QUOTA
{code:java}
REM_SPACE_QUOTA -2.5GB
QUOTA REM_QUOTA SPACE_QUOTA REM_SPACE_QUOTA DIR_COUNT FILE_COUNT
CONTENT_SIZE PATHNAME
none inf 512 M -2.5 G 1
1 1 G /hdfs_dir
{code}
The same situation with setQuota method, if we try to set name quota, for
example, 3 for folder already contained 5 files – it doesn’t fail and shows
negative value for REM_QUOTA
The reason for this behavior is absent quota check during SET operations
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
