Jinglun created HDFS-14547:
------------------------------
Summary: DirectoryWithQuotaFeature.quota costs additional memory
even the storage type quota is not set.
Key: HDFS-14547
URL: https://issues.apache.org/jira/browse/HDFS-14547
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 3.1.0
Reporter: Jinglun
Our XiaoMi HDFS is considering upgrading from 2.6 to 3.1. We notice the storage
type quota 'tsCounts' is instantiated to
EnumCounters<StorageType>(StorageType.class), so it will cost a long[5] even if
we don't have any storage type quota on this
inode(StorageType.class.getEnumConstants() == 5).
In our cluster we have many dirs with quota and the NameNode's memory is in
tension, so the additional cost will be a problem.
See DirectoryWithQuotaFeature.Builder().
{code:java}
class DirectoryWithQuotaFeature$Builder {
public Builder() {
this.quota = new QuotaCounts.Builder().nameSpace(DEFAULT_NAMESPACE_QUOTA).
storageSpace(DEFAULT_STORAGE_SPACE_QUOTA).
typeSpaces(DEFAULT_STORAGE_SPACE_QUOTA).build();// set default value -1.
this.usage = new QuotaCounts.Builder().nameSpace(1).build();
}
public Builder typeSpaces(long val) {// set default value.
this.tsCounts.reset(val);
return this;
}
}
class QuotaCounts$Builder {
public Builder() {
this.nsSsCounts = new EnumCounters<Quota>(Quota.class);
this.tsCounts = new EnumCounters<StorageType>(StorageType.class);
}
}
class EnumCounters {
public EnumCounters(final Class<E> enumClass) {
final E[] enumConstants = enumClass.getEnumConstants();
Preconditions.checkNotNull(enumConstants);
this.enumClass = enumClass;
this.counters = new long[enumConstants.length];// new a long array here.
}
}
{code}
Related to HDFS-14542.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
