[
https://issues.apache.org/jira/browse/HDFS-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394755#comment-16394755
]
He Xiaoqiao commented on HDFS-13234:
------------------------------------
Thanks [~kihwal] for your detailed comments.
It is interesting issues (HADOOP-11223 and HADOOP-9570) for resolving
duplicated Configuration instances. But I am not sure if these issue are
complete solution for huge memory footprint waste of the case mentioned above.
Beside HADOOP-11223 and HADOOP-9570, I think it is necessary to maintain
incremental change for Configuration, thus Configuration::getDefault() +
Incremental Change could form the complete configuration and no unintended conf
update propagation, meaning while it could reduce memory footprint. If I am
wrong please correct me.
Thanks again.
> Remove renew configuration instance in ConfiguredFailoverProxyProvider and
> reduce memory footprint for client
> -------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-13234
> URL: https://issues.apache.org/jira/browse/HDFS-13234
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: fs, ha, hdfs-client
> Reporter: He Xiaoqiao
> Priority: Major
> Attachments: HDFS-13234.001.patch
>
>
> The memory footprint of #DFSClient is very considerable in some special
> scenario since there are many #Configuration instances and occupy much memory
> resource (In an extreme case, org.apache.hadoop.conf.Configuration occupies
> over 600MB we meet under HDFS Federation an HA with QJM and there are dozens
> of NameNodes). I think some new Configuration instance is not necessary. Such
> as #ConfiguredFailoverProxyProvider initialization.
> {code:java}
> public ConfiguredFailoverProxyProvider(Configuration conf, URI uri,
> Class<T> xface, HAProxyFactory<T> factory) {
> this.xface = xface;
> this.conf = new Configuration(conf);
> ......
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
