[
https://issues.apache.org/jira/browse/HDFS-12748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226859#comment-16226859
]
Daryn Sharp commented on HDFS-12748:
------------------------------------
No. Absolutely not. While it "fixes" the reported issue, it causes the
filesystem to always be removed from the cache which is wrong in the general
case.
The real problem is the NN should _never_ be using a filesystem instance for
itself. In this case, it's creating a filesystem just to return /user/blah.
The GETHOMEDIRECTORY method can just construct the string w/o creating any
filesystem instances.
> NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY
> ------------------------------------------------------------
>
> Key: HDFS-12748
> URL: https://issues.apache.org/jira/browse/HDFS-12748
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
> Affects Versions: 2.8.2
> Reporter: Jiandan Yang
> Assignee: Weiwei Yang
> Attachments: HDFS-12748.001.patch
>
>
> In our production environment, the standby NN often do fullgc, through mat we
> found the largest object is FileSystem$Cache, which contains 7,844,890
> DistributedFileSystem.
> By view hierarchy of method FileSystem.get() , I found only
> NamenodeWebHdfsMethods#get call FileSystem.get(). I don't know why creating
> different DistributedFileSystem every time instead of get a FileSystem from
> cache.
> {code:java}
> case GETHOMEDIRECTORY: {
> final String js = JsonUtil.toJsonString("Path",
> FileSystem.get(conf != null ? conf : new Configuration())
> .getHomeDirectory().toUri().getPath());
> return Response.ok(js).type(MediaType.APPLICATION_JSON).build();
> }
> {code}
> When we close FileSystem when GETHOMEDIRECTORY, NN don't do fullgc.
> {code:java}
> case GETHOMEDIRECTORY: {
> FileSystem fs = null;
> try {
> fs = FileSystem.get(conf != null ? conf : new Configuration());
> final String js = JsonUtil.toJsonString("Path",
> fs.getHomeDirectory().toUri().getPath());
> return Response.ok(js).type(MediaType.APPLICATION_JSON).build();
> } finally {
> if (fs != null) {
> fs.close();
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
