[
https://issues.apache.org/jira/browse/HBASE-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441029#comment-16441029
]
Wei-Chiu Chuang edited comment on HBASE-20433 at 4/17/18 3:34 PM:
------------------------------------------------------------------
Hi Voyta,
The reason for not caching FileSystem object is explained in HADOOP-13971. The
internal design causes memory leak and has resulted in quite a few tickets in
other projects.
Yes, therefore the responsibility falls on to the application (like HBase) to
manage the FileSystem object & close when appropriate.
Do you have a heapdump or profiler that shows fs object occupies memory in your
ExportSnapshot?
was (Author: jojochuang):
Hi Voyta,
The reason for not caching FileSystem object is explained in HADOOP-13971. The
internal design causes memory leak and has resulted in quite a few tickets in
other projects.
Yes, therefore the responsibility falls on to the application (like HBase) to
manage the FileSystem object & close when appropriate.
> HBase Export Snapshot utility does not close FileSystem instances
> -----------------------------------------------------------------
>
> Key: HBASE-20433
> URL: https://issues.apache.org/jira/browse/HBASE-20433
> Project: HBase
> Issue Type: Bug
> Components: Client, fs, snapshots
> Affects Versions: 1.2.6, 1.4.3
> Reporter: Voyta
> Priority: Major
>
> It seems org.apache.hadoop.hbase.snapshot.ExportSnapshot disallows FileSystem
> instance caching.
> When verifySnapshot method is being run it calls often methods like
> org.apache.hadoop.hbase.util.FSUtils#getRootDir that instantiate FileSystem
> but never calls org.apache.hadoop.fs.FileSystem#close method. This behaviour
> allows allocation of unwanted objects potentially causing memory leaks.
> Related issue: https://issues.apache.org/jira/browse/HADOOP-15392
>
> Expectation:
> * HBase should properly release/close all objects, especially FileSystem
> instances.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
