[
https://issues.apache.org/jira/browse/HBASE-26336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault updated HBASE-26336:
--------------------------------------
Description:
I'm working on HDFS-16262, in which we add tracking of DFSInputStreams. The
DFSInputStreams are registered on creation and de-registered on call to
close(). As part of testing I noticed that extra DFSInputStreams were being
created and never closed. I tracked it down to HBASE-15441 in which the
r.close() was removed here:
[https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L7480]
This has not been a huge issue because DFSInputStreams don't currently keep
resources unless you do a non-positional read or use extended read buffers. So
for the warmup case, the DFSInputStream is created and then eventually GC'd.
That said, the contract of a Closeable is typically that you should close them
to release resources. So we're kind of lucking out that it's not strictly
necessary until you do a read, and that could change at any time (like in
HDFS-16262).
I think we should either add back a call to r.close(), or maybe it's possible
to skip creating the Reader altogether for warmup. It doesn't look like the
reader itself is necessary for warmup, but I could be missing something.
was:
I'm working on HDFS-16262, in which we add tracking of DFSInputStreams. The
DFSInputStreams are registered on creation and de-registered on call to
close(). As part of testing I noticed that extra DFSInputStreams were being
created and never closed. I tracked it down to
https://issues.apache.org/jira/browse/HBASE-15441, in which the r.close() was
removed here:
[https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L7480]
This has not been a huge issue because DFSInputStreams don't currently keep
resources unless you do a non-positional read or use extended read buffers. So
for the warmup case, the DFSInputStream is created and then eventually GC'd.
That said, the contract of a Closeable is typically that you should close them
to release resources. So we're kind of lucking out that it's not strictly
necessary until you do a read, and that could change at any time (like in
HDFS-16262).
I think we should either add back a call to r.close(), or maybe it's possible
to skip creating the Reader altogether for warmup. It doesn't look like the
reader itself is necessary for warmup, but I could be missing something.
> RegionServer leaks DFSInputStreams due to not closing region after warmup
> -------------------------------------------------------------------------
>
> Key: HBASE-26336
> URL: https://issues.apache.org/jira/browse/HBASE-26336
> Project: HBase
> Issue Type: Bug
> Reporter: Bryan Beaudreault
> Assignee: Bryan Beaudreault
> Priority: Minor
>
> I'm working on HDFS-16262, in which we add tracking of DFSInputStreams. The
> DFSInputStreams are registered on creation and de-registered on call to
> close(). As part of testing I noticed that extra DFSInputStreams were being
> created and never closed. I tracked it down to HBASE-15441 in which the
> r.close() was removed here:
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L7480]
> This has not been a huge issue because DFSInputStreams don't currently keep
> resources unless you do a non-positional read or use extended read buffers.
> So for the warmup case, the DFSInputStream is created and then eventually
> GC'd.
> That said, the contract of a Closeable is typically that you should close
> them to release resources. So we're kind of lucking out that it's not
> strictly necessary until you do a read, and that could change at any time
> (like in HDFS-16262).
> I think we should either add back a call to r.close(), or maybe it's possible
> to skip creating the Reader altogether for warmup. It doesn't look like the
> reader itself is necessary for warmup, but I could be missing something.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
