[
https://issues.apache.org/jira/browse/HBASE-21551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710954#comment-16710954
]
Zheng Hu commented on HBASE-21551:
----------------------------------
Made a stupid mistake when committing, I add more asserts in the committed UT
[1]. here I close the region which result in the failure of other cases in the
same UT. so I commit an addendum[2]. Sorry for the noise. (patch for branch-2
no need the addendum, because I merged them into one)
1.
https://github.com/apache/hbase/commit/3b854859f6fad44cbf31164374569a6ab23f3623#diff-801daeaf9f3c8ddb85e743c06d79c7edR142
2.
https://github.com/apache/hbase/commit/67ab8b888f8b393979624a2bd7d527fefd9dd6d7
> Memory leak when use scan with STREAM at server side
> ----------------------------------------------------
>
> Key: HBASE-21551
> URL: https://issues.apache.org/jira/browse/HBASE-21551
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Zheng Hu
> Assignee: Zheng Hu
> Priority: Blocker
> Fix For: 3.0.0, 2.2.0, 2.1.2, 2.0.4
>
> Attachments: HBASE-21551.v1.patch, HBASE-21551.v2.patch, heap-dump.jpg
>
>
> We open the RegionServerScanner with STREAM as following:
> {code}
> RegionScannerImpl#initializeScanners
> |---> HStore#getScanner
> |----------> StoreScanner()
> |------->
> StoreFileScanner#getScannersForStoreFiles
> |------>
> HStoreFile#getStreamScanner #1
> {code}
> In #1, we put the StoreFileReader into a concurrent hash map streamReaders,
> but not remove the StreamReader from streamReaders until closing the store
> file.
> So if we scan with stream with so many times, the streamReaders hash map
> will be exploded. we can see the heap dump in the attached heap-dump.jpg.
> I found this bug, because when i benchmark the scan performance by using YCSB
> in a cluster (heap size of RS is 50g), the Rs was easy to occur a long time
> full gc ( ~ 110 sec)....
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
