[
https://issues.apache.org/jira/browse/HBASE-26488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26488:
-----------------------------
Description:
{{HStore.flushCache}} use following {{DefaultStoreFlusher.flushSnapshot}} to
flush memStore and retry this method when this method throws exception,but
increasing reference count of {{MemStoreLABImpl}} in following line 55 and
decreasing reference count of {{MemStoreLABImpl}} in following line 82 is
asymmetrical：
{code:java}
47 public List<Path> flushSnapshot(MemStoreSnapshot snapshot, long cacheFlushId,
48 MonitoredTask status, ThroughputController throughputController,
49 FlushLifeCycleTracker tracker) throws IOException {
50 ArrayList<Path> result = new ArrayList<>();
51 int cellsCount = snapshot.getCellsCount();
52 if (cellsCount == 0) return result; // don't flush if there are no entries
53
54 // Use a store scanner to find which rows to flush.
55 InternalScanner scanner = createScanner(snapshot.getScanners(), tracker);
56 StoreFileWriter writer;
57 try {
58 // TODO: We can fail in the below block before we complete adding this
flush to
59 // list of store files. Add cleanup of anything put on
filesystem if we fail.
60 synchronized (flushLock) {
61 status.setStatus("Flushing " + store + ": creating writer");
62 // Write the map out to the disk
63 writer = store.createWriterInTmp(cellsCount,
64 store.getColumnFamilyDescriptor().getCompressionType(), false,
true,
65 snapshot.isTagsPresent(), false);
66 IOException e = null;
67 try {
68 performFlush(scanner, writer, throughputController);
69 } catch (IOException ioe) {
70 e = ioe;
71 // throw the exception out
72 throw ioe;
73 } finally {
74 if (e != null) {
75 writer.close();
76 } else {
77 finalizeWriter(writer, cacheFlushId, status);
78 }
79 }
80 }
81 } finally {
82 scanner.close();
83 }
......
{code}
The asymmetry because above line 55 {{snapshot.getScanners}} ,which is
following {{MemStoreSnapshot.getScanners}}：
{code:java}
public List<KeyValueScanner> getScanners() {
return scanners;
}
{code}
{{MemStoreSnapshot.getScanners}} not create new {{SegmentScanner}}, and reuse
the {{SegmentScanner}}
was:
{{HStore.flushCache}} use following {{DefaultStoreFlusher.flushSnapshot}} to
flush memStore and retry this method when this method throws exception,but
increasing reference count of {{MemStoreLABImpl}} in following line 55 and
decreasing reference count of {{MemStoreLABImpl}} in following line 82 is not
symmetrical:
{code:java}
47 public List<Path> flushSnapshot(MemStoreSnapshot snapshot, long cacheFlushId,
48 MonitoredTask status, ThroughputController throughputController,
49 FlushLifeCycleTracker tracker) throws IOException {
50 ArrayList<Path> result = new ArrayList<>();
51 int cellsCount = snapshot.getCellsCount();
52 if (cellsCount == 0) return result; // don't flush if there are no entries
53
54 // Use a store scanner to find which rows to flush.
55 InternalScanner scanner = createScanner(snapshot.getScanners(), tracker);
56 StoreFileWriter writer;
57 try {
58 // TODO: We can fail in the below block before we complete adding this
flush to
59 // list of store files. Add cleanup of anything put on
filesystem if we fail.
60 synchronized (flushLock) {
61 status.setStatus("Flushing " + store + ": creating writer");
62 // Write the map out to the disk
63 writer = store.createWriterInTmp(cellsCount,
64 store.getColumnFamilyDescriptor().getCompressionType(), false,
true,
65 snapshot.isTagsPresent(), false);
66 IOException e = null;
67 try {
68 performFlush(scanner, writer, throughputController);
69 } catch (IOException ioe) {
70 e = ioe;
71 // throw the exception out
72 throw ioe;
73 } finally {
74 if (e != null) {
75 writer.close();
76 } else {
77 finalizeWriter(writer, cacheFlushId, status);
78 }
79 }
80 }
81 } finally {
82 scanner.close();
83 }
......
{code}
> Memory leak when MemStore retrying
> ----------------------------------
>
> Key: HBASE-26488
> URL: https://issues.apache.org/jira/browse/HBASE-26488
> Project: HBase
> Issue Type: Bug
> Affects Versions: 3.0.0-alpha-1, 2.4.8
> Reporter: chenglei
> Assignee: chenglei
> Priority: Major
> Labels: regionserver
>
> {{HStore.flushCache}} use following {{DefaultStoreFlusher.flushSnapshot}} to
> flush memStore and retry this method when this method throws exception,but
> increasing reference count of {{MemStoreLABImpl}} in following line 55 and
> decreasing reference count of {{MemStoreLABImpl}} in following line 82 is
> asymmetrical：
> {code:java}
> 47 public List<Path> flushSnapshot(MemStoreSnapshot snapshot, long
> cacheFlushId,
> 48 MonitoredTask status, ThroughputController throughputController,
> 49 FlushLifeCycleTracker tracker) throws IOException {
> 50 ArrayList<Path> result = new ArrayList<>();
> 51 int cellsCount = snapshot.getCellsCount();
> 52 if (cellsCount == 0) return result; // don't flush if there are no
> entries
> 53
> 54 // Use a store scanner to find which rows to flush.
> 55 InternalScanner scanner = createScanner(snapshot.getScanners(),
> tracker);
> 56 StoreFileWriter writer;
> 57 try {
> 58 // TODO: We can fail in the below block before we complete adding
> this flush to
> 59 // list of store files. Add cleanup of anything put on
> filesystem if we fail.
> 60 synchronized (flushLock) {
> 61 status.setStatus("Flushing " + store + ": creating writer");
> 62 // Write the map out to the disk
> 63 writer = store.createWriterInTmp(cellsCount,
> 64 store.getColumnFamilyDescriptor().getCompressionType(), false,
> true,
> 65 snapshot.isTagsPresent(), false);
> 66 IOException e = null;
> 67 try {
> 68 performFlush(scanner, writer, throughputController);
> 69 } catch (IOException ioe) {
> 70 e = ioe;
> 71 // throw the exception out
> 72 throw ioe;
> 73 } finally {
> 74 if (e != null) {
> 75 writer.close();
> 76 } else {
> 77 finalizeWriter(writer, cacheFlushId, status);
> 78 }
> 79 }
> 80 }
> 81 } finally {
> 82 scanner.close();
> 83 }
> ......
> {code}
> The asymmetry because above line 55 {{snapshot.getScanners}} ,which is
> following {{MemStoreSnapshot.getScanners}}：
> {code:java}
> public List<KeyValueScanner> getScanners() {
> return scanners;
> }
> {code}
> {{MemStoreSnapshot.getScanners}} not create new {{SegmentScanner}}, and reuse
> the {{SegmentScanner}}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
