[ https://issues.apache.org/jira/browse/HBASE-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574347#comment-16574347 ]
Yu Li commented on HBASE-21029: ------------------------------- +1, we should always increase heap size when data size increases. Minor: store the {{succ || mslabUsed}} result to avoid multiple check? > Miscount of memstore's heap/offheap size if same cell was put > ------------------------------------------------------------- > > Key: HBASE-21029 > URL: https://issues.apache.org/jira/browse/HBASE-21029 > Project: HBase > Issue Type: Bug > Affects Versions: 2.1.0, 2.0.1 > Reporter: Allan Yang > Assignee: Allan Yang > Priority: Major > Attachments: HBASE-21029.branch-2.0.001.patch > > > We are now using memstore.heapSize() + memstore.offheapSize() to decide > whether a flush is needed. But, if a same cell was put again in memstore, > only the memstore's dataSize will be increased, the heap/offheap size won't. > We encountered the problem that a user was putting the same kv again and > again, but the memstore won't flush since the heap size was not counted > properly. The RS was killed by system since not enough memory in the end. > Actually, if MSLAB is used, the heap/offheap will increase no matter the cell > is added or not. IIRC, memstore's heap/offheap size should always bigger than > data size. We introduced heap/offheap size besides data size in order to > reflect memory footprint more precisely. > {code} > // If there's already a same cell in the CellSet and we are using MSLAB, > we must count in the > // MSLAB allocation size as well, or else there will be memory leak > (occupied heap size larger > // than the counted number) > if (succ || mslabUsed) { > cellSize = getCellLength(cellToAdd); > } > // heap/offheap size is changed only if the cell is truly added in the > cellSet > long heapSize = heapSizeChange(cellToAdd, succ); > long offHeapSize = offHeapSizeChange(cellToAdd, succ); > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)