[
https://issues.apache.org/jira/browse/HBASE-26210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17409005#comment-17409005
]
Hudson commented on HBASE-26210:
--------------------------------
Results for branch branch-2.3
[build #285 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/285/]:
(x) *{color:red}-1 overall{color}*
----
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/285/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/285/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/285/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/285/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize
> for CompactingMemStore
> ----------------------------------------------------------------------------------------------------
>
> Key: HBASE-26210
> URL: https://issues.apache.org/jira/browse/HBASE-26210
> Project: HBase
> Issue Type: Bug
> Components: in-memory-compaction
> Affects Versions: 3.0.0-alpha-1, 2.4.5
> Reporter: chenglei
> Assignee: chenglei
> Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.6, 2.3.7
>
>
> Besides HBASE-26026, there is another case HBase Write would be hang when
> using {{CompactingMemstore}} :
> The problem is caused by {{CompactingMemStore.checkAndAddToActiveSize}} and
> {{CompactingMemStore.shouldFlushInMemory}} :
> {code:java}
> 425 private boolean checkAndAddToActiveSize(MutableSegment currActive, Cell
> cellToAdd,
> 426 MemStoreSizing memstoreSizing) {
> 427 if (shouldFlushInMemory(currActive, cellToAdd, memstoreSizing)) {
> 428 if (currActive.setInMemoryFlushed()) {
> 429 flushInMemory(currActive);
> 430 if (setInMemoryCompactionFlag()) {
> 431 // The thread is dispatched to do in-memory compaction in the
> background
> ......
> }
> {code}
> {{CompactingMemStore.checkAndAddToActiveSize}} invokes following
> {{CompactingMemStore.shouldFlushInMemory}} firstly , when the size of the
> {{cellToAdd}} is larger than {{inmemoryFlushSize}}(the default value is 2MB),
> this {{CompactingMemStore.shouldFlushInMemory}} method always return true
> and current HBase Write Thread may repeatedly enter
> {{CompactingMemStore.shouldFlushInMemory}} again and again and never get a
> chance to add the {{cellToAdd}} .At the same time, because of the MVCC, all
> the HBase writes after this write would be hang.
> {code:java}
> protected boolean shouldFlushInMemory(MutableSegment currActive, Cell
> cellToAdd,
> MemStoreSizing memstoreSizing) {
> long cellSize = MutableSegment.getCellLength(cellToAdd);
> long segmentDataSize = currActive.getDataSize();
> while (segmentDataSize + cellSize < inmemoryFlushSize || inWalReplay) {
> // when replaying edits from WAL there is no need in in-memory flush
> regardless the size
> // otherwise size below flush threshold try to update atomically
> if (currActive.compareAndSetDataSize(segmentDataSize, segmentDataSize
> + cellSize)) {
> if (memstoreSizing != null) {
> memstoreSizing.incMemStoreSize(cellSize, 0, 0, 0);
> }
> // enough space for cell - no need to flush
> return false;
> }
> segmentDataSize = currActive.getDataSize();
> }
> // size above flush threshold
> return true;
> }
> {code}
> My PR not skip adding the size，it still add the size before really adding the
> cell to {{currActive}} , but it just change checking {{currActive}} size
> logic in {{CompactingMemStore.shouldFlushInMemory}} a little to make it
> similar to normal {{DefaultMemStore}} flushing to disk : it not make sure the
> size of {{currActive}} strictly less than {{inmemoryFlushSize}} any more,
> instead, it can tolerate the size of {{currActive}} just crossing the
> {{inmemoryFlushSize}} boundary at most one cell. Before adding the cell size,
> we make sure {{currActive}} size less than {{inmemoryFlushSize}} , and after
> adding cell size to the {{currActive}} size, we check whether its size
> exceeding {{inmemoryFlushSize}} , if it is , we flush the {{currActive}} in
> memory.
> Further more, I also flatten the {{CompactingMemStore.shouldFlushInMemory}}
> into {{CompactingMemStore.checkAndAddToActiveSize}} and extract the {{flush
> in memory}} logic into a separate method to make the workflow more readable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
