[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891904#comment-15891904 ] Anastasia Braginsky commented on HBASE-17662: - [~anoop.hbase], thank you very much! > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 2.0.0 > > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch, > HBASE-17662-V08.patch, HBASE-17662-V09-II.patch, HBASE-17662-V09.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890903#comment-15890903 ] Hudson commented on HBASE-17662: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2594 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2594/]) HBASE-17662 Disable in-memory flush when replaying from WAL (anoopsamjohn: rev 613bcb3622ecb1783c030f34ea2975280e1c43c1) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultMemStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AbstractMemStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 2.0.0 > > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch, > HBASE-17662-V08.patch, HBASE-17662-V09-II.patch, HBASE-17662-V09.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[
https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890499#comment-15890499
]
Hadoop QA commented on HBASE-17662:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 48s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m 11s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 137m 35s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855371/HBASE-17662-V09-II.patch
|
| JIRA Issue | HBASE-17662 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 2bffb1d1efdc 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 4a5eba5 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5898/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5898/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Disable in-memory flush when replaying from WAL
> ---
>
> Key: HBASE-17662
> URL: https://issues.apache.org/jira/browse/HBASE-17662
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Anastasia Braginsky
>Assignee: Anastasia Braginsky
> Fix For: 2.0.0
>
> Attachments: HBASE-17662-V02.patch, HBASE-1
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[
https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890197#comment-15890197
]
Hadoop QA commented on HBASE-17662:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 51s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 95m 38s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 134m 59s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.replication.regionserver.TestTableBasedReplicationSourceManagerImpl
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855352/HBASE-17662-V09.patch
|
| JIRA Issue | HBASE-17662 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 6afe5f6f80c5 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 4a5eba5 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5897/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5897/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5897/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5897/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Disable in-memory flush when replaying from WAL
>
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890010#comment-15890010 ] Anastasia Braginsky commented on HBASE-17662: - Hey [~anoop.hbase], the new patch (V09) according to your request is attached. Please let me know if anything else is needed. Thanks! > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch, > HBASE-17662-V08.patch, HBASE-17662-V09.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888365#comment-15888365 ] Anastasia Braginsky commented on HBASE-17662: - Woo Hoo! :) > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch, > HBASE-17662-V08.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[
https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888328#comment-15888328
]
Hadoop QA commented on HBASE-17662:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 95m 35s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 136m 57s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855156/HBASE-17662-V08.patch
|
| JIRA Issue | HBASE-17662 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 6a5c0f6a096a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f4e0ea2 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5875/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5875/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Disable in-memory flush when replaying from WAL
> ---
>
> Key: HBASE-17662
> URL: https://issues.apache.org/jira/browse/HBASE-17662
> Project: HBase
> Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>Assignee: Anastasia Braginsky
> Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch,
> HBASE-17662-V04.patch, HBASE-17662-V05.patc
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888248#comment-15888248 ] Anoop Sam John commented on HBASE-17662: Will commit after QA run.. Thanks. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch, > HBASE-17662-V08.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888033#comment-15888033 ] Anastasia Braginsky commented on HBASE-17662: - The (hopefully) final patch is attached! :) > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch, > HBASE-17662-V08.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887935#comment-15887935 ] Anastasia Braginsky commented on HBASE-17662: - Hi [~anoop.hbase], We found the problem with sizes. It is due to double alignment. Do not waste your time for debugging, I am going to publish new patch just in 10 minutes. Sorry for bothering you and Thanks! > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887899#comment-15887899 ] Anoop Sam John commented on HBASE-17662: I am trying to debug it.. Ya the test case fails now for the Deep overhead heapsize for CompactingMemstore. But debugging it, I can see the AtomicBoolean is considered as a reference also. (am sure)... Not matching the expected and actual heap size is some other issue. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887592#comment-15887592 ] Anastasia Braginsky commented on HBASE-17662: - [~anoop.hbase], I will take another look to check how we can overcome the ClassSize.estimateBase() side effects. But can you please also take a look? When CompactingMemStore.DEEP_OVERHEAD seems correct, it differs from ClassSize.estimateBase()'s resolution in 8 bytes. [~anoop.hbase], you have also spent long time dealing with the heap sizes, so your opinion is very valuable. Thanks, Anastasia > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886409#comment-15886409 ] Edward Bortnikov commented on HBASE-17662: -- [~anastas], [~anoop.hbase] - do we have a resolution here? > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885334#comment-15885334 ] Anastasia Braginsky commented on HBASE-17662: - Hi [~anoop.hbase], I spent really long time investigating deeply the sizes of CompactingMemStore issue. It appears that (at least by ClassSize.estimateBase method) AtomicBoolean is not counted together with references. If I put there 6 references (which is also seems correct to me) the test TestHeapSize fails. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885224#comment-15885224 ] stack commented on HBASE-17662: --- +1 (after addressing [~anoop.hbase]) comment. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[
https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885118#comment-15885118
]
Anoop Sam John commented on HBASE-17662:
Sorry for the delay.
{code}
- public static final long DEEP_OVERHEAD = AbstractMemStore.DEEP_OVERHEAD
- + 6 * ClassSize.REFERENCE // Store, RegionServicesForStores,
CompactionPipeline,
-// MemStoreCompactor, inMemoryFlushInProgress,
allowCompaction
- + Bytes.SIZEOF_LONG // inmemoryFlushSize
+
+ public static final long DEEP_OVERHEAD = ClassSize.align(
AbstractMemStore.DEEP_OVERHEAD
+ + 4 * ClassSize.REFERENCE // Store, RegionServicesForStores,
CompactionPipeline,
+// MemStoreCompactor
+ + Bytes.SIZEOF_LONG // inmemoryFlushSize
+ + 2 * Bytes.SIZEOF_BOOLEAN// compositeSnapshot and inWalReplay
+ 2 * ClassSize.ATOMIC_BOOLEAN// inMemoryFlushInProgress and
allowCompaction
- + CompactionPipeline.DEEP_OVERHEAD + MemStoreCompactor.DEEP_OVERHEAD;
+ + CompactionPipeline.DEEP_OVERHEAD + MemStoreCompactor.DEEP_OVERHEAD);
{code}
Why there is a change from 6 REFERENCEs to 4? inMemoryFlushInProgress,
allowCompaction - These are AtomicBoolean fields means they have reference
overhead. So 6 is the correct value. Do we need any change in tests to change
it back to 6 (existing code)? Pls check and correct..
> Disable in-memory flush when replaying from WAL
> ---
>
> Key: HBASE-17662
> URL: https://issues.apache.org/jira/browse/HBASE-17662
> Project: HBase
> Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>Assignee: Anastasia Braginsky
> Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch,
> HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch
>
>
> When replaying the edits from WAL, the region's updateLock is not taken,
> because a single threaded action is assumed. However, the thread-safeness of
> the in-memory flush of CompactingMemStore is based on taking the region's
> updateLock.
> The in-memory flush can be skipped in the replay time (anyway everything is
> flushed to disk just after the replay). Therefore it is acceptable to just
> skip the in-memory flush action while the updates come as part of replay from
> WAL.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884893#comment-15884893 ] Edward Bortnikov commented on HBASE-17662: -- Folks, Apologies for pushing again, but please help us turning some fire under this Jira and the others remaining in this project ... This one is the last exposed bug that prevents us from turning BASIC compaction into default. Seems like this is a small patch, can we commit it? Thanks. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[
https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884797#comment-15884797
]
Hadoop QA commented on HBASE-17662:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
7s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 8s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 16s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 138m 57s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12854769/HBASE-17662-V06.patch
|
| JIRA Issue | HBASE-17662 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 6d799a16c9e7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ce64e7e |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5843/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5843/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5843/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Disable in-memory flush when replaying from WAL
> ---
>
> Key: HBASE-17662
> URL: https://issues.apache.org/jira/browse/HBASE-17662
> Project: HBase
> Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>Assignee: Anastasia Braginsky
>
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884751#comment-15884751 ] Anastasia Braginsky commented on HBASE-17662: - Just added a rebased patch > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch, HBASE-17662-V06.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884714#comment-15884714 ] Anastasia Braginsky commented on HBASE-17662: - Hey everyone! Here are my answers: [~stack]: bq. Will the thread that sets the state be same as the one reading it? Yes, the thread that sets the 'inWalReplay' flag is the replay thread, which is the same thread that performs the adds and thus the same thread that is going to try to flush in memory and then read the 'inWalReplay' flag when the memstore size grows above the threshold. bq. Is this what single-threaded presumption around wal replay means? No, the single-threaded presumption is that there are no two (or more) replays simultaneously on the same store. bq. If single-threaded why are there concerns around in-memory flush? It only works if update lock taken? Because in-memory flush is done one a separate thread (T), which is dispatched once the memstore size grows above the threshold. This thread T takes the update lock in order to protect movement of the active segment to pipeline and creating the new active segment. Threre should be no concurrent adds to the memstore in the same time. Thread T assumes the adds are taking the update lock in shared mode and thus when T takes it in exclusive mode there are no concurrent updates. However, this assumption doesn't hold in replay case, where memstore is updated without taking this lock. From here, T takes the lock successfully and moves the active segment under the arms of ongoing concurrent updates. This is the concern. bq. (Flag can't be volatile; that'd be too expensive if we have to check it on each update to memstore). Flag is not volatile and I changed it to be simple boolean and not atomic boolean. The flag is checked only when the memstore size grows above the threshold. Therefore in common code path, it only will be checked once in a while. [~anoop.hbase] and [~ram_krish], thanks for your comments. As I said above, I've changed 'inWalReplay' flag to be simple boolean and it is checked only under the 'if' condition that checks for the size. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15882009#comment-15882009 ] ramkrishna.s.vasudevan commented on HBASE-17662: +1 on this patch if we agree with moving the check under the 'if' condition that checks for the size. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881686#comment-15881686 ] Anoop Sam John commented on HBASE-17662: I was just asking whether the WAL replay handling (And so this boolean write and read) being done by single thread only or not. Did not read the code. My worry on the 1st patch was that, it was doing a volatile read on every Cell addition.. Now we have changed it so that once the in memory flush size limit reached, we will do the boolean read to confirm it is not a replay time. So we are ok IMO.Understood u made it AtomicBoolean in a preventive way. Am ok with that way also provided we moved the boolean check as 2nd check now. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[
https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881591#comment-15881591
]
Hadoop QA commented on HBASE-17662:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 30s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 24s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 139m 0s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12854293/HBASE-17662-V05.patch
|
| JIRA Issue | HBASE-17662 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux fe16eff01be7 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8fb44fa |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5817/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5817/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5817/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Disable in-memory flush when replaying from WAL
> ---
>
> Key: HBASE-17662
> URL: https://issues.apache.org/jira/browse/HBASE-17662
> Project: HBase
> Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>Assignee: Anastasia Braginsky
>
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881524#comment-15881524 ] stack commented on HBASE-17662: --- Ok. Thanks for explanation Will the thread that sets the state be same as the one reading it? Is this what single-threaded presumption around wal replay means? If single-threaded why are there concerns around in-memory flush? It only works if update lock taken? (Flag can't be volatile; that'd be too expensive if we have to check it on each update to memstore). > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch, HBASE-17662-V05.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880213#comment-15880213 ] Anastasia Braginsky commented on HBASE-17662: - Hi Guys! Great to see, you paid attention to this patch :) [~anoop.hbase], regarding the Boolean vs AtomicBoolean issue: I made the Boolean Atomic not because it was critical in the current path, but to be on the safe side for the future, if the concurrency of the replay might change. I feel that using AtomicBoolean may not cause any performance degradation, but I can change it to simple boolean. [~stack], please pay attention that we are talking about in-memory flush and not flush to disk. Turning off flushing in-memory, will not mess anything up. The flush to disk will still happen by the end of replay similarly to the case with DefaultMemStore. As you have mention, requiring the replay process to take the update lock is meaningless, because this will cause all in-memory flushes to happen all together by the end of replay process and just before flushing to disk. [~ram_krish], in case we have lot of duplicates in the actual write path, then those duplicates (after replay) will be removed by flush-to-disk instead of in-memory-flush. So the final state of the data (after replay) will be the same with and without the crash. Do you agree? > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879887#comment-15879887 ] ramkrishna.s.vasudevan commented on HBASE-17662: The changes looks good to me. did you check for [~anoop.hbase]'s suggestion that if AtomicBoolean is a must here? Once you feel you have checked it then we can get this in. [~saint@gmail.com] What ever Anoop says. They are just avoiding in memory flushes not the flush to disk. The only thing could be that in case we have lot of duplicates in the actual write path and this in memory flush had avoided those duplicates but the server crashed on replay we may get lot of duplicates for those WAL entries. I think it is not going to affect the data that can be seen but just that they will be removed on a compaction only. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879775#comment-15879775 ] Anoop Sam John commented on HBASE-17662: I dont think she is saying to skip the flush as such.. Ya when the size is more than memstore size, we will need a flush (FLush to disk).. Here she mean abt the in memory flush (This happens when memstore size reaches 25% of the region flush size - Default). Make sense boss? > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879497#comment-15879497 ] stack commented on HBASE-17662: --- ...though taking the lock, would that mean no flush during replay... If so, then that wouldn't be correct either. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879490#comment-15879490 ] stack commented on HBASE-17662: --- bq. Therefore it is acceptable to just skip the in-memory flush action while the updates come as part of replay from WAL. IIRC, you would flush during replay when the replayed data was bigger than your memory could hold Turning it off would mess us up. Could you instead take the update lock during replay? > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878365#comment-15878365 ] Anoop Sam John commented on HBASE-17662: So the need for volatile/Atomic boolean is there? Am not sure as did not see in detail which thread deals with this variable. (Single only or multiple).. Pls check once and answer. Ya at least moving down the check against AtomicBoolean and we are good there. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, > HBASE-17662-V04.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877747#comment-15877747 ] Anastasia Braginsky commented on HBASE-17662: - [~anoop.hbase], [~ram_krish], I have addressed all the issues you raised in the code review. Please take a look on the recent version in the review board. Please consider for commit as the last patch passes the QA. Thanks! > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[
https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876984#comment-15876984
]
Hadoop QA commented on HBASE-17662:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
1s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 17s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 94m 4s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 133m 41s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853805/HBASE-17662-V03.patch
|
| JIRA Issue | HBASE-17662 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux d64019fba3dc 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d7ffa00 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5787/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5787/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Disable in-memory flush when replaying from WAL
> ---
>
> Key: HBASE-17662
> URL: https://issues.apache.org/jira/browse/HBASE-17662
> Project: HBase
> Issue Type: Sub-task
>Reporter: Anastasia Braginsky
> Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch
>
>
> When replaying the edits from WAL, the region's updateLock is not taken,
> because a
[jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
[ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874496#comment-15874496 ] Anoop Sam John commented on HBASE-17662: bq. if (inWalReplay.get()) This check and the set and reset of the inWalReplay state will be done from one thread only right? I may be wrong.. If so, do we really need a Atomic boolean or just simple boolean is ok? The above if comes in all normal write path and now we add a AtomicBoolean read which is not that cheap. If the state is being accessed from multiple thread and an Atomic/volatile is unavoidable, I suggest we add this boolean check after the size check. I mean after if (this.active.keySize() > inmemoryFlushSize) Or else for every write op we will end up in this Atomic boolean read. > Disable in-memory flush when replaying from WAL > --- > > Key: HBASE-17662 > URL: https://issues.apache.org/jira/browse/HBASE-17662 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > Attachments: HBASE-17662-V02.patch > > > When replaying the edits from WAL, the region's updateLock is not taken, > because a single threaded action is assumed. However, the thread-safeness of > the in-memory flush of CompactingMemStore is based on taking the region's > updateLock. > The in-memory flush can be skipped in the replay time (anyway everything is > flushed to disk just after the replay). Therefore it is acceptable to just > skip the in-memory flush action while the updates come as part of replay from > WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
