[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13959014#comment-13959014 ] Jeffrey Zhong commented on HBASE-10888: --- +1. Looks good to me! Yeah, 7 regions aren't for normal application. In normal situation, we should have around 70 regions per region server. Our reference guide recommends 100 regions / region server(https://hbase.apache.org/book/regions.arch.html). Therefore, in a more real situation there will be 10 times more recovered edits files are created which will result a better performance for DLR because the number of files created/written to during recovering in DLR won't increase much. The old way(recovered edits) will have 70 * 33 small recovered edits files are created/written which are random writes. In current DLR, we haven't implemented SKIP_WAL recovering that's the reason we don't see performance gain with small number of regions/log files. The recovering for writes should be a clear win:-). Thanks. Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt, 10888v2.txt, 10888v2.txt, 10888v2.txt, 10888v3.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13959102#comment-13959102 ] stack commented on HBASE-10888: --- Makes sense [~jeffreyz] In my little tests above, DLR actually assigned twice the number of regions and was still a little faster so we are headed in right direction. Let me commit. What else needs to be done here [~jeffreyz]? We should do SKIP_WAL. And then what about taking writes immediately? You have the issues handy? Thanks boss. Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt, 10888v2.txt, 10888v2.txt, 10888v2.txt, 10888v3.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13959103#comment-13959103 ] stack commented on HBASE-10888: --- If you have the issues handy we should peg them against 1.0 I'd say. Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt, 10888v2.txt, 10888v2.txt, 10888v2.txt, 10888v3.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13959256#comment-13959256 ] Jeffrey Zhong commented on HBASE-10888: --- So far there is no pending issue. Let me try to do the SKIP_WAL thing. Before we talked about the SKIP_WAL and the reason we didn't do it is due to that will complicate chain failure recovery situation. Thanks. Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt, 10888v2.txt, 10888v2.txt, 10888v2.txt, 10888v3.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13959287#comment-13959287 ] stack commented on HBASE-10888: --- [~jeffreyz] Smile. Lets KIS if we can. What about taking writes while recovering? Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt, 10888v2.txt, 10888v2.txt, 10888v2.txt, 10888v3.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13959292#comment-13959292 ] Jeffrey Zhong commented on HBASE-10888: --- Writes while recovering is on by default. Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt, 10888v2.txt, 10888v2.txt, 10888v2.txt, 10888v3.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[
https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958503#comment-13958503
]
stack commented on HBASE-10888:
---
Another test with more log files where I evened the count of regionservers has
the two log splitting systems at about the same with DLR coming in just
slightly faster:
{code}
2014-04-02 09:42:57,017 INFO [main-EventThread] zookeeper.RegionServerTracker:
RegionServer ephemeral node deleted, processing expiration
[c2024.halxg.cloudera.com,60020,1396454105817]
2014-04-02 09:43:08,559 INFO [MASTER_SERVER_OPERATIONS-c2020:6-4]
master.SplitLogManager: finished splitting (more than or equal to) 4358519947
bytes in 34 log files in
[hdfs://c2020.halxg.cloudera.com:8020/hbase/WALs/c2024.halxg.cloudera.com,60020,1396454105817-splitting]
in 11513ms
2014-04-02 09:43:10,900 INFO [AM.ZK.Worker-pool2-t88] master.RegionStates:
Onlined a6b2b9160737269b9a745cd58e9c5112 on
c2023.halxg.cloudera.com,60020,1396454098240
{code}
End-to-end DLR
{code}
2014-04-02 21:02:24,015 INFO [main-EventThread] zookeeper.RegionServerTracker:
RegionServer ephemeral node deleted, processing expiration
[c2023.halxg.cloudera.com,16020,1396482188465]
2014-04-02 21:02:37,499 INFO [M_LOG_REPLAY_OPS-c2020:16020-1]
master.SplitLogManager: finished splitting (more than or equal to) 4180462510
bytes in 33 log files in
[hdfs://c2020.halxg.cloudera.com:8020/hbase/WALs/c2023.halxg.cloudera.com,16020,1396482188465-splitting]
in 12645ms
2014-04-02 21:02:37,499 INFO [M_LOG_REPLAY_OPS-c2020:16020-1]
master.DeadServer: Finished processing
c2023.halxg.cloudera.com,16020,1396482188465
{code}
13.5 vs 13.9seconds.
Its only 7 regions. Is that why we don't see much difference in the timings
[~jeffreyz]? DLR does less work and facilitates further improvement in MTTR so
should go in.
Need a +1.
Enable distributed log replay as default
Key: HBASE-10888
URL: https://issues.apache.org/jira/browse/HBASE-10888
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: 10888.txt, 10888v2.txt, 10888v2.txt, 10888v2.txt,
10888v3.txt
Enable 'distributed log replay' by default. Depends on hfilev3 being enabled.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13956728#comment-13956728 ] stack commented on HBASE-10888: --- FYI [~jeffreyz] You seen any issues w/ this sir? Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13956778#comment-13956778 ] Jeffrey Zhong commented on HBASE-10888: --- No, let's go for it. Cheers! Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[
https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13956975#comment-13956975
]
Hadoop QA commented on HBASE-10888:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12638104/10888v2.txt
against trunk revision .
ATTACHMENT ID: 12638104
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterFileSystem
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9162//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9162//console
This message is automatically generated.
Enable distributed log replay as default
Key: HBASE-10888
URL: https://issues.apache.org/jira/browse/HBASE-10888
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: 10888.txt, 10888v2.txt
Enable 'distributed log replay' by default. Depends on hfilev3 being enabled.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957027#comment-13957027 ] Jeffrey Zhong commented on HBASE-10888: --- [~saint@gmail.com] The test failures are due to distributedLogReplay is turned on by two configuration settings(FORMAT_VERSION_KEY DISTRIBUTED_LOG_REPLAY_KEY) instead of one before. So if we add conf.setInt(hfile.format.version, 3); into those failed test cases. They should pass. We can also enable conf.setInt(hfile.format.version, 3); for all tests of TestDistributedLogSplitting. Thanks. Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt, 10888v2.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957125#comment-13957125 ] stack commented on HBASE-10888: --- Excellent [~jeffreyz] Thank you for taking a look. It would have taken me ages go figure it. I just committed v3 patch so let me rerun this patch... hopefully it will work now. Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt, 10888v2.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[
https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957213#comment-13957213
]
Hadoop QA commented on HBASE-10888:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12638148/10888v2.txt
against trunk revision .
ATTACHMENT ID: 12638148
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.master.TestMasterNoCluster.testNotPullingDeadRegionServerFromZK(TestMasterNoCluster.java:298)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9165//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9165//console
This message is automatically generated.
Enable distributed log replay as default
Key: HBASE-10888
URL: https://issues.apache.org/jira/browse/HBASE-10888
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: 10888.txt, 10888v2.txt, 10888v2.txt
Enable 'distributed log replay' by default. Depends on hfilev3 being enabled.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957306#comment-13957306 ] stack commented on HBASE-10888: --- Playing with this on cluster, seems to basically work. Logs are split and stuff comes back again afterward. Would need to run the linked list and chaos monkey for a while to make sure all really good but good enough to commit I'd say. Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt, 10888v2.txt, 10888v2.txt, 10888v2.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[ https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957316#comment-13957316 ] stack commented on HBASE-10888: --- I also ran a 0.98 cluster, crashed it, and then started a 0.99 cluster over it w/ this patch and it split logs and made progress Running IntegrationTestMTTR seems to keep going too after fixing the conf dirs so it restarted the master w/ proper configs (it kills master all the time thinking it a regionserver). Let me see if I can get numbers to compare the recovery times. Enable distributed log replay as default Key: HBASE-10888 URL: https://issues.apache.org/jira/browse/HBASE-10888 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 10888.txt, 10888v2.txt, 10888v2.txt, 10888v2.txt Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[
https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957334#comment-13957334
]
Hadoop QA commented on HBASE-10888:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12638187/10888v2.txt
against trunk revision .
ATTACHMENT ID: 12638187
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestTableMapReduceBase.testMultiRegionTable(TestTableMapReduceBase.java:96)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9169//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9169//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9169//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9169//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9169//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9169//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9169//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9169//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9169//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9169//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9169//console
This message is automatically generated.
Enable distributed log replay as default
Key: HBASE-10888
URL: https://issues.apache.org/jira/browse/HBASE-10888
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: 10888.txt, 10888v2.txt, 10888v2.txt, 10888v2.txt
Enable 'distributed log replay' by default. Depends on hfilev3 being enabled.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10888) Enable distributed log replay as default
[
https://issues.apache.org/jira/browse/HBASE-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957372#comment-13957372
]
stack commented on HBASE-10888:
---
Some rough numbers have it that they are about the same:
0.98.1 took just over 6 seconds for ten logs (but one less splitter in the
cluster since master now participates on trunk)
{code}
2014-04-01 22:35:24,015 INFO [main-EventThread] zookeeper.RegionServerTracker:
RegionServer ephemeral node deleted, processing expiration
[c2023.halxg.cloudera.com,60020,1396416642448]
2014-04-01 22:35:28,726 INFO [MASTER_SERVER_OPERATIONS-c2020:6-0]
master.SplitLogManager: finished splitting (more than or equal to) 1172247640
bytes in 10 log files in
[hdfs://c2020.halxg.cloudera.com:8020/hbase/WALs/c2023.halxg.cloudera.com,60020,1396416642448-splitting]
in 4668ms
2014-04-01 22:35:30,039 INFO [MASTER_SERVER_OPERATIONS-c2020:6-0]
handler.ServerShutdownHandler: Finished processing of shutdown of
c2023.halxg.cloudera.com,60020,1396416642448
{code}
For trunk/0.99, took 6.3seconds which is a little longer.
{code}
2014-04-01 22:25:33,011 INFO [main-EventThread] zookeeper.RegionServerTracker:
RegionServer ephemeral node deleted, processing expiration
[c2021.halxg.cloudera.com,16020,1396415234624]
2014-04-01 22:25:39,388 INFO [M_LOG_REPLAY_OPS-c2020:16020-1]
master.SplitLogManager: finished splitting (more than or equal to) 1303360768
bytes in 11 log files in
[hdfs://c2020.halxg.cloudera.com:8020/hbase/WALs/c2021.halxg.cloudera.com,16020,1396415234624-splitting]
in 5746ms
{code}
Let me try and do a bigger test, more like what [~jeffreyz] had over in
HBASE-7006
Enable distributed log replay as default
Key: HBASE-10888
URL: https://issues.apache.org/jira/browse/HBASE-10888
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: 10888.txt, 10888v2.txt, 10888v2.txt, 10888v2.txt,
10888v3.txt
Enable 'distributed log replay' by default. Depends on hfilev3 being enabled.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
