[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583346#comment-13583346 ] Jonathan Hsieh commented on HBASE-7290: --- According to the svn manual, normally you svn delete a branch after it has been merged in. Since this did completely work out, I'll keep the branch alive with the patch-by-patch commit record and is browseable in these places: http://svn.apache.org/viewvc/hbase/branches/hbase-7290v2/ https://github.com/apache/hbase/tree/hbase-7290v2 > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, > hbase-7290.mega.patch, hbase-7290.mega-v4.patch, hbase-7290.mega-v5.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583169#comment-13583169 ] Hudson commented on HBASE-7290: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #414 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/414/]) HBASE-7290 / HBASE-6055 Online and Offline table snapshots. Merged to trunk. (Jesse Yates, Matteo Bertozzi, Jonathan Hsieh, Ted Yu) (Revision 1448506) Result = FAILURE jmhsieh : Files : * /hbase/trunk * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ErrorHandlingProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/ErrorHandling.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/MasterAdmin.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/hbase.proto * /hbase/trunk/hbase-server/src/main/avro * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/Chore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/backup/HFileArchiver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignException.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignExceptionDispatcher.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignExceptionListener.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignExceptionSnare.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/TimeoutException.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/TimeoutExceptionInjector.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/HFileLink.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/HLogLink.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/SnapshotSentinel.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DisableTableHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/metrics * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/MasterSnapshotVerifier.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582978#comment-13582978 ] Hudson commented on HBASE-7290: --- Integrated in HBase-TRUNK #3888 (See [https://builds.apache.org/job/HBase-TRUNK/3888/]) HBASE-7290 / HBASE-6055 Online and Offline table snapshots. Merged to trunk. (Jesse Yates, Matteo Bertozzi, Jonathan Hsieh, Ted Yu) (Revision 1448506) Result = FAILURE jmhsieh : Files : * /hbase/trunk * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ErrorHandlingProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/ErrorHandling.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/MasterAdmin.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/hbase.proto * /hbase/trunk/hbase-server/src/main/avro * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/Chore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/backup/HFileArchiver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignException.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignExceptionDispatcher.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignExceptionListener.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/ForeignExceptionSnare.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/TimeoutException.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/errorhandling/TimeoutExceptionInjector.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/HFileLink.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/HLogLink.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/SnapshotSentinel.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DisableTableHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/metrics * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/MasterSnapshotVerifier.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582919#comment-13582919 ] Jonathan Hsieh commented on HBASE-7290: --- I think I'm going to leave the sub issues in here for the time being -- as we knock these off please promote them to be full issues and make sure the jira name mentions table snapshots. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, > hbase-7290.mega.patch, hbase-7290.mega-v4.patch, hbase-7290.mega-v5.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582913#comment-13582913 ] Jonathan Hsieh commented on HBASE-7290: --- in the preivous message, the remove empty directories should be a HBASE-7290 merge followup. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, > hbase-7290.mega.patch, hbase-7290.mega-v4.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582593#comment-13582593 ] Jesse Yates commented on HBASE-7290: +1 on merging trunk (a little late to the party :) > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh >Priority: Blocker > Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, > hbase-7290.mega.patch, hbase-7290.mega-v4.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582507#comment-13582507
]
Hadoop QA commented on HBASE-7290:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570172/hbase-7290.mega-v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 91 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4470//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4470//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4470//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4470//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4470//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4470//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4470//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4470//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4470//console
This message is automatically generated.
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>Priority: Blocker
> Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt,
> hbase-7290.mega.patch, hbase-7290.mega-v4.patch
>
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch. (was: HBASE-6055 will be closed when the online
> snapshots pieces get merged with trunk. This umbrella issue has all the
> online snapshot specific patches. This will get merged once one of the
> implementations makes it into trunk. Other flavors of online snapshots can
> then be done as normal patches instead of on a development branch.)
> (not a fan of the quick edit descirption jira feature)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582462#comment-13582462 ] Ted Yu commented on HBASE-7290: --- +1 on merging to trunk. We can tackle remaining sub-tasks post-merge. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh >Priority: Blocker > Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, > hbase-7290.mega.patch, hbase-7290.mega-v4.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582458#comment-13582458 ] Matteo Bertozzi commented on HBASE-7290: I'm also +1 on the merge > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh >Priority: Blocker > Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, > hbase-7290.mega.patch, hbase-7290.mega-v4.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582454#comment-13582454 ] Jonathan Hsieh commented on HBASE-7290: --- I believe ted's run was against commit. I've run the unit test suite successfully against hadoop 1 and with 11 known failures against hadoop2 (TestImportExport/TestImportTsv has problems with hadoop2 mr/hdfs minicluster setup). I'm +1 to merge. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh >Priority: Blocker > Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, > hbase-7290.mega.patch, hbase-7290.mega-v4.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582428#comment-13582428
]
Ted Yu commented on HBASE-7290:
---
I got a green build for hbase-7290v2 svn branch.
{code}
[INFO] HBase . SUCCESS [1.780s]
[INFO] HBase - Common SUCCESS [10.211s]
[INFO] HBase - Protocol .. SUCCESS [0.292s]
[INFO] HBase - Client SUCCESS [0.505s]
[INFO] HBase - Prefix Tree ... SUCCESS [2.493s]
[INFO] HBase - Hadoop Compatibility .. SUCCESS [0.370s]
[INFO] HBase - Hadoop One Compatibility .. SUCCESS [0.783s]
[INFO] HBase - Server SUCCESS [53:37.531s]
[INFO] HBase - Integration Tests . SUCCESS [1.313s]
[INFO] HBase - Examples .. SUCCESS [39.355s]
[INFO]
[INFO] BUILD SUCCESS
{code}
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>Priority: Blocker
> Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, hbase-7290.mega.patch
>
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch. (was: HBASE-6055 will be closed when the online
> snapshots pieces get merged with trunk. This umbrella issue has all the
> online snapshot specific patches. This will get merged once one of the
> implementations makes it into trunk. Other flavors of online snapshots can
> then be done as normal patches instead of on a development branch.)
> (not a fan of the quick edit descirption jira feature)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582378#comment-13582378 ] Jonathan Hsieh commented on HBASE-7290: --- I've just committed a clean sync up merge with trunk into the hbase-7290v2 svn branch. After some in person discussions from yesterday (summarized in the mailing list notes), we'll be wrapping up the outstanding reviews of the megapatch, posting one more version that will hopefully be the last pre merge versions of the megapatch. We should put the +1 votes on that and then merge with trunk afterwards (and before the 0.95/0.96 branch is made). > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh >Priority: Blocker > Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, hbase-7290.mega.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13578858#comment-13578858 ] Ted Yu commented on HBASE-7290: --- I am more than half-way through code review. Ruby script references non-existent method: @admin.renameSnapshot() which we can defer to future JIRA. I think we should give 3 (out of 4) pairs of coprocessor hooks better names to reflect the fact that the underlying calls are asynchronous. More reviews to follow. I can provide patches addressing review comments in sub-tasks next week. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, hbase-7290.mega.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13578436#comment-13578436
]
Jonathan Hsieh commented on HBASE-7290:
---
>From the mailing list:
{quote}
I've successfully ported the git repo to an svn branch. The branch is
called hbase-7290v2 (I made some mistakes on v1). Here's a link to
what the svn snapshots repo looks like:
http://svn.apache.org/viewvc/hbase/branches/hbase-7290v2/
This is the official branch now -- we'll now commit fixes from the
pre-trunk-merge mega-patch reviews to that branch.
{quote}
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, hbase-7290.mega.patch
>
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch. (was: HBASE-6055 will be closed when the online
> snapshots pieces get merged with trunk. This umbrella issue has all the
> online snapshot specific patches. This will get merged once one of the
> implementations makes it into trunk. Other flavors of online snapshots can
> then be done as normal patches instead of on a development branch.)
> (not a fan of the quick edit descirption jira feature)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13577366#comment-13577366 ] stack commented on HBASE-7290: -- Quick review of patch. All below are total nits not to get in the way of commit (anything can be done about above findbugs and javadoc warnings -- they have been showing green of later ... would be pity to go red again). On the patch, the ErrorHandling.proto looks generally applicable... Fix tab spacing in protos on commit? Nice... they show in UI! Is the change in MasterAdminProtocol right? It removes a critical looking line. Where is the doc? At least add fat release note on commit (The stuff in SnapshotDescriptionUtils looks great in the javadoc class comment) FSVisitor looks generally useful too... ModifyRegionUtils probably belongs under regionserver package? And support in shell! +1 on commit even you would like to address findbugs/javadoc in follow on issue. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, hbase-7290.mega.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13577313#comment-13577313
]
Hadoop QA commented on HBASE-7290:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12569144/7290-mega-v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 91 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.constraint.TestConstraint
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4432//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4432//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4432//console
This message is automatically generated.
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, hbase-7290.mega.patch
>
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch. (was: HBASE-6055 will be closed when the online
> snapshots pieces get merged with trunk. This umbrella issue has all the
> online snapshot specific patches. This will get merged once one of the
> implementations makes it into trunk. Other flavors of online snapshots can
> then be done as normal patches instead of on a development branch.)
> (not a fan of the quick edit descirption jira feature)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13577312#comment-13577312 ] Ted Yu commented on HBASE-7290: --- TestConstraint failed in QA run for both v2 and v3. I ran it locally and it passed. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: 7290-mega-v2.txt, 7290-mega-v3.txt, hbase-7290.mega.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13577219#comment-13577219
]
Hadoop QA commented on HBASE-7290:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12569099/7290-mega-v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 91 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 7 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.constraint.TestConstraint
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4427//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4427//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4427//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4427//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4427//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4427//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4427//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4427//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4427//console
This message is automatically generated.
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: 7290-mega-v2.txt, hbase-7290.mega.patch
>
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch. (was: HBASE-6055 will be closed when the online
> snapshots pieces get merged with trunk. This umbrella issue has all the
> online snapshot specific patches. This will get merged once one of the
> implementations makes it into trunk. Other flavors of online snapshots can
> then be done as normal patches instead of on a development branch.)
> (not a fan of the quick edit descirption jira feature)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13577186#comment-13577186 ] Jonathan Hsieh commented on HBASE-7290: --- Hey Ted, Let's these javadoc/findbugs/etc fixes as separate subtasks instead of burying them in the mega patch. Thanks. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: 7290-mega-v2.txt, hbase-7290.mega.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13577164#comment-13577164 ] Ted Yu commented on HBASE-7290: --- I have created https://reviews.apache.org/r/9416/ I will go over the whole patch once more. Will also get the diff for new findbugs warnings. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: 7290-mega-v2.txt, hbase-7290.mega.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13577112#comment-13577112 ] Jonathan Hsieh commented on HBASE-7290: --- We can fix these qabot issues before the merge. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: hbase-7290.mega.patch > > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13577104#comment-13577104
]
Hadoop QA commented on HBASE-7290:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12569057/hbase-7290.mega.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 91 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 7 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4425//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4425//console
This message is automatically generated.
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-7290.mega.patch
>
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch. (was: HBASE-6055 will be closed when the online
> snapshots pieces get merged with trunk. This umbrella issue has all the
> online snapshot specific patches. This will get merged once one of the
> implementations makes it into trunk. Other flavors of online snapshots can
> then be done as normal patches instead of on a development branch.)
> (not a fan of the quick edit descirption jira feature)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13574128#comment-13574128 ] Jonathan Hsieh commented on HBASE-7290: --- Sounds like this explains some of the extra zk exiting delays mentioned in HBASE-7778. Please file subtask and fix? > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573720#comment-13573720
]
Ted Yu commented on HBASE-7290:
---
ZKProcedureUtil has a reference to ZooKeeperWatcher.
This reference is obtained in two places.
1. SnapshotManager ctor:
{code}
ProcedureCoordinatorRpcs comms = new ZKProcedureCoordinatorRpcs(
master.getZooKeeper(),
SnapshotManager.ONLINE_SNAPSHOT_CONTROLLER_DESCRIPTION, name);
{code}
2. RegionServerSnapshotManager ctor:
{code}
ZooKeeperWatcher zkw = rss.getZooKeeper();
String nodeName = rss.getServerName().toString();
this.memberRpcs = new ZKProcedureMemberRpcs(zkw,
SnapshotManager.ONLINE_SNAPSHOT_CONTROLLER_DESCRIPTION, nodeName);
{code}
This means the watcher would be closed when master / region server goes down.
Looks like ZKProcedureUtil.close() doesn't have to call watcher.close()
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch. (was: HBASE-6055 will be closed when the online
> snapshots pieces get merged with trunk. This umbrella issue has all the
> online snapshot specific patches. This will get merged once one of the
> implementations makes it into trunk. Other flavors of online snapshots can
> then be done as normal patches instead of on a development branch.)
> (not a fan of the quick edit descirption jira feature)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573014#comment-13573014 ] Jonathan Hsieh commented on HBASE-7290: --- The Ted's comment above was copied to and discussed in HBASE-7778. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. (was: HBASE-6055 will be closed when the online > snapshots pieces get merged with trunk. This umbrella issue has all the > online snapshot specific patches. This will get merged once one of the > implementations makes it into trunk. Other flavors of online snapshots can > then be done as normal patches instead of on a development branch.) > (not a fan of the quick edit descirption jira feature) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572610#comment-13572610
]
Ted Yu commented on HBASE-7290:
---
In RegionServerSnapshotManager#cancelTasks():
{code}
while (!futures.isEmpty()) {
// block to remove cancelled futures;
LOG.warn("Removing cancelled elements from taskPool");
futures.remove(taskPool.take());
}
{code}
I think the log should be outside while loop.
taskPool.take() may throw InterruptedException. I think the futures.remove()
call should be wrapped in try block.
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572608#comment-13572608 ] Jonathan Hsieh commented on HBASE-7290: --- Problem describe is filed as HBASE-7778 and more details described there. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572508#comment-13572508 ] Jonathan Hsieh commented on HBASE-7290: --- The large tests fail because a interrupt status flag in the thread that the hbaseminicluster is shutdown from. I've got a hack patches that clears the state in unit tests. The right thing to do is to hunt down the root cause of new interrupt state to preserve previous behavior. I suspect something in the merge introduces -- probably the something in the shutdown of the snapshot related managers. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13571079#comment-13571079 ] Ted Yu commented on HBASE-7290: --- I ran TestRestoreSnapshotFromClient 5 times on MacBook and they passed. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13570808#comment-13570808
]
Jonathan Hsieh commented on HBASE-7290:
---
Some of these tests seem to be failing with a table delete/create race.
{code}
org.apache.hadoop.hbase.TableNotFoundException:
clonedtb-1360017431921
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1098)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:974)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:923)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:246)
at org.apache.hadoop.hbase.client.HTable.(HTable.java:187)
at
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient.verifyRowCount(TestRestoreSnapshotFromClient.java:380)
at
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient.testCloneSnapshot(TestRestoreSnapshotFromClient.java:252)
at
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient.testCloneSnapshot(TestRestoreSnapshotFromClient.java:245)
...
{code}
Since the snapshot branch contains a commit of HBASE-7365 which [~mbertozzi]'s
been fighting with on a newer version trunk, my guess is that some of the
remaining failures have to do with this.
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13570709#comment-13570709
]
Jonathan Hsieh commented on HBASE-7290:
---
Here's a branch I'm working on currently.
https://github.com/jmhsieh/hbase/commits/snapshot-work-0204
I'm down to this one on the small/medium
{code}
Running org.apache.hadoop.hbase.client.TestMultiParallel
09:25:53 Tests run: 11, Failures: 2, Errors: 1, Skipped: 0, Time elapsed:
63.74 sec <<< FAILURE!
{code}
but a whole bunch, many of which seem unrelated, of the large unit tests fail.
{code}
10:39:15 Running org.apache.hadoop.hbase.master.TestRestartCluster
10:39:15 Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
58.153 sec <<< FAILURE!
...
11:25:01 Running org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster
11:25:01 Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
26.298 sec <<< FAILURE!
14:20:06 Running
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
14:20:06 Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 33.31
sec <<< FAILURE!
...
{code}
One of the snapshot large test seems to pass for me locally but has a failed a
few times on my internal jenkins instance.
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13570695#comment-13570695 ] Ted Yu commented on HBASE-7290: --- TestRestoreSnapshotFromClient passed in my local workspace. I am looking at the 3 test failures in TestSnapshotCloneIndependence. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569915#comment-13569915 ] Jonathan Hsieh commented on HBASE-7290: --- To reduce confusion, I've added the [snapshot 130201 merge] prefix to issues related to the snapshot branch trunk branch merge on feb 1 2013. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569619#comment-13569619 ] Ted Yu commented on HBASE-7290: --- I filed HBASE-7753 and attached a patch there. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569593#comment-13569593 ] Jonathan Hsieh commented on HBASE-7290: --- [~te...@apache.org] file and fix it? I actually prefer this style (someone merges then we fix in subsequent patches) -- we quickly get to a place where more people can find and work on problems, and each problem we fix is in a patch that we can understand in isolation. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569591#comment-13569591
]
Ted Yu commented on HBASE-7290:
---
Found the following in HRegionServer.java:
{code}
<<< HEAD
* @return result to return to client if default operation should be
* bypassed as indicated by RegionObserver, null otherwise
===
* @return the Result
>>> apache/trunk
{code}
In ProtobufUtil.java:
{code}
<<< HEAD
* @return protocol buffer DeleteType
===
* @return a DeleteType
>>> apache/trunk
{code}
And in HRegion.java:
{code}
<<< HEAD
===
* @see HRegion#batchMutate(Pair[])
>>> apache/trunk
*/
public OperationStatus[] put(Put[] puts) throws IOException {
{code}
Since the above is in javadoc, it doesn't affect compilation.
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569580#comment-13569580
]
Ted Yu commented on HBASE-7290:
---
I think we should go over the manual merges in the following files:
{code}
hbase-protocol/src/main/protobuf/hbase.proto
hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
hbase-server/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java
hbase-server/src/main/java/org/apache/hadoop/hbase/backup/HFileArchiver.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
hbase-server/src/main/resources/hbase-default.xml
hbase-server/src/main/ruby/hbase/admin.rb
hbase-server/src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
{code}
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569551#comment-13569551 ] Matteo Bertozzi commented on HBASE-7290: I still haven't look at the merge, but some changes to the HFileArchiver seems to got lost, maybe is the main problem of the test failures > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569547#comment-13569547
]
Jonathan Hsieh commented on HBASE-7290:
---
More failures from -PrunLargeTests
{code}
15:23:28 Running org.apache.hadoop.hbase.client.TestSnapshotFromClient
15:23:28 Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
122.156 sec <<< FAILURE!
...
15:41:35 Running org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient
15:41:49 Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
777.882 sec <<< FAILURE!
...
15:48:26 Running org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence
15:48:26 Tests run: 6, Failures: 0, Errors: 3, Skipped: 0, Time elapsed:
89.216 sec <<< FAILURE!
15:50:26 Running org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
15:50:30 Tests run: 5, Failures: 0, Errors: 5, Skipped: 0, Time elapsed:
119.637 sec <<< FAILURE!
...
16:10:03 Running org.apache.hadoop.hbase.master.TestRestartCluster
16:10:03 Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
57.298 sec <<< FAILURE!
..
16:44:35 Running org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster
16:44:35 Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
26.825 sec <<< FAILURE!
...
17:12:13 Running
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
17:12:13 Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
36.894 sec <<< FAILURE!
{code}
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569248#comment-13569248 ] Jonathan Hsieh commented on HBASE-7290: --- I've merged the snapshots branch with apache/trunk @HBASE-3890 (hash 3d8721b) which is dated 1/31/12 10:59:14pm. Working on unit test fixes now. We will file post merge fixes as sub-issues. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569242#comment-13569242 ] Ted Yu commented on HBASE-7290: --- Jon: Your suggestion above is good. TestMultiParallel failed in trunk too: https://builds.apache.org/job/HBase-TRUNK/3841/console > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569233#comment-13569233 ] Jonathan Hsieh commented on HBASE-7290: --- I have a fix for TestStoreFile, and initial look at TestMultiParallel indicates the fix may be subtle. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569232#comment-13569232 ] Jonathan Hsieh commented on HBASE-7290: --- [~te...@apache.org], ok how about I commit the current merge commit as the new snapshot head and then you file a sub jira with this fix? > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569231#comment-13569231
]
Ted Yu commented on HBASE-7290:
---
The following change allows the test to pass:
{code}
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestSnapshotFromMaster.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/clean
index c9f6949..4c1e535 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestSnapshotFromMaster.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestSnapshotFromMaster.java
@@ -48,13 +48,13 @@ import
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos.IsSnapshotDo
import
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos.IsSnapshotDoneResponse;
import
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos.ListSnapshotRequest;
import
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos.ListSnapshotResponse;
+import org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy;
import org.apache.hadoop.hbase.regionserver.HRegion;
import org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils;
import org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils;
import org.apache.hadoop.hbase.snapshot.UnknownSnapshotException;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.FSUtils;
-import org.apache.hadoop.hbase.util.HFileArchiveUtil;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
@@ -121,6 +121,9 @@ public class TestSnapshotFromMaster {
// Enable snapshot
conf.setBoolean(SnapshotManager.HBASE_SNAPSHOT_ENABLED, true);
conf.setLong(SnapshotHFileCleaner.HFILE_CACHE_REFRESH_PERIOD_CONF_KEY,
cacheRefreshPeriod);
+conf.set(HConstants.HBASE_REGION_SPLIT_POLICY_KEY,
+ ConstantSizeRegionSplitPolicy.class.getName());
+
}
@Before
{code}
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569174#comment-13569174
]
Ted Yu commented on HBASE-7290:
---
Looking at test output from TestSnapshotFromMaster:
{code}
2013-02-01 14:10:49,234 INFO
[RegionServer:1;10.11.3.105,53794,1359756635266-splits-1359756649234]
regionserver.SplitTransaction(215): Starting split of region
test,,1359756646966.e9c665350a0d6d059cad8ae03f8dadae.
2013-02-01 14:10:49,235 DEBUG
[RegionServer:1;10.11.3.105,53794,1359756635266-splits-1359756649234]
regionserver.SplitTransaction(886): regionserver:53794-0x13c97cfa9760002
Creating ephemeral node for e9c665350a0d6d059cad8ae03f8dadae in SPLITTING state
2013-02-01 14:10:49,236 DEBUG [main-EventThread]
zookeeper.ZooKeeperWatcher(265): master:53791-0x13c97cfa976 Received
ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected,
path=/hbase/region-in-transition
2013-02-01 14:10:49,236 DEBUG
[RegionServer:1;10.11.3.105,53794,1359756635266-splits-1359756649234]
zookeeper.ZKAssign(720): regionserver:53794-0x13c97cfa9760002 Attempting to
transition node e9c665350a0d6d059cad8ae03f8dadae from RS_ZK_REGION_SPLITTING to
RS_ZK_REGION_SPLITTING
2013-02-01 14:10:49,238 DEBUG
[RegionServer:1;10.11.3.105,53794,1359756635266-splits-1359756649234]
zookeeper.ZKUtil(1643): regionserver:53794-0x13c97cfa9760002 Retrieved 94
byte(s) of data from znode
/hbase/region-in-transition/e9c665350a0d6d059cad8ae03f8dadae;
data=region=test,,1359756646966.e9c665350a0d6d059cad8ae03f8dadae.,
state=RS_ZK_REGION_SPLITTING, servername=10.11.3.105,53794,1359756635266,
createTime=1359756649235, payload.length=0
2013-02-01 14:10:49,238 DEBUG [hbase-am-zkevent-worker-pool-2-thread-16]
zookeeper.ZKUtil(1643): master:53791-0x13c97cfa976 Retrieved 94 byte(s) of
data from znode /hbase/region-in-transition/e9c665350a0d6d059cad8ae03f8dadae
and set watcher; region=test,,1359756646966.e9c665350a0d6d059cad8ae03f8dadae.,
state=RS_ZK_REGION_SPLITTING, servername=10.11.3.105,53794,1359756635266,
createTime=1359756649235, payload.length=0
2013-02-01 14:10:49,239 DEBUG [main-EventThread]
zookeeper.ZooKeeperWatcher(265): master:53791-0x13c97cfa976 Received
ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
path=/hbase/region-in-transition/e9c665350a0d6d059cad8ae03f8dadae
2013-02-01 14:10:49,240 DEBUG
[RegionServer:1;10.11.3.105,53794,1359756635266-splits-1359756649234]
zookeeper.ZKAssign(782): regionserver:53794-0x13c97cfa9760002 Successfully
transitioned node e9c665350a0d6d059cad8ae03f8dadae from RS_ZK_REGION_SPLITTING
to RS_ZK_REGION_SPLITTING
2013-02-01 14:10:49,240 DEBUG
[RegionServer:1;10.11.3.105,53794,1359756635266-splits-1359756649234]
zookeeper.ZKAssign(720): regionserver:53794-0x13c97cfa9760002 Attempting to
transition node e9c665350a0d6d059cad8ae03f8dadae from RS_ZK_REGION_SPLITTING to
RS_ZK_REGION_SPLITTING
2013-02-01 14:10:49,240 DEBUG [hbase-am-zkevent-worker-pool-2-thread-17]
zookeeper.ZKUtil(1643): master:53791-0x13c97cfa976 Retrieved 94 byte(s) of
data from znode /hbase/region-in-transition/e9c665350a0d6d059cad8ae03f8dadae
and set watcher; region=test,,1359756646966.e9c665350a0d6d059cad8ae03f8dadae.,
state=RS_ZK_REGION_SPLITTING, servername=10.11.3.105,53794,1359756635266,
createTime=1359756649238, payload.length=0
2013-02-01 14:10:49,241 DEBUG [hbase-am-zkevent-worker-pool-2-thread-17]
master.AssignmentManager(641): Handling transition=RS_ZK_REGION_SPLITTING,
server=10.11.3.105,53794,1359756635266,
region=e9c665350a0d6d059cad8ae03f8dadae, current state from region state map
=null
2013-02-01 14:10:49,241 INFO [hbase-am-zkevent-worker-pool-2-thread-17]
master.RegionStates(242): Region {NAME =>
'test,,1359756646966.e9c665350a0d6d059cad8ae03f8dadae.', STARTKEY
=> '', ENDKEY => '', ENCODED =>
e9c665350a0d6d059cad8ae03f8dadae,} transitioned from
{test,,1359756646966.e9c665350a0d6d059cad8ae03f8dadae. state=OPEN,
ts=1359756647443, server=10.11.3.105,53794,1359756635266} to
{test,,1359756646966.e9c665350a0d6d059cad8ae03f8dadae. state=SPLITTING,
ts=1359756649241, server=10.11.3.105,53794,1359756635266}
2013-02-01 14:10:49,241 DEBUG
[RegionServer:1;10.11.3.105,53794,1359756635266-splits-1359756649234]
zookeeper.ZKUtil(1643): regionserver:53794-0x13c97cfa9760002 Retrieved 94
byte(s) of data from znode
/hbase/region-in-transition/e9c665350a0d6d059cad8ae03f8dadae;
data=region=test,,1359756646966.e9c665350a0d6d059cad8ae03f8dadae.,
state=RS_ZK_REGION_SPLITTING, servername=10.11.3.105,53794,1359756635266,
createTime=1359756649238, payload.length=0
2013-02-01 14:10:49,243 DEBUG [main-EventThread]
zookeeper.ZooKeeperWatcher(265): master:53791-0x13c97cfa976 Received
ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
path=/hbase/region-in-transition/e9c665350a0d6d059cad8ae03f8dadae
2013-02-01 14:10:49,243 DEBUG
[RegionServer:1;10.11.3.105,53794,135975663526
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569071#comment-13569071 ] Jonathan Hsieh commented on HBASE-7290: --- Here's a link to where the merged branch is https://github.com/jmhsieh/hbase/tree/snapshots-merge-0201 Currently, there are several tests failing, which I will file as follow-ons if they take longer than an hour to resolve. org.apache.hadoop.hbase.regionserver.TestStoreFile.testHFileLink org.apache.hadoop.hbase.client.TestMultiParallel org.apache.hadoop.hbase.backup.TestHFileArchiving org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster (still running, more coming). > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[
https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568841#comment-13568841
]
Jonathan Hsieh commented on HBASE-7290:
---
Online snapshots has been fairly stable for the past day or two on a 20 node
test cluster. I believe we are close to a trunk merge now.
I am currently syncing this HBASE-7290 branch to the current trunk. Most
fixups are trivial but fixups to these classes will probably warrant some
review.
{code}
# both modified:
hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java
# both modified:
hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
# both modified:
hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
{code}
I'll post a git branch to inspect.
After some testing of this I'll post a mega review and branch to review and
start the merge process.
> Online snapshots
> -
>
> Key: HBASE-7290
> URL: https://issues.apache.org/jira/browse/HBASE-7290
> Project: HBase
> Issue Type: Bug
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> HBASE-6055 will be closed when the offline snapshots pieces get merged with
> trunk. This umbrella issue has all the online snapshot specific patches.
> This will get merged once one of the implementations makes it into trunk.
> Other flavors of online snapshots can then be done as normal patches instead
> of on a development branch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13531137#comment-13531137 ] Jonathan Hsieh commented on HBASE-7290: --- Restored snapshot replay problem: HBASE-7346. I've filed it as part of offline snapshots since it affects that case as well. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13531135#comment-13531135 ] Jonathan Hsieh commented on HBASE-7290: --- Let's call this problem the 'snapshot replay problem', and move discussion to a new issue. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13531133#comment-13531133 ] Jonathan Hsieh commented on HBASE-7290: --- When I say 'restore' in step 4, I am specifically referring to a 'snapshot restore' operation. It doesn't necessarily have to happen after a crash. Suppose a user creates data, snapshots it, then puts bad data in. The use wants the old data back so he restores the snapshot (there is no crash anywhere here). While what you suggest is true, (they could restore again), this is unexpected, even in our weaker consistency model. If we have a table, have a crash and come back we essentially expect the table to be essentially the same as before. (without potentially tons of new data appear. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529168#comment-13529168 ] Ted Yu commented on HBASE-7290: --- Thanks for clarification, Jon. To my understanding, the restore action normally happens after a crash. Meaning, if customer really wants data as of ss1, they can restore ss1 again, right ? > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529155#comment-13529155 ] Jonathan Hsieh commented on HBASE-7290: --- It is not a concurrency problem at that granularity. The situation is a coarse-grained problem. The key problem is that writes that shouldn't be replayed (since they don't belong to the restored image), would not normally get replayed, but would potentially get replayed if recovery was triggered. Previously, without restore, we could depend on the timestamps -- if something was replayed but there was newer data, the newer data would win. In a restore situation, the "newer" data is has the old time stamps from before recovery, and new data that shouldn't get replayed could be. ex: 1) write 100 rows 2) ss1 (with logs) 3) write 50 rows 4) restore ss1 5) crash 6) writes from 1 and 3 both get replayed in log splitting recovery. Oops. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529144#comment-13529144 ] Ted Yu commented on HBASE-7290: --- @Jon: In the scenario you described above, do snapshot and restore happen on the same table in the same cluster ? If so, would you be able to utilize the following to control concurrency ? HBASE-7305 ZK based Read/Write locks for table operations > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528783#comment-13528783 ] Jonathan Hsieh commented on HBASE-7290: --- For now I'm going to focus on the simple flush hfile only snapshot. Log-based snapshots (simple log, and global consistent) actually has non-trivial problems when: 1) a snapshot is taken, 2) edits occur, 3) a restore happens and 4) a crash causes recovery to be triggered. This seems to allow writes from the period after the snapshot and before the restore to be recovered in a crash scenario. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528780#comment-13528780 ] Jonathan Hsieh commented on HBASE-7290: --- I've rebased on top of trunk 12/11/12's 4c9e80bacadf8e8fe0e0eee5b6477d6889e58456. Ongoing work lives here: (this gets force pushed freqently). https://github.com/jmhsieh/hbase/tree/snapshots-work There are also many branches with snapshot-work-MMDD (ex: snapshot-work-1210) with incremental progress. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511493#comment-13511493 ] Jonathan Hsieh commented on HBASE-7290: --- Ted, thanks, I've updated the description. > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > HBASE-6055 will be closed when the offline snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7290) Online snapshots
[ https://issues.apache.org/jira/browse/HBASE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511249#comment-13511249 ] Ted Yu commented on HBASE-7290: --- Were you referring to offline snapshot in the first sentence of the description ? > Online snapshots > - > > Key: HBASE-7290 > URL: https://issues.apache.org/jira/browse/HBASE-7290 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > HBASE-6055 will be closed when the online snapshots pieces get merged with > trunk. This umbrella issue has all the online snapshot specific patches. > This will get merged once one of the implementations makes it into trunk. > Other flavors of online snapshots can then be done as normal patches instead > of on a development branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
