[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14492731#comment-14492731 ] Jerry He commented on HBASE-11148: -- Canceled the patch. Will revisit after Procedure2. > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 2.0.0 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14492668#comment-14492668 ] Nick Dimiduk commented on HBASE-11148: -- Makes sense. These patches no longer apply then? If not, please cancel patch status so it doesn't trigger my filter. Thanks. > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 2.0.0, 1.1.0 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14491176#comment-14491176 ] Jerry He commented on HBASE-11148: -- Hi, [~ndimiduk] I had a discussion with [~mbertozzi] a while ago on this one. This can be better done on top of the new Procedure2 > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 2.0.0, 1.1.0 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484732#comment-14484732 ] Nick Dimiduk commented on HBASE-11148: -- Any progress here? Would be good to get for 1.1. > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 2.0.0, 1.1.0 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[
https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168382#comment-14168382
]
Hadoop QA commented on HBASE-11148:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12652564/HBASE-11148-trunk-v2.patch
against trunk revision .
ATTACHMENT ID: 12652564
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11315//console
This message is automatically generated.
> Provide a distributed procedure to globally roll logs
> -
>
> Key: HBASE-11148
> URL: https://issues.apache.org/jira/browse/HBASE-11148
> Project: HBase
> Issue Type: New Feature
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 2.0.0, 0.99.2
>
> Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch
>
>
> Propose a distributed procedure here to globally roll logs.
> Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a
> single RS.
> Some use cases may require that all the RS roll the logs at the same time and
> in a coordinated way. Also there may be requirement that some tasks be done
> together with the roll log on each region server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168377#comment-14168377 ] Enis Soztutar commented on HBASE-11148: --- Ping [~jmhsieh]. This looks like almost there. > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 2.0.0, 0.99.2 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060357#comment-14060357 ] Jerry He commented on HBASE-11148: -- Ping [~jmhsieh]. Have you got a chance to review the patch? Any additional comment? > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 0.99.0 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044966#comment-14044966 ] Jonathan Hsieh commented on HBASE-11148: Please add docs or file a docs jira to add these commands to the hbase ref guide. I'll do a deeper review of the main patch later today. > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 0.99.0 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[
https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044814#comment-14044814
]
Ted Yu commented on HBASE-11148:
lgtm
There're some typo's, e.g. "procdure":
{code}
+ new ForeignException("Interrupted while waiting for roll log
procdure to finish", e);
{code}
Add annotation for audience:
{code}
+public class RollLogSubprocedure extends Subprocedure {
{code}
Is the following method needed in RollLogSubprocedure ?
{code}
+ public void releaseBarrier() {
+// NO OP
+ }
{code}
> Provide a distributed procedure to globally roll logs
> -
>
> Key: HBASE-11148
> URL: https://issues.apache.org/jira/browse/HBASE-11148
> Project: HBase
> Issue Type: New Feature
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 0.99.0
>
> Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch
>
>
> Propose a distributed procedure here to globally roll logs.
> Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a
> single RS.
> Some use cases may require that all the RS roll the logs at the same time and
> in a coordinated way. Also there may be requirement that some tasks be done
> together with the roll log on each region server.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044433#comment-14044433 ] Jerry He commented on HBASE-11148: -- Hi, Matteo Thanks for the comment. I agree. On the other hand, Procedure is still a good mechanism for centrally kicked off tasks that need to be executed on RS in parallel. We can discuss if we need its 'strong consistency' (all have be successful to report success) -- whether this is needed in many use cases. We can also improve it to allow and report partial success in the future. For this particular use case, do you see any potential problem? Reporting a overall error even when half RS are successful seem a little 'wasteful'. But it is consistent with the name of the API and command. I can add comments in the API and command let user be aware. > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 0.99.0 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044405#comment-14044405 ] Matteo Bertozzi commented on HBASE-11148: - quick look at the code and looks ok, its kind of copy paste of what was done for the Flush. my main concern again is about the use of the Procedure for this kind of use. If you have a failure on a RS, the client will receive an error but half of the logs may be rolled. > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 0.99.0 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044404#comment-14044404 ] Jerry He commented on HBASE-11148: -- [~mbertozzi], [~jmhsieh], [~tedyu] Could you take a look? Your comments are much appreciated. > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 0.99.0 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[
https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044399#comment-14044399
]
Jerry He commented on HBASE-11148:
--
Since the QA job has not been kicked off, I am pasting my local 'mvn test'
result here (with sun jdk 7):
{code}
[INFO] Reactor Summary:
[INFO]
[INFO] HBase . SUCCESS [ 1.789 s]
[INFO] HBase - Common SUCCESS [ 23.715 s]
[INFO] HBase - Protocol .. SUCCESS [ 0.282 s]
[INFO] HBase - Client SUCCESS [ 47.981 s]
[INFO] HBase - Hadoop Compatibility .. SUCCESS [ 6.474 s]
[INFO] HBase - Hadoop Two Compatibility .. SUCCESS [ 1.418 s]
[INFO] HBase - Prefix Tree ... SUCCESS [ 2.981 s]
[INFO] HBase - Server SUCCESS [42:25 min]
[INFO] HBase - Testing Util .. SUCCESS [ 1.462 s]
[INFO] HBase - Thrift SUCCESS [01:38 min]
[INFO] HBase - Shell . SUCCESS [ 0.994 s]
[INFO] HBase - Integration Tests . SUCCESS [ 0.910 s]
[INFO] HBase - Examples .. SUCCESS [ 0.945 s]
[INFO] HBase - Assembly .. SUCCESS [ 0.983 s]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 45:34 min
[INFO] Finished at: 2014-06-25T23:48:19-08:00
[INFO] Final Memory: 47M/329M
[INFO]
{code}
> Provide a distributed procedure to globally roll logs
> -
>
> Key: HBASE-11148
> URL: https://issues.apache.org/jira/browse/HBASE-11148
> Project: HBase
> Issue Type: New Feature
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 0.99.0
>
> Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch
>
>
> Propose a distributed procedure here to globally roll logs.
> Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a
> single RS.
> Some use cases may require that all the RS roll the logs at the same time and
> in a coordinated way. Also there may be requirement that some tasks be done
> together with the roll log on each region server.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044395#comment-14044395 ] Jerry He commented on HBASE-11148: -- Attached v2 that added a shell ruby command. > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 0.99.0 > > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043346#comment-14043346 ] Jerry He commented on HBASE-11148: -- Attached patch for master branch. > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He > Fix For: 0.99.0 > > Attachments: HBASE-11148-trunk-v1.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995818#comment-13995818 ] stack commented on HBASE-11148: --- [~nidmhbase] Is there supposed to be a patch here sir? > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He > Fix For: 0.99.0 > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996529#comment-13996529 ] Demai Ni commented on HBASE-11148: -- [~jinghe], can we also have this one for 0.98? thanks... Demai > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He > Fix For: 0.99.0 > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995877#comment-13995877 ] Jerry He commented on HBASE-11148: -- [~stack] I will provide a patch shortly. > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He > Fix For: 0.99.0 > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
