[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451262#comment-13451262 ] Hadoop QA commented on HBASE-6730: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544343/HBASE-6730-4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause mvn compile goal to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2829//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2829//console This message is automatically generated. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, > HBASE-6730-3.patch, HBASE-6730-4.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6730: - Attachment: HBASE-6730-4.patch Since the numbers come in at regular intervals we don't need the more complex method. I implemented the simplest version. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, > HBASE-6730-3.patch, HBASE-6730-4.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451252#comment-13451252 ] stack commented on HBASE-6730: -- bq. I think his model would give us better result. Sure. Make a patch? Unless you have objection, will commit over the w/e. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, > HBASE-6730-3.patch, HBASE-6730-4.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6241) HBaseCluster interface for interacting with the cluster from system tests
[ https://issues.apache.org/jira/browse/HBASE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451249#comment-13451249 ] stack commented on HBASE-6241: -- I took a second pass (it looks great...a few more minor items you could get in another pass). No need to break it down. Lets get it into trunk. Backport to 0.94 could be ugly. Its nice as it is here in trunk in own module. Looking forward to Thursday chat on this (Lets have it committed before then -- smile). Good on you Enis. > HBaseCluster interface for interacting with the cluster from system tests > -- > > Key: HBASE-6241 > URL: https://issues.apache.org/jira/browse/HBASE-6241 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: HBASE-6241_v0.2.patch, HBASE-6241_v1.patch > > > We need to abstract away the cluster interactions for system tests running on > actual clusters. > MiniHBaseCluster and RealHBaseCluster should both implement this interface, > and system tests should work with both. > I'll split Devaraj's patch in HBASE-6053 for the initial version. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451247#comment-13451247 ] Ted Yu commented on HBASE-6730: --- See Ted's blog: http://tdunning.blogspot.com/2011/03/exponential-weighted-averages-with.html I think his model would give us better result. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, > HBASE-6730-3.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[
https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451246#comment-13451246
]
Ted Yu commented on HBASE-3909:
---
I think the test failure was due to the following in
initializeZKBasedSystemTrackers():
{code}
this.catalogTracker = createCatalogTracker(this.zooKeeper, this.conf,
this, conf.getInt("hbase.master.catalog.timeout", 60));
{code}
nit: exchange the order of the parameters so that assertion is easier to read:
{code}
assertEquals(catalogTimeout, Integer.MAX_VALUE);
{code}
In this particular case, I think the assertion is not needed - the actual value
may change if someone modifies the snippet cited above.
> Add dynamic config
> --
>
> Key: HBASE-3909
> URL: https://issues.apache.org/jira/browse/HBASE-3909
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Fix For: 0.96.0
>
> Attachments: 3909_090712-2.patch, 3909.v1, 3909-v1.patch, HBase
> Cluster Config Details.xlsx, patch-v2.patch
>
>
> I'm sure this issue exists already, at least as part of the discussion around
> making online schema edits possible, but no hard this having its own issue.
> Ted started a conversation on this topic up on dev and Todd suggested we
> lookd at how Hadoop did it over in HADOOP-7001
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451243#comment-13451243 ] stack commented on HBASE-6730: -- What are you talking of Ted? I'll commit unless objection. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, > HBASE-6730-3.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451237#comment-13451237 ] Hadoop QA commented on HBASE-3909: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544331/3909_090712-2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause mvn compile goal to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestHBaseDynamicConfiguration org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.client.TestShell Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2828//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2828//console This message is automatically generated. > Add dynamic config > -- > > Key: HBASE-3909 > URL: https://issues.apache.org/jira/browse/HBASE-3909 > Project: HBase > Issue Type: Bug >Reporter: stack > Fix For: 0.96.0 > > Attachments: 3909_090712-2.patch, 3909.v1, 3909-v1.patch, HBase > Cluster Config Details.xlsx, patch-v2.patch > > > I'm sure this issue exists already, at least as part of the discussion around > making online schema edits possible, but no hard this having its own issue. > Ted started a conversation on this topic up on dev and Todd suggested we > lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451236#comment-13451236 ] Ted Yu commented on HBASE-6730: --- This effort reminded me of discussion titled 'decaying moving average' from last year. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, > HBASE-6730-3.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451231#comment-13451231 ] Elliott Clark commented on HBASE-6730: -- Test errors were OOM's. They pass on my machine. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, > HBASE-6730-3.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451229#comment-13451229 ] Hadoop QA commented on HBASE-6730: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544339/HBASE-6730-3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause mvn compile goal to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2827//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2827//console This message is automatically generated. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, > HBASE-6730-3.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-3909: -- Status: Patch Available (was: Open) > Add dynamic config > -- > > Key: HBASE-3909 > URL: https://issues.apache.org/jira/browse/HBASE-3909 > Project: HBase > Issue Type: Bug >Reporter: stack > Fix For: 0.96.0 > > Attachments: 3909_090712-2.patch, 3909.v1, 3909-v1.patch, HBase > Cluster Config Details.xlsx, patch-v2.patch > > > I'm sure this issue exists already, at least as part of the discussion around > making online schema edits possible, but no hard this having its own issue. > Ted started a conversation on this topic up on dev and Todd suggested we > lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451221#comment-13451221 ] Elliott Clark commented on HBASE-6730: -- I had missed a few break statements. Fixed. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, > HBASE-6730-3.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6730: - Attachment: HBASE-6730-3.patch > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, > HBASE-6730-3.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6340) HBase RPC should allow protocol extension with common interfaces.
[ https://issues.apache.org/jira/browse/HBASE-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451213#comment-13451213 ] Hudson commented on HBASE-6340: --- Integrated in HBase-0.94 #456 (See [https://builds.apache.org/job/HBase-0.94/456/]) HBASE-6340 HBase RPC should allow protocol extension with common interfaces. (Revision 1382207) HBASE-6340 HBase RPC should allow protocol extension with common interfaces. (Revision 1382206) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/coprocessor/Exec.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java > HBase RPC should allow protocol extension with common interfaces. > - > > Key: HBASE-6340 > URL: https://issues.apache.org/jira/browse/HBASE-6340 > Project: HBase > Issue Type: Bug > Components: coprocessors, regionserver >Affects Versions: 0.92.0 >Reporter: Konstantin Shvachko >Assignee: Konstantin Shvachko > Fix For: 0.94.2 > > Attachments: 6340-RPCInvocation.patch, RPCInvocation.patch > > > HBase RPC fails if MyProtocol extends an interface, which is not a > VersionedProtocol even if MyProtocol also directly extends VersionedProtocol. > The reason is that rpc Invocation uses Method.getDeclaringClass(), which > returns the interface class rather than the class of MyProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451199#comment-13451199
]
Ted Yu commented on HBASE-6730:
---
Please fix test failure in TestRegionRebalancing:
{code}
java.lang.AssertionError: RegionLoad cost type not supported.
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.getRegionLoadCost(StochasticLoadBalancer.java:686)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.computeRegionLoadCost(StochasticLoadBalancer.java:654)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.computeCost(StochasticLoadBalancer.java:429)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:192)
at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1302)
at
org.apache.hadoop.hbase.TestRegionRebalancing.testRebalanceOnRegionServerNumberChange(TestRegionRebalancing.java:118)
{code}
> Enable rolling averages in StochasticLoadBalancer
> --
>
> Key: HBASE-6730
> URL: https://issues.apache.org/jira/browse/HBASE-6730
> Project: HBase
> Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.96.0
>
> Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch
>
>
> Now that all of the ServerLoad transitions into pb are done. the load
> balancer should get more updates about the current state of the cluster.
> This should be used to allow StochasticLoadBalancer to get rolling averages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6658) Rename WritableByteArrayComparable to something not mentioning Writable
[ https://issues.apache.org/jira/browse/HBASE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6658: -- Status: Open (was: Patch Available) > Rename WritableByteArrayComparable to something not mentioning Writable > --- > > Key: HBASE-6658 > URL: https://issues.apache.org/jira/browse/HBASE-6658 > Project: HBase > Issue Type: Bug >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6658.patch, HBASE-6658-v3.patch > > > After HBASE-6477, WritableByteArrayComparable will no longer be Writable, so > should be renamed. > Current idea is ByteArrayComparator (since all the derived classes are > *Comparator not *Comparable), but I'm open to suggestions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6658) Rename WritableByteArrayComparable to something not mentioning Writable
[ https://issues.apache.org/jira/browse/HBASE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6658: -- Status: Patch Available (was: Open) > Rename WritableByteArrayComparable to something not mentioning Writable > --- > > Key: HBASE-6658 > URL: https://issues.apache.org/jira/browse/HBASE-6658 > Project: HBase > Issue Type: Bug >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6658.patch, HBASE-6658-v3.patch > > > After HBASE-6477, WritableByteArrayComparable will no longer be Writable, so > should be renamed. > Current idea is ByteArrayComparator (since all the derived classes are > *Comparator not *Comparable), but I'm open to suggestions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6658) Rename WritableByteArrayComparable to something not mentioning Writable
[ https://issues.apache.org/jira/browse/HBASE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451194#comment-13451194 ] Hadoop QA commented on HBASE-6658: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544322/HBASE-6658.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause mvn compile goal to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient org.apache.hadoop.hbase.replication.TestReplication Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2823//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2823//console This message is automatically generated. > Rename WritableByteArrayComparable to something not mentioning Writable > --- > > Key: HBASE-6658 > URL: https://issues.apache.org/jira/browse/HBASE-6658 > Project: HBase > Issue Type: Bug >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6658.patch, HBASE-6658-v3.patch > > > After HBASE-6477, WritableByteArrayComparable will no longer be Writable, so > should be renamed. > Current idea is ByteArrayComparator (since all the derived classes are > *Comparator not *Comparable), but I'm open to suggestions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451178#comment-13451178 ] Hadoop QA commented on HBASE-6730: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544319/HBASE-6730-2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause mvn compile goal to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestRegionRebalancing Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2822//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2822//console This message is automatically generated. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6658) Rename WritableByteArrayComparable to something not mentioning Writable
[ https://issues.apache.org/jira/browse/HBASE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-6658: -- Attachment: (was: HBASE-6710-v2.patch) > Rename WritableByteArrayComparable to something not mentioning Writable > --- > > Key: HBASE-6658 > URL: https://issues.apache.org/jira/browse/HBASE-6658 > Project: HBase > Issue Type: Bug >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6658.patch, HBASE-6658-v3.patch > > > After HBASE-6477, WritableByteArrayComparable will no longer be Writable, so > should be renamed. > Current idea is ByteArrayComparator (since all the derived classes are > *Comparator not *Comparable), but I'm open to suggestions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6658) Rename WritableByteArrayComparable to something not mentioning Writable
[ https://issues.apache.org/jira/browse/HBASE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-6658: -- Attachment: HBASE-6658-v3.patch Messed that patch up, trying again. > Rename WritableByteArrayComparable to something not mentioning Writable > --- > > Key: HBASE-6658 > URL: https://issues.apache.org/jira/browse/HBASE-6658 > Project: HBase > Issue Type: Bug >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6658.patch, HBASE-6658-v3.patch > > > After HBASE-6477, WritableByteArrayComparable will no longer be Writable, so > should be renamed. > Current idea is ByteArrayComparator (since all the derived classes are > *Comparator not *Comparable), but I'm open to suggestions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Subbu M Iyer updated HBASE-3909: Attachment: 3909_090712-2.patch > Add dynamic config > -- > > Key: HBASE-3909 > URL: https://issues.apache.org/jira/browse/HBASE-3909 > Project: HBase > Issue Type: Bug >Reporter: stack > Fix For: 0.96.0 > > Attachments: 3909_090712-2.patch, 3909.v1, 3909-v1.patch, HBase > Cluster Config Details.xlsx, patch-v2.patch > > > I'm sure this issue exists already, at least as part of the discussion around > making online schema edits possible, but no hard this having its own issue. > Ted started a conversation on this topic up on dev and Todd suggested we > lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451167#comment-13451167 ] Subbu M Iyer commented on HBASE-3909: - Added latest patch with protobuf support and addressed the review comments. All 7 tests pass. Here is a sample of how things work from shell: hbase(main):007:0> get_config 'hbase.balancer.period' config_value = : 2 0 row(s) in 0.0170 seconds hbase(main):008:0> update_config 'hbase.balancer.period', '35000' updated config key with new value 0 row(s) in 0.0200 seconds hbase(main):009:0> get_config 'hbase.balancer.period' config_value = : 35000 0 row(s) in 0.0180 seconds > Add dynamic config > -- > > Key: HBASE-3909 > URL: https://issues.apache.org/jira/browse/HBASE-3909 > Project: HBase > Issue Type: Bug >Reporter: stack > Fix For: 0.96.0 > > Attachments: 3909.v1, 3909-v1.patch, HBase Cluster Config > Details.xlsx, patch-v2.patch > > > I'm sure this issue exists already, at least as part of the discussion around > making online schema edits possible, but no hard this having its own issue. > Ted started a conversation on this topic up on dev and Todd suggested we > lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6713) Stopping META/ROOT RS may take 50mins when some region is splitting
[ https://issues.apache.org/jira/browse/HBASE-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451166#comment-13451166 ] Hudson commented on HBASE-6713: --- Integrated in HBase-0.92 #562 (See [https://builds.apache.org/job/HBase-0.92/562/]) HBASE-6713 Stopping META/ROOT RS may take 50mins when some region is splitting (Chunhui) (Revision 1382162) Result = FAILURE tedyu : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Stopping META/ROOT RS may take 50mins when some region is splitting > --- > > Key: HBASE-6713 > URL: https://issues.apache.org/jira/browse/HBASE-6713 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0, 0.92.3, 0.94.3 > > Attachments: 6713.92-94, 6713v3.patch, HBASE-6713.patch, > HBASE-6713v2.patch > > > When we stop the RS carrying ROOT/META, if it is in the splitting for some > region, the whole stopping process may take 50 mins. > The reason is : > 1.ROOT/META region is closed when stopping the regionserver > 2.The Split Transaction failed updating META and it will retry > 3.The retry num is 100, and the total time is about 50 mins as default; > This configuration is set by > HConnectionManager#setServerSideHConnectionRetries > I think 50 mins is too long to acceptable, my suggested solution is closing > MetaTable regions after the compact/split thread is closed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6658) Rename WritableByteArrayComparable to something not mentioning Writable
[ https://issues.apache.org/jira/browse/HBASE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451164#comment-13451164 ] Hadoop QA commented on HBASE-6658: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544329/HBASE-6710-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2824//console This message is automatically generated. > Rename WritableByteArrayComparable to something not mentioning Writable > --- > > Key: HBASE-6658 > URL: https://issues.apache.org/jira/browse/HBASE-6658 > Project: HBase > Issue Type: Bug >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6658.patch, HBASE-6710-v2.patch > > > After HBASE-6477, WritableByteArrayComparable will no longer be Writable, so > should be renamed. > Current idea is ByteArrayComparator (since all the derived classes are > *Comparator not *Comparable), but I'm open to suggestions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6658) Rename WritableByteArrayComparable to something not mentioning Writable
[ https://issues.apache.org/jira/browse/HBASE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-6658: -- Attachment: HBASE-6710-v2.patch Seems reasonable to me, Ted. Attached patch doing as you suggested. > Rename WritableByteArrayComparable to something not mentioning Writable > --- > > Key: HBASE-6658 > URL: https://issues.apache.org/jira/browse/HBASE-6658 > Project: HBase > Issue Type: Bug >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6658.patch, HBASE-6710-v2.patch > > > After HBASE-6477, WritableByteArrayComparable will no longer be Writable, so > should be renamed. > Current idea is ByteArrayComparator (since all the derived classes are > *Comparator not *Comparable), but I'm open to suggestions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6714) TestMultiSlaveReplication#testMultiSlaveReplication may fail
[ https://issues.apache.org/jira/browse/HBASE-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451159#comment-13451159 ] Himanshu Vashishtha commented on HBASE-6714: jenkins is green; looks like an unrelated failure. > TestMultiSlaveReplication#testMultiSlaveReplication may fail > > > Key: HBASE-6714 > URL: https://issues.apache.org/jira/browse/HBASE-6714 > Project: HBase > Issue Type: Bug > Components: replication, test >Affects Versions: 0.92.0, 0.94.0 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha >Priority: Minor > Attachments: HBase-6714-v1.patch > > > java.lang.AssertionError: expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:472) > at org.junit.Assert.assertEquals(Assert.java:456) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.checkRow(TestMultiSlaveReplication.java:203) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.testMultiSlaveReplication(TestMultiSlaveReplication.java:188) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > TestMultiSlaveReplication->testMultiSlaveReplication failed in our local > build citing that "row" was not replicated to second peer. This is because > after inserting "row", log is rolled and we look for "row2" in both the > clusters and then we check for existence of "row" in both clusters. > Meanwhile, Replication thread was sleeping for the second cluster and Row > "row2" is not present in the second cluster from the very beginning. So, the > "row2" existence check succeeds and control move on to find "row" in both > clusters where it fails for the second cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6743) Move EnvironmentEdge classes to hbase-common
[ https://issues.apache.org/jira/browse/HBASE-6743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HBASE-6743: Attachment: 6743v2.txt Here is an updated patch using git. > Move EnvironmentEdge classes to hbase-common > > > Key: HBASE-6743 > URL: https://issues.apache.org/jira/browse/HBASE-6743 > Project: HBase > Issue Type: Improvement >Reporter: Chris Trezzo >Assignee: Chris Trezzo >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6743v1.txt, 6743v2.txt > > > Move the classes related to EnvironmentEdge to the hbase-common module. If we > want to replace all occurrences of System.currentTimeMillis() with > EnvironmentEdgeManager.currentTimeMillis(), we need to be able to reference > these classes from multiple modules. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6713) Stopping META/ROOT RS may take 50mins when some region is splitting
[ https://issues.apache.org/jira/browse/HBASE-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451155#comment-13451155 ] Hudson commented on HBASE-6713: --- Integrated in HBase-TRUNK #3315 (See [https://builds.apache.org/job/HBase-TRUNK/3315/]) HBASE-6713 Stopping META/ROOT RS may take 50mins when some region is splitting (Chunhui) (Revision 1382164) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Stopping META/ROOT RS may take 50mins when some region is splitting > --- > > Key: HBASE-6713 > URL: https://issues.apache.org/jira/browse/HBASE-6713 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0, 0.92.3, 0.94.3 > > Attachments: 6713.92-94, 6713v3.patch, HBASE-6713.patch, > HBASE-6713v2.patch > > > When we stop the RS carrying ROOT/META, if it is in the splitting for some > region, the whole stopping process may take 50 mins. > The reason is : > 1.ROOT/META region is closed when stopping the regionserver > 2.The Split Transaction failed updating META and it will retry > 3.The retry num is 100, and the total time is about 50 mins as default; > This configuration is set by > HConnectionManager#setServerSideHConnectionRetries > I think 50 mins is too long to acceptable, my suggested solution is closing > MetaTable regions after the compact/split thread is closed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6742) Change default test parallelisation level to 5
[ https://issues.apache.org/jira/browse/HBASE-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451154#comment-13451154 ] Hudson commented on HBASE-6742: --- Integrated in HBase-TRUNK #3315 (See [https://builds.apache.org/job/HBase-TRUNK/3315/]) HBASE-6742 Change default test parallelisation level to 5 (Revision 1382127) Result = FAILURE nkeywal : Files : * /hbase/trunk/pom.xml > Change default test parallelisation level to 5 > -- > > Key: HBASE-6742 > URL: https://issues.apache.org/jira/browse/HBASE-6742 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: hbase-6742.v1.patch > > > Tests will be faster. > Not visible if a test hangs for 15 minutes. But they should not hang. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6714) TestMultiSlaveReplication#testMultiSlaveReplication may fail
[ https://issues.apache.org/jira/browse/HBASE-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451149#comment-13451149 ] Hadoop QA commented on HBASE-6714: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544318/HBase-6714-v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause mvn compile goal to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2821//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2821//console This message is automatically generated. > TestMultiSlaveReplication#testMultiSlaveReplication may fail > > > Key: HBASE-6714 > URL: https://issues.apache.org/jira/browse/HBASE-6714 > Project: HBase > Issue Type: Bug > Components: replication, test >Affects Versions: 0.92.0, 0.94.0 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha >Priority: Minor > Attachments: HBase-6714-v1.patch > > > java.lang.AssertionError: expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:472) > at org.junit.Assert.assertEquals(Assert.java:456) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.checkRow(TestMultiSlaveReplication.java:203) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.testMultiSlaveReplication(TestMultiSlaveReplication.java:188) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > TestMultiSlaveReplication->testMultiSlaveReplication failed in our local > build citing that "row" was not replicated to second peer. This is because > after inserting "row", log is rolled and we look for "row2" in both the > clusters and then we check for existence of "row" in both clusters. > Meanwhile, Replication thread was sleeping for the second cluster and Row > "row2" is not present in the second cluster from the very beginning. So, the > "row2" existence check succeeds and control move on to find "row" in both > clusters where it fails for the second cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6658) Rename WritableByteArrayComparable to something not mentioning Writable
[ https://issues.apache.org/jira/browse/HBASE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451145#comment-13451145 ] Ted Yu commented on HBASE-6658: --- See http://docs.oracle.com/javase/6/docs/api/java/util/Comparator.html While Comparable defines compareTo() method which is not in Comparator interface. I suggest keeping Comparable in class names and only dropping Writable. > Rename WritableByteArrayComparable to something not mentioning Writable > --- > > Key: HBASE-6658 > URL: https://issues.apache.org/jira/browse/HBASE-6658 > Project: HBase > Issue Type: Bug >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6658.patch > > > After HBASE-6477, WritableByteArrayComparable will no longer be Writable, so > should be renamed. > Current idea is ByteArrayComparator (since all the derived classes are > *Comparator not *Comparable), but I'm open to suggestions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6742) Change default test parallelisation level to 5
[ https://issues.apache.org/jira/browse/HBASE-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451141#comment-13451141 ] Hudson commented on HBASE-6742: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #165 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/165/]) HBASE-6742 Change default test parallelisation level to 5 (Revision 1382127) Result = FAILURE nkeywal : Files : * /hbase/trunk/pom.xml > Change default test parallelisation level to 5 > -- > > Key: HBASE-6742 > URL: https://issues.apache.org/jira/browse/HBASE-6742 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: hbase-6742.v1.patch > > > Tests will be faster. > Not visible if a test hangs for 15 minutes. But they should not hang. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6713) Stopping META/ROOT RS may take 50mins when some region is splitting
[ https://issues.apache.org/jira/browse/HBASE-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451142#comment-13451142 ] Hudson commented on HBASE-6713: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #165 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/165/]) HBASE-6713 Stopping META/ROOT RS may take 50mins when some region is splitting (Chunhui) (Revision 1382164) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Stopping META/ROOT RS may take 50mins when some region is splitting > --- > > Key: HBASE-6713 > URL: https://issues.apache.org/jira/browse/HBASE-6713 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0, 0.92.3, 0.94.3 > > Attachments: 6713.92-94, 6713v3.patch, HBASE-6713.patch, > HBASE-6713v2.patch > > > When we stop the RS carrying ROOT/META, if it is in the splitting for some > region, the whole stopping process may take 50 mins. > The reason is : > 1.ROOT/META region is closed when stopping the regionserver > 2.The Split Transaction failed updating META and it will retry > 3.The retry num is 100, and the total time is about 50 mins as default; > This configuration is set by > HConnectionManager#setServerSideHConnectionRetries > I think 50 mins is too long to acceptable, my suggested solution is closing > MetaTable regions after the compact/split thread is closed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5937) Refactor HLog into an interface.
[ https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira updated HBASE-5937: Attachment: org.apache.hadoop.hbase.client.TestMultiParallel-output.txt Attached... > Refactor HLog into an interface. > > > Key: HBASE-5937 > URL: https://issues.apache.org/jira/browse/HBASE-5937 > Project: HBase > Issue Type: Sub-task >Reporter: Li Pi >Assignee: Flavio Junqueira >Priority: Minor > Attachments: > org.apache.hadoop.hbase.client.TestMultiParallel-output.txt > > > What the summary says. Create HLog interface. Make current implementation use > it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6658) Rename WritableByteArrayComparable to something not mentioning Writable
[ https://issues.apache.org/jira/browse/HBASE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-6658: -- Status: Patch Available (was: Open) > Rename WritableByteArrayComparable to something not mentioning Writable > --- > > Key: HBASE-6658 > URL: https://issues.apache.org/jira/browse/HBASE-6658 > Project: HBase > Issue Type: Bug >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6658.patch > > > After HBASE-6477, WritableByteArrayComparable will no longer be Writable, so > should be renamed. > Current idea is ByteArrayComparator (since all the derived classes are > *Comparator not *Comparable), but I'm open to suggestions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6658) Rename WritableByteArrayComparable to something not mentioning Writable
[ https://issues.apache.org/jira/browse/HBASE-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-6658: -- Attachment: HBASE-6658.patch * Attached HBASE-6658.patch * Does the following: - Renames WritableByteArrayComparable to ByteArrayComparator - On the protobuf side, renames ByteArrayComparable to ByteArrayComparator. - In the rest ScannerModel, renames WritableByteArrayComparableModel to ByteArrayComparatorModel (I think this is okay to do, don't know the REST code well). > Rename WritableByteArrayComparable to something not mentioning Writable > --- > > Key: HBASE-6658 > URL: https://issues.apache.org/jira/browse/HBASE-6658 > Project: HBase > Issue Type: Bug >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-6658.patch > > > After HBASE-6477, WritableByteArrayComparable will no longer be Writable, so > should be renamed. > Current idea is ByteArrayComparator (since all the derived classes are > *Comparator not *Comparable), but I'm open to suggestions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6412) Move external servers to metrics2 (thrift,thrift2,rest)
[ https://issues.apache.org/jira/browse/HBASE-6412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451133#comment-13451133 ] Hadoop QA commented on HBASE-6412: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544309/HBASE-6412-5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 73 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause mvn compile goal to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2819//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2819//console This message is automatically generated. > Move external servers to metrics2 (thrift,thrift2,rest) > --- > > Key: HBASE-6412 > URL: https://issues.apache.org/jira/browse/HBASE-6412 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.96.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Blocker > Attachments: HBASE-6412-0.patch, HBASE-6412-1.patch, > HBASE-6412-2.patch, HBASE-6412-3.patch, HBASE-6412-4.patch, HBASE-6412-5.patch > > > Implement metrics2 for all the external servers: > * Thrift > * Thrift2 > * Rest -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6340) HBase RPC should allow protocol extension with common interfaces.
[ https://issues.apache.org/jira/browse/HBASE-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6340. -- Resolution: Fixed Fix Version/s: 0.94.2 Hadoop Flags: Reviewed Committed to 0.94 after running tests locally including the one this patch adds (Hope that is OK Lars; committed to help a downstream project). Closing this issue. Lets open new one Konstantin to deal w/ trunk gymastics. Thanks for patch Konstantin (and Ted) > HBase RPC should allow protocol extension with common interfaces. > - > > Key: HBASE-6340 > URL: https://issues.apache.org/jira/browse/HBASE-6340 > Project: HBase > Issue Type: Bug > Components: coprocessors, regionserver >Affects Versions: 0.92.0 >Reporter: Konstantin Shvachko >Assignee: Konstantin Shvachko > Fix For: 0.94.2 > > Attachments: 6340-RPCInvocation.patch, RPCInvocation.patch > > > HBase RPC fails if MyProtocol extends an interface, which is not a > VersionedProtocol even if MyProtocol also directly extends VersionedProtocol. > The reason is that rpc Invocation uses Method.getDeclaringClass(), which > returns the interface class rather than the class of MyProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6241) HBaseCluster interface for interacting with the cluster from system tests
[ https://issues.apache.org/jira/browse/HBASE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451125#comment-13451125 ] Enis Soztutar commented on HBASE-6241: -- I think I need to rebase the patch, and incorporate the review suggestions. You were going to take a second pass, i believe. I can also break down the patch into 2, if it is too big to review properly. The reason our system tests are not converted to this is that the patch is not finalized yet. But after this, I will spend some time on converting more tests. Also I will backport this to 0.94, and expect the community to pick up and write/convert more tests using this framework. I'll provide an overview of this issue for the developer meetup on Tuesday, and I'll attach the bits here, so that we can further discuss. > HBaseCluster interface for interacting with the cluster from system tests > -- > > Key: HBASE-6241 > URL: https://issues.apache.org/jira/browse/HBASE-6241 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: HBASE-6241_v0.2.patch, HBASE-6241_v1.patch > > > We need to abstract away the cluster interactions for system tests running on > actual clusters. > MiniHBaseCluster and RealHBaseCluster should both implement this interface, > and system tests should work with both. > I'll split Devaraj's patch in HBASE-6053 for the initial version. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451126#comment-13451126
]
Ted Yu commented on HBASE-6730:
---
{code}
+ private static final String KEEP_REGION_LOADS =
"hbase.master.balancer.stochastic.numRegionLoadsToRemember";
{code}
Line too long. The convention is that we don't need to capitalize letters in
config key.
{code}
+ //We're only going to keep 15. So if there are that many already
take the last 14
{code}
I think we should refer to numRegionLoadsToRemember instead of hardcoded values
(15 and 14)
{code}
+for(RegionLoad rl:regionLoadList) {
{code}
nit: insert spaces in the above statement.
> Enable rolling averages in StochasticLoadBalancer
> --
>
> Key: HBASE-6730
> URL: https://issues.apache.org/jira/browse/HBASE-6730
> Project: HBase
> Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.96.0
>
> Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch
>
>
> Now that all of the ServerLoad transitions into pb are done. the load
> balancer should get more updates about the current state of the cluster.
> This should be used to allow StochasticLoadBalancer to get rolling averages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451121#comment-13451121 ] stack commented on HBASE-6730: -- I'm +1 on commit. Will try removing the public from the new classes on commit so they are package private. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps
[ https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451119#comment-13451119 ] Hadoop QA commented on HBASE-6707: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544304/hbase-6707-v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause mvn compile goal to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2820//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2820//console This message is automatically generated. > TEST > org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables > flaps > > > Key: HBASE-6707 > URL: https://issues.apache.org/jira/browse/HBASE-6707 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sameer Vaishampayan >Assignee: Jesse Yates >Priority: Critical > Fix For: 0.96.0 > > Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch > > > https://builds.apache.org/job/HBase-TRUNK/3293/ > Error Message > Archived HFiles > (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam) > should have gotten deleted, but didn't, remaining > files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c] > Stacktrace > java.lang.AssertionError: Archived HFiles > (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam) > should have gotten deleted, but didn't, remaining > files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c] > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.assertTrue(Assert.java:43) > at org.junit.Assert.assertNull(Assert.java:551) > at > org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6730: - Attachment: HBASE-6730-2.patch New patch that makes more clear what's being removed and makes sure that there are no concurrent accesses. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451117#comment-13451117 ] Ted Yu commented on HBASE-6730: --- Yes, a check would be fine. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451116#comment-13451116 ] stack commented on HBASE-6730: -- @Ted Just to say we don't run w/ assertions enabled so you mean add a check? > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6714) TestMultiSlaveReplication#testMultiSlaveReplication may fail
[ https://issues.apache.org/jira/browse/HBASE-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-6714: --- Priority: Minor (was: Major) > TestMultiSlaveReplication#testMultiSlaveReplication may fail > > > Key: HBASE-6714 > URL: https://issues.apache.org/jira/browse/HBASE-6714 > Project: HBase > Issue Type: Bug > Components: replication, test >Affects Versions: 0.92.0, 0.94.0 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha >Priority: Minor > Attachments: HBase-6714-v1.patch > > > java.lang.AssertionError: expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:472) > at org.junit.Assert.assertEquals(Assert.java:456) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.checkRow(TestMultiSlaveReplication.java:203) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.testMultiSlaveReplication(TestMultiSlaveReplication.java:188) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > TestMultiSlaveReplication->testMultiSlaveReplication failed in our local > build citing that "row" was not replicated to second peer. This is because > after inserting "row", log is rolled and we look for "row2" in both the > clusters and then we check for existence of "row" in both clusters. > Meanwhile, Replication thread was sleeping for the second cluster and Row > "row2" is not present in the second cluster from the very beginning. So, the > "row2" existence check succeeds and control move on to find "row" in both > clusters where it fails for the second cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6713) Stopping META/ROOT RS may take 50mins when some region is splitting
[ https://issues.apache.org/jira/browse/HBASE-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451112#comment-13451112 ] Hudson commented on HBASE-6713: --- Integrated in HBase-0.94 #455 (See [https://builds.apache.org/job/HBase-0.94/455/]) HBASE-6713 Stopping META/ROOT RS may take 50mins when some region is splitting (Chunhui) (Revision 1382163) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Stopping META/ROOT RS may take 50mins when some region is splitting > --- > > Key: HBASE-6713 > URL: https://issues.apache.org/jira/browse/HBASE-6713 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0, 0.92.3, 0.94.3 > > Attachments: 6713.92-94, 6713v3.patch, HBASE-6713.patch, > HBASE-6713v2.patch > > > When we stop the RS carrying ROOT/META, if it is in the splitting for some > region, the whole stopping process may take 50 mins. > The reason is : > 1.ROOT/META region is closed when stopping the regionserver > 2.The Split Transaction failed updating META and it will retry > 3.The retry num is 100, and the total time is about 50 mins as default; > This configuration is set by > HConnectionManager#setServerSideHConnectionRetries > I think 50 mins is too long to acceptable, my suggested solution is closing > MetaTable regions after the compact/split thread is closed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451109#comment-13451109 ] Ted Yu commented on HBASE-6730: --- bq. When the list comes in it will have indices 0 - 14 So rLoads.size() > numRegionLoadsToRemember+1 won't happen. Mind adding an assertion ? > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6714) TestMultiSlaveReplication#testMultiSlaveReplication may fail
[ https://issues.apache.org/jira/browse/HBASE-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-6714: --- Attachment: HBase-6714-v1.patch Passed locally in a loop of 15; enqueued jenkins. > TestMultiSlaveReplication#testMultiSlaveReplication may fail > > > Key: HBASE-6714 > URL: https://issues.apache.org/jira/browse/HBASE-6714 > Project: HBase > Issue Type: Bug > Components: replication, test >Affects Versions: 0.92.0, 0.94.0 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Attachments: HBase-6714-v1.patch > > > java.lang.AssertionError: expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:472) > at org.junit.Assert.assertEquals(Assert.java:456) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.checkRow(TestMultiSlaveReplication.java:203) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.testMultiSlaveReplication(TestMultiSlaveReplication.java:188) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > TestMultiSlaveReplication->testMultiSlaveReplication failed in our local > build citing that "row" was not replicated to second peer. This is because > after inserting "row", log is rolled and we look for "row2" in both the > clusters and then we check for existence of "row" in both clusters. > Meanwhile, Replication thread was sleeping for the second cluster and Row > "row2" is not present in the second cluster from the very beginning. So, the > "row2" existence check succeeds and control move on to find "row" in both > clusters where it fails for the second cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6714) TestMultiSlaveReplication#testMultiSlaveReplication may fail
[ https://issues.apache.org/jira/browse/HBASE-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-6714: --- Status: Patch Available (was: Open) > TestMultiSlaveReplication#testMultiSlaveReplication may fail > > > Key: HBASE-6714 > URL: https://issues.apache.org/jira/browse/HBASE-6714 > Project: HBase > Issue Type: Bug > Components: replication, test >Affects Versions: 0.94.0, 0.92.0 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Attachments: HBase-6714-v1.patch > > > java.lang.AssertionError: expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:472) > at org.junit.Assert.assertEquals(Assert.java:456) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.checkRow(TestMultiSlaveReplication.java:203) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.testMultiSlaveReplication(TestMultiSlaveReplication.java:188) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > TestMultiSlaveReplication->testMultiSlaveReplication failed in our local > build citing that "row" was not replicated to second peer. This is because > after inserting "row", log is rolled and we look for "row2" in both the > clusters and then we check for existence of "row" in both clusters. > Meanwhile, Replication thread was sleeping for the second cluster and Row > "row2" is not present in the second cluster from the very beginning. So, the > "row2" existence check succeeds and control move on to find "row" in both > clusters where it fails for the second cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5354) Source to standalone deployment script
[ https://issues.apache.org/jira/browse/HBASE-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451108#comment-13451108 ] Elliott Clark commented on HBASE-5354: -- Looks good. Minor nit. There some odd indenting at the beginning of the script. > Source to standalone deployment script > -- > > Key: HBASE-5354 > URL: https://issues.apache.org/jira/browse/HBASE-5354 > Project: HBase > Issue Type: New Feature > Components: build, scripts >Affects Versions: 0.94.0 >Reporter: Jesse Yates >Assignee: Jesse Yates >Priority: Minor > Attachments: bash_HBASE-5354.patch, bash_HBASE-5354-v0.patch, > bash_HBASE-5354-v1.patch > > > Automating the testing of source code in a 'real' instance can be a bit of a > pain, even getting it into standalone mode. > Steps you need to go through: > 1) Build the project > 2) Copy it to the deployment directory > 3) Shutdown the current cluster (if it is running) > 4) Untar the tar > 5) Update the configs to point to a local data cluster > 6) Startup the new deployment > Yeah, its not super difficult, but it would be nice to just have a script to > make it button push easy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451105#comment-13451105 ] Elliott Clark commented on HBASE-6730: -- bq.If rLoads.size() > numRegionLoadsToRemember+1, we trim some element(s) off the tail of the list. Is that desirable ? We're trimming the first one off. When the list comes in it will have indices 0 - 14. We then say keep [1 - 15). (I thought subList is non-inclusive of the second index passed. I could be wrong here.) > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5937) Refactor HLog into an interface.
[ https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451103#comment-13451103 ] stack commented on HBASE-5937: -- Post log Flavio. > Refactor HLog into an interface. > > > Key: HBASE-5937 > URL: https://issues.apache.org/jira/browse/HBASE-5937 > Project: HBase > Issue Type: Sub-task >Reporter: Li Pi >Assignee: Flavio Junqueira >Priority: Minor > > What the summary says. Create HLog interface. Make current implementation use > it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6733) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2]
[ https://issues.apache.org/jira/browse/HBASE-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451102#comment-13451102 ] stack commented on HBASE-6733: -- You are finding bugs in replication DD. Good on you. > [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2] > --- > > Key: HBASE-6733 > URL: https://issues.apache.org/jira/browse/HBASE-6733 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das > Fix For: 0.92.3 > > > The failure is in TestReplication.queueFailover (fails due to unreplicated > rows). I have come across two problems: > 1. The sleepMultiplier is not properly reset when the currentPath is changed > (in ReplicationSource.java). > 2. ReplicationExecutor sometime removes files to replicate from the queue too > early, resulting in corresponding edits missing. Here the problem is due to > the fact the log-file length that the replication executor finds is not the > most updated one, and hence it doesn't read anything from there, and > ultimately, when there is a log roll, the replication-queue gets a new entry, > and the executor drops the old entry out of the queue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5843) Improve HBase MTTR - Mean Time To Recover
[ https://issues.apache.org/jira/browse/HBASE-5843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451100#comment-13451100 ] nkeywal commented on HBASE-5843: Improve HBase MTTR ? Unsexy? Really? More seriously: agreed. Will do. > Improve HBase MTTR - Mean Time To Recover > - > > Key: HBASE-5843 > URL: https://issues.apache.org/jira/browse/HBASE-5843 > Project: HBase > Issue Type: Umbrella >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > > A part of the approach is described here: > https://docs.google.com/document/d/1z03xRoZrIJmg7jsWuyKYl6zNournF_7ZHzdi0qz_B4c/edit > The ideal target is: > - failure impact client applications only by an added delay to execute a > query, whatever the failure. > - this delay is always inferior to 1 second. > We're not going to achieve that immediately... > Priority will be given to the most frequent issues. > Short term: > - software crash > - standard administrative tasks as stop/start of a cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5937) Refactor HLog into an interface.
[
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451098#comment-13451098
]
Flavio Junqueira commented on HBASE-5937:
-
I've been looking at TestMultiParallel because and I'm getting this exception:
{noformat}
2012-09-07 19:38:31,560 WARN [Thread-371]
client.HConnectionManager$HConnectionImplementation(1020): Encountered problems
when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: multi_test_table, row=multi_test_table,\x00,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:166)
at
org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:56)
at
org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:135)
at
org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:132)
at
org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:394)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:132)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:107)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:1017)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1071)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:959)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:916)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$Process.submit(HConnectionManager.java:1917)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$Process.doRetry(HConnectionManager.java:1957)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$Process.processBatchCallback(HConnectionManager.java:2064)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$Process.access$900(HConnectionManager.java:1859)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1848)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1827)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1003)
at
org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:246)
at
org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:213)
{noformat}
I'm not sure why the table is not present, and by skimming through the logs I
couldn't find any hint. If anyone has a hint, I'd be happy to hear, otherwise
I'll keep looking. I can also post the log if anyone is interested.
> Refactor HLog into an interface.
>
>
> Key: HBASE-5937
> URL: https://issues.apache.org/jira/browse/HBASE-5937
> Project: HBase
> Issue Type: Sub-task
>Reporter: Li Pi
>Assignee: Flavio Junqueira
>Priority: Minor
>
> What the summary says. Create HLog interface. Make current implementation use
> it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5843) Improve HBase MTTR - Mean Time To Recover
[ https://issues.apache.org/jira/browse/HBASE-5843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451096#comment-13451096 ] stack commented on HBASE-5843: -- Nicolas, the above should be put on the dev list. Its great stuff. Too good to be buried out here as a comment on issue with such an unsexy subject. > Improve HBase MTTR - Mean Time To Recover > - > > Key: HBASE-5843 > URL: https://issues.apache.org/jira/browse/HBASE-5843 > Project: HBase > Issue Type: Umbrella >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > > A part of the approach is described here: > https://docs.google.com/document/d/1z03xRoZrIJmg7jsWuyKYl6zNournF_7ZHzdi0qz_B4c/edit > The ideal target is: > - failure impact client applications only by an added delay to execute a > query, whatever the failure. > - this delay is always inferior to 1 second. > We're not going to achieve that immediately... > Priority will be given to the most frequent issues. > Short term: > - software crash > - standard administrative tasks as stop/start of a cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451095#comment-13451095 ] Elliott Clark commented on HBASE-6730: -- in HMaster a new instance of Chore was created with the function being defined. There was never a class, which I wanted to fix. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451093#comment-13451093
]
Ted Yu commented on HBASE-6730:
---
{code}
+ if (rLoads.size() >= numRegionLoadsToRemember) {
+rLoads = rLoads.subList(1, numRegionLoadsToRemember);
+ }
{code}
If rLoads.size() > numRegionLoadsToRemember+1, we trim some element(s) off the
tail of the list. Is that desirable ?
> Enable rolling averages in StochasticLoadBalancer
> --
>
> Key: HBASE-6730
> URL: https://issues.apache.org/jira/browse/HBASE-6730
> Project: HBase
> Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.96.0
>
> Attachments: HBASE-6730-1.patch
>
>
> Now that all of the ServerLoad transitions into pb are done. the load
> balancer should get more updates about the current state of the cluster.
> This should be used to allow StochasticLoadBalancer to get rolling averages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451087#comment-13451087 ] stack commented on HBASE-6730: -- bq. Currently the chores need to be interrupted when the master is being brought down and the balancer never gets a notification that things are shutting down. Is that an oversight? Should balance get a close message? On point #2, ok. We might revisit when we get balancer #3 implementation. Why do I not BalancerChore class added but not removed elsewhere (imagining it is an inner class somewhere that you moved out here standalone?) > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6746) Impacts of HBASE-6435 vs. HDFS 2.0 trunk
[ https://issues.apache.org/jira/browse/HBASE-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451075#comment-13451075 ] stack commented on HBASE-6746: -- +1 on commit > Impacts of HBASE-6435 vs. HDFS 2.0 trunk > > > Key: HBASE-6746 > URL: https://issues.apache.org/jira/browse/HBASE-6746 > Project: HBase > Issue Type: Bug > Components: master, regionserver, test >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 6746.v1.patch > > > When using the trunk of HDFS branch 2, I had two errors linked to HBASE-6435: > - a missing test to null > - a method removed. > This fixes it: > - add the test > - make the test case less dependant on HDFS internal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6740) Build out a toolset to make regular user operations on HBase tables
[ https://issues.apache.org/jira/browse/HBASE-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451073#comment-13451073 ] stack commented on HBASE-6740: -- We could on a period review the hbase-tools repo to roll stuff back into core (this is opposite of what I said above but hey, brainstorming). > Build out a toolset to make regular user operations on HBase tables > --- > > Key: HBASE-6740 > URL: https://issues.apache.org/jira/browse/HBASE-6740 > Project: HBase > Issue Type: Improvement >Reporter: Amandeep Khurana > > We currently have some stock MR jobs like import/export, copytable, importtsv > etc and tools like hbck, hfile viewer, WAL viewer etc that are a part of > HBase core. These are sometimes used just as is and often times need some > sort of customization to make them work in user environments. I propose we > build out a more extensive and extensible tool set and move it out to a > separate package outside core so we can add to those and release those as > they evolve. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451071#comment-13451071 ] Elliott Clark commented on HBASE-6730: -- bq.May I ask how the default of 15 was obtained It seemed like a good balance between remembering too much (keeping a lot of objects resident) and remembering only the last one which wasn't useful. Unix uses 15 mins in it's load metrics so it seemed like a good number to follow. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6746) Impacts of HBASE-6435 vs. HDFS 2.0 trunk
[ https://issues.apache.org/jira/browse/HBASE-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451069#comment-13451069 ] nkeywal commented on HBASE-6746: Order is random, so by running it multiple times it ensures it's not ok by accident... > Impacts of HBASE-6435 vs. HDFS 2.0 trunk > > > Key: HBASE-6746 > URL: https://issues.apache.org/jira/browse/HBASE-6746 > Project: HBase > Issue Type: Bug > Components: master, regionserver, test >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 6746.v1.patch > > > When using the trunk of HDFS branch 2, I had two errors linked to HBASE-6435: > - a missing test to null > - a method removed. > This fixes it: > - add the test > - make the test case less dependant on HDFS internal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6740) Build out a toolset to make regular user operations on HBase tables
[ https://issues.apache.org/jira/browse/HBASE-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451070#comment-13451070 ] stack commented on HBASE-6740: -- Hmm... what would having tools in a module do for us? Wondering more what this means "...and often times need some sort of customization to make them work in user environments"? This involve changes to the core or are you finding yourself copy/pasting to add some needed facility that you'd like to commit back and have available at a rate that is higher than that at which we release? I suppose this facility would be for toolsmiths to share their improvements. Do we need to move current tools to the toolsmiths' repo? Or can they stay where they are and toolsmiths use hbase-tools repo for new stuff or amendments to core tools? > Build out a toolset to make regular user operations on HBase tables > --- > > Key: HBASE-6740 > URL: https://issues.apache.org/jira/browse/HBASE-6740 > Project: HBase > Issue Type: Improvement >Reporter: Amandeep Khurana > > We currently have some stock MR jobs like import/export, copytable, importtsv > etc and tools like hbck, hfile viewer, WAL viewer etc that are a part of > HBase core. These are sometimes used just as is and often times need some > sort of customization to make them work in user environments. I propose we > build out a more extensive and extensible tool set and move it out to a > separate package outside core so we can add to those and release those as > they evolve. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451067#comment-13451067 ] Elliott Clark commented on HBASE-6730: -- For a couple of reasons. * Currently the chores need to be interrupted when the master is being brought down and the balancer never gets a notification that things are shutting down. * The balancer only has an instance of MasterServices which doesn't give access to getClusterStatus and I didn't want to change that interface. However I agree that having a bunch of chores hanging off of the master just seems wrong, way too many threads and way too much in the master. I can look at moving our chore's into something a little easier to remove from the master if you want, however that would be a little more code to change. > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451063#comment-13451063
]
Ted Yu commented on HBASE-6730:
---
{code}
+ private int numRegionLoadsToRemember = 15;
{code}
May I ask how the default of 15 was obtained ?
{code}
+ master.getConfiguration().getInt("hbase.balancer.statusPeriod",
6),
{code}
So we keep maximum of 15 minutes worth of status, by default ?
> Enable rolling averages in StochasticLoadBalancer
> --
>
> Key: HBASE-6730
> URL: https://issues.apache.org/jira/browse/HBASE-6730
> Project: HBase
> Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.96.0
>
> Attachments: HBASE-6730-1.patch
>
>
> Now that all of the ServerLoad transitions into pb are done. the load
> balancer should get more updates about the current state of the cluster.
> This should be used to allow StochasticLoadBalancer to get rolling averages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6412) Move external servers to metrics2 (thrift,thrift2,rest)
[ https://issues.apache.org/jira/browse/HBASE-6412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451061#comment-13451061 ] stack commented on HBASE-6412: -- Did another overall rough review. This is gruesome application of a pattern making metrics work over hadoop 1 and 2. It looks good to me. I'm up for commit. Wondering Alex if you had anything to say before I do? Otherwise, will commit in next day or so. > Move external servers to metrics2 (thrift,thrift2,rest) > --- > > Key: HBASE-6412 > URL: https://issues.apache.org/jira/browse/HBASE-6412 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.96.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Blocker > Attachments: HBASE-6412-0.patch, HBASE-6412-1.patch, > HBASE-6412-2.patch, HBASE-6412-3.patch, HBASE-6412-4.patch, HBASE-6412-5.patch > > > Implement metrics2 for all the external servers: > * Thrift > * Thrift2 > * Rest -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6740) Build out a toolset to make regular user operations on HBase tables
[ https://issues.apache.org/jira/browse/HBASE-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451054#comment-13451054 ] Amandeep Khurana commented on HBASE-6740: - Yup, separate maven module works too. > Build out a toolset to make regular user operations on HBase tables > --- > > Key: HBASE-6740 > URL: https://issues.apache.org/jira/browse/HBASE-6740 > Project: HBase > Issue Type: Improvement >Reporter: Amandeep Khurana > > We currently have some stock MR jobs like import/export, copytable, importtsv > etc and tools like hbck, hfile viewer, WAL viewer etc that are a part of > HBase core. These are sometimes used just as is and often times need some > sort of customization to make them work in user environments. I propose we > build out a more extensive and extensible tool set and move it out to a > separate package outside core so we can add to those and release those as > they evolve. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6742) Change default test parallelisation level to 5
[ https://issues.apache.org/jira/browse/HBASE-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451041#comment-13451041 ] stack commented on HBASE-6742: -- +1 on patch > Change default test parallelisation level to 5 > -- > > Key: HBASE-6742 > URL: https://issues.apache.org/jira/browse/HBASE-6742 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: hbase-6742.v1.patch > > > Tests will be faster. > Not visible if a test hangs for 15 minutes. But they should not hang. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451038#comment-13451038 ] stack commented on HBASE-6730: -- Why clusterstatuschore in HMaster and not in the balancer? Or is it used by more than balancer? Can we not change balancer Interface to add feeding it cluster status? Or have those interested in cluster status register a listener (is that going overboard?) > Enable rolling averages in StochasticLoadBalancer > -- > > Key: HBASE-6730 > URL: https://issues.apache.org/jira/browse/HBASE-6730 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6730-1.patch > > > Now that all of the ServerLoad transitions into pb are done. the load > balancer should get more updates about the current state of the cluster. > This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5354) Source to standalone deployment script
[ https://issues.apache.org/jira/browse/HBASE-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-5354: --- Attachment: bash_HBASE-5354-v1.patch Updated version - works on my Mac (OSX) and Chris Trezzo's linux box. > Source to standalone deployment script > -- > > Key: HBASE-5354 > URL: https://issues.apache.org/jira/browse/HBASE-5354 > Project: HBase > Issue Type: New Feature > Components: build, scripts >Affects Versions: 0.94.0 >Reporter: Jesse Yates >Assignee: Jesse Yates >Priority: Minor > Attachments: bash_HBASE-5354.patch, bash_HBASE-5354-v0.patch, > bash_HBASE-5354-v1.patch > > > Automating the testing of source code in a 'real' instance can be a bit of a > pain, even getting it into standalone mode. > Steps you need to go through: > 1) Build the project > 2) Copy it to the deployment directory > 3) Shutdown the current cluster (if it is running) > 4) Untar the tar > 5) Update the configs to point to a local data cluster > 6) Startup the new deployment > Yeah, its not super difficult, but it would be nice to just have a script to > make it button push easy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6412) Move external servers to metrics2 (thrift,thrift2,rest)
[ https://issues.apache.org/jira/browse/HBASE-6412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6412: - Attachment: HBASE-6412-5.patch > Move external servers to metrics2 (thrift,thrift2,rest) > --- > > Key: HBASE-6412 > URL: https://issues.apache.org/jira/browse/HBASE-6412 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.96.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Blocker > Attachments: HBASE-6412-0.patch, HBASE-6412-1.patch, > HBASE-6412-2.patch, HBASE-6412-3.patch, HBASE-6412-4.patch, HBASE-6412-5.patch > > > Implement metrics2 for all the external servers: > * Thrift > * Thrift2 > * Rest -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps
[ https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451036#comment-13451036 ] Jesse Yates commented on HBASE-6707: Also ran it 10x locally (and another 10x before rebasing onto trunk) without issue, though that clearly doesn't mean all that much. > TEST > org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables > flaps > > > Key: HBASE-6707 > URL: https://issues.apache.org/jira/browse/HBASE-6707 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sameer Vaishampayan >Assignee: Jesse Yates >Priority: Critical > Fix For: 0.96.0 > > Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch > > > https://builds.apache.org/job/HBase-TRUNK/3293/ > Error Message > Archived HFiles > (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam) > should have gotten deleted, but didn't, remaining > files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c] > Stacktrace > java.lang.AssertionError: Archived HFiles > (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam) > should have gotten deleted, but didn't, remaining > files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c] > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.assertTrue(Assert.java:43) > at org.junit.Assert.assertNull(Assert.java:551) > at > org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps
[ https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-6707: --- Status: Patch Available (was: Open) > TEST > org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables > flaps > > > Key: HBASE-6707 > URL: https://issues.apache.org/jira/browse/HBASE-6707 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sameer Vaishampayan >Assignee: Jesse Yates >Priority: Critical > Fix For: 0.96.0 > > Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch > > > https://builds.apache.org/job/HBase-TRUNK/3293/ > Error Message > Archived HFiles > (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam) > should have gotten deleted, but didn't, remaining > files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c] > Stacktrace > java.lang.AssertionError: Archived HFiles > (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam) > should have gotten deleted, but didn't, remaining > files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c] > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.assertTrue(Assert.java:43) > at org.junit.Assert.assertNull(Assert.java:551) > at > org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps
[ https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-6707: --- Attachment: hbase-6707-v1.patch New version - this time we ignore it if we can't compact the files and just make sure there are some files that have been archived. I was trying both in the original and the previously attached patch to ensure that we had enough files before trying the compaction, but its just easier to try-and-fail than be defensive about it (though slightly more brittle, its unlikely to change anytime soon...and much simpler to reason about code-wise). > TEST > org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables > flaps > > > Key: HBASE-6707 > URL: https://issues.apache.org/jira/browse/HBASE-6707 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sameer Vaishampayan >Assignee: Jesse Yates >Priority: Critical > Fix For: 0.96.0 > > Attachments: hbase-6707-v0.patch, hbase-6707-v1.patch > > > https://builds.apache.org/job/HBase-TRUNK/3293/ > Error Message > Archived HFiles > (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam) > should have gotten deleted, but didn't, remaining > files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c] > Stacktrace > java.lang.AssertionError: Archived HFiles > (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam) > should have gotten deleted, but didn't, remaining > files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c] > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.assertTrue(Assert.java:43) > at org.junit.Assert.assertNull(Assert.java:551) > at > org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6707) TEST org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables flaps
[ https://issues.apache.org/jira/browse/HBASE-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-6707: --- Status: Open (was: Patch Available) > TEST > org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables > flaps > > > Key: HBASE-6707 > URL: https://issues.apache.org/jira/browse/HBASE-6707 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sameer Vaishampayan >Assignee: Jesse Yates >Priority: Critical > Fix For: 0.96.0 > > Attachments: hbase-6707-v0.patch > > > https://builds.apache.org/job/HBase-TRUNK/3293/ > Error Message > Archived HFiles > (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam) > should have gotten deleted, but didn't, remaining > files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c] > Stacktrace > java.lang.AssertionError: Archived HFiles > (hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam) > should have gotten deleted, but didn't, remaining > files:[hdfs://localhost:59986/user/jenkins/hbase/.archive/otherTable/01ced3b55d7220a9c460273a4a57b198/fam/fc872572a1f5443eb55b6e2567cfeb1c] > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.assertTrue(Assert.java:43) > at org.junit.Assert.assertNull(Assert.java:551) > at > org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient.testMultipleTables(TestZooKeeperTableArchiveClient.java:291) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6743) Move EnvironmentEdge classes to hbase-common
[ https://issues.apache.org/jira/browse/HBASE-6743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451027#comment-13451027 ] Chris Trezzo commented on HBASE-6743: - Yes. Woops. Moving files with svn can be a pain. Will fix. Thanks! > Move EnvironmentEdge classes to hbase-common > > > Key: HBASE-6743 > URL: https://issues.apache.org/jira/browse/HBASE-6743 > Project: HBase > Issue Type: Improvement >Reporter: Chris Trezzo >Assignee: Chris Trezzo >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6743v1.txt > > > Move the classes related to EnvironmentEdge to the hbase-common module. If we > want to replace all occurrences of System.currentTimeMillis() with > EnvironmentEdgeManager.currentTimeMillis(), we need to be able to reference > these classes from multiple modules. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6744) Per table balancing could cause regions unbalanced overall
[ https://issues.apache.org/jira/browse/HBASE-6744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451019#comment-13451019 ] Elliott Clark commented on HBASE-6744: -- Once HBASE-6730 goes in, the SLB will imo be ready for prime time. > Per table balancing could cause regions unbalanced overall > -- > > Key: HBASE-6744 > URL: https://issues.apache.org/jira/browse/HBASE-6744 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang > > Per table balancing just balances regions based on tables. However, overall, > regions could be seriously unbalanced. > For example, if you shutdown all most all region serves in a cluster, then > create tons of new tables (no region pre-split), then start up all region > servers. You will see the regions won't move to other region servers since > they are balanced per table (only one region for a table at this moment). > If we can make the balance algorithm sophisticated enough, we don't need the > configuration hbase.master.loadbalance.bytable. We can do the regular and > bytable balancing at the same time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6744) Per table balancing could cause regions unbalanced overall
[ https://issues.apache.org/jira/browse/HBASE-6744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451016#comment-13451016 ] stack commented on HBASE-6744: -- +1 on making SLB default now. Testing for 0.96.0 should shake out any bugs. On its face alone, it has a potential far in excess of the current, hard to untangle, served-us well up-to-this, but needs-to-be retired now current balancer. > Per table balancing could cause regions unbalanced overall > -- > > Key: HBASE-6744 > URL: https://issues.apache.org/jira/browse/HBASE-6744 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang > > Per table balancing just balances regions based on tables. However, overall, > regions could be seriously unbalanced. > For example, if you shutdown all most all region serves in a cluster, then > create tons of new tables (no region pre-split), then start up all region > servers. You will see the regions won't move to other region servers since > they are balanced per table (only one region for a table at this moment). > If we can make the balance algorithm sophisticated enough, we don't need the > configuration hbase.master.loadbalance.bytable. We can do the regular and > bytable balancing at the same time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6710) 0.92/0.94 compatibility issues due to HBASE-5206
[
https://issues.apache.org/jira/browse/HBASE-6710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451013#comment-13451013
]
Ted Yu commented on HBASE-6710:
---
Here is the link to review request: https://reviews.apache.org/r/6934
> 0.92/0.94 compatibility issues due to HBASE-5206
>
>
> Key: HBASE-6710
> URL: https://issues.apache.org/jira/browse/HBASE-6710
> Project: HBase
> Issue Type: Bug
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Critical
> Fix For: 0.94.2
>
>
> HBASE-5206 introduces some compatibility issues between {0.94,0.94.1} and
> {0.92.0,0.92.1}. The release notes of HBASE-5155 describes the issue
> (HBASE-5206 is a backport of HBASE-5155).
> I think we can make 0.94.2 compatible with both {0.94.0,0.94.1} and
> {0.92.0,0.92.1}, although one of those sets will require configuration
> changes.
> The basic problem is that there is a znode for each table
> "zookeeper.znode.tableEnableDisable" that is handled differently.
> On 0.92.0 and 0.92.1 the states for this table are:
> [ disabled, disabling, enabling ] or deleted if the table is enabled
> On 0.94.1 and 0.94.2 the states for this table are:
> [ disabled, disabling, enabling, enabled ]
> What saves us is that the location of this znode is configurable. So the
> basic idea is to have the 0.94.2 master write two different znodes,
> "zookeeper.znode.tableEnableDisabled92" and
> "zookeeper.znode.tableEnableDisabled94" where the 92 node is in 92 format,
> the 94 node is in 94 format. And internally, the master would only use the
> 94 format in order to solve the original bug HBASE-5155 solves.
> We can of course make one of these the same default as exists now, so we
> don't need to make config changes for one of 0.92 or 0.94 clients. I argue
> that 0.92 clients shouldn't have to make config changes for the same reason I
> argued above. But that is debatable.
> Then, I think the only question left is the question of how to bring along
> the {0.94.0, 0.94.1} crew. A {0.94.0, 0.94.1} client would work against a
> 0.94.2 cluster by just configuring "zookeeper.znode.tableEnableDisable" in
> the client to be whatever "zookeeper.znode.tableEnableDisabled94" is in the
> cluster. A 0.94.2 client would work against both a {0.94.0, 0.94.1} and
> {0.92.0, 0.92.1} cluster if it had HBASE-6268 applied. About rolling upgrade
> from {0.94.0, 0.94.1} to 0.94.2 -- I'd have to think about that. Do the
> regionservers ever read the tableEnableDisabled znode?
> On the mailing list, Lars H suggested the following:
> "The only input I'd have is that format we'll use going forward will not have
> a version attached to it.
> So maybe the 92 version would still be called
> "zookeeper.znode.tableEnableDisable" and the new node could have a different
> name "zookeeper.znode.tableEnableDisableNew" (or something)."
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6412) Move external servers to metrics2 (thrift,thrift2,rest)
[ https://issues.apache.org/jira/browse/HBASE-6412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6412: - Attachment: HBASE-6412-4.patch Renamed all *Test classes to Test*. Changed TestCheckTestClasses to ignore hbase-hadoop-compat jars. > Move external servers to metrics2 (thrift,thrift2,rest) > --- > > Key: HBASE-6412 > URL: https://issues.apache.org/jira/browse/HBASE-6412 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.96.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Blocker > Attachments: HBASE-6412-0.patch, HBASE-6412-1.patch, > HBASE-6412-2.patch, HBASE-6412-3.patch, HBASE-6412-4.patch > > > Implement metrics2 for all the external servers: > * Thrift > * Thrift2 > * Rest -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6743) Move EnvironmentEdge classes to hbase-common
[ https://issues.apache.org/jira/browse/HBASE-6743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451006#comment-13451006 ] stack commented on HBASE-6743: -- Did you forget to svn add the classes back in hbase-common Chris? They are not in the patch? > Move EnvironmentEdge classes to hbase-common > > > Key: HBASE-6743 > URL: https://issues.apache.org/jira/browse/HBASE-6743 > Project: HBase > Issue Type: Improvement >Reporter: Chris Trezzo >Assignee: Chris Trezzo >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6743v1.txt > > > Move the classes related to EnvironmentEdge to the hbase-common module. If we > want to replace all occurrences of System.currentTimeMillis() with > EnvironmentEdgeManager.currentTimeMillis(), we need to be able to reference > these classes from multiple modules. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6746) Impacts of HBASE-6435 vs. HDFS 2.0 trunk
[
https://issues.apache.org/jira/browse/HBASE-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451000#comment-13451000
]
Ted Yu commented on HBASE-6746:
---
{code}
+for (int i=0; i<10; i++){
{code}
Why running the same test 10 times ?
> Impacts of HBASE-6435 vs. HDFS 2.0 trunk
>
>
> Key: HBASE-6746
> URL: https://issues.apache.org/jira/browse/HBASE-6746
> Project: HBase
> Issue Type: Bug
> Components: master, regionserver, test
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Assignee: nkeywal
> Fix For: 0.96.0
>
> Attachments: 6746.v1.patch
>
>
> When using the trunk of HDFS branch 2, I had two errors linked to HBASE-6435:
> - a missing test to null
> - a method removed.
> This fixes it:
> - add the test
> - make the test case less dependant on HDFS internal.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6740) Build out a toolset to make regular user operations on HBase tables
[ https://issues.apache.org/jira/browse/HBASE-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450995#comment-13450995 ] Andrew Purtell commented on HBASE-6740: --- Why not a separate Maven module? Why a separate repo? People aren't going to want tools shipped with HBase? > Build out a toolset to make regular user operations on HBase tables > --- > > Key: HBASE-6740 > URL: https://issues.apache.org/jira/browse/HBASE-6740 > Project: HBase > Issue Type: Improvement >Reporter: Amandeep Khurana > > We currently have some stock MR jobs like import/export, copytable, importtsv > etc and tools like hbck, hfile viewer, WAL viewer etc that are a part of > HBase core. These are sometimes used just as is and often times need some > sort of customization to make them work in user environments. I propose we > build out a more extensive and extensible tool set and move it out to a > separate package outside core so we can add to those and release those as > they evolve. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6746) Impacts of HBASE-6435 vs. HDFS 2.0 trunk
[ https://issues.apache.org/jira/browse/HBASE-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-6746: --- Fix Version/s: 0.96.0 Status: Patch Available (was: Open) > Impacts of HBASE-6435 vs. HDFS 2.0 trunk > > > Key: HBASE-6746 > URL: https://issues.apache.org/jira/browse/HBASE-6746 > Project: HBase > Issue Type: Bug > Components: master, regionserver, test >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 6746.v1.patch > > > When using the trunk of HDFS branch 2, I had two errors linked to HBASE-6435: > - a missing test to null > - a method removed. > This fixes it: > - add the test > - make the test case less dependant on HDFS internal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6746) Impacts of HBASE-6435 vs. HDFS 2.0 trunk
[ https://issues.apache.org/jira/browse/HBASE-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-6746: --- Attachment: 6746.v1.patch > Impacts of HBASE-6435 vs. HDFS 2.0 trunk > > > Key: HBASE-6746 > URL: https://issues.apache.org/jira/browse/HBASE-6746 > Project: HBase > Issue Type: Bug > Components: master, regionserver, test >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 6746.v1.patch > > > When using the trunk of HDFS branch 2, I had two errors linked to HBASE-6435: > - a missing test to null > - a method removed. > This fixes it: > - add the test > - make the test case less dependant on HDFS internal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6714) TestMultiSlaveReplication#testMultiSlaveReplication may fail
[ https://issues.apache.org/jira/browse/HBASE-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-6714: --- Description: java.lang.AssertionError: expected:<1> but was:<0> at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.junit.Assert.assertEquals(Assert.java:456) at org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.checkRow(TestMultiSlaveReplication.java:203) at org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.testMultiSlaveReplication(TestMultiSlaveReplication.java:188) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) TestMultiSlaveReplication->testMultiSlaveReplication failed in our local build citing that "row" was not replicated to second peer. This is because after inserting "row", log is rolled and we look for "row2" in both the clusters and then we check for existence of "row" in both clusters. Meanwhile, Replication thread was sleeping for the second cluster and Row "row2" is not present in the second cluster from the very beginning. So, the "row2" existence check succeeds and control move on to find "row" in both clusters where it fails for the second cluster. was:TestMultiSlaveReplication->testMultiSlaveReplication failed in our local build citing that "row" was not replicated to second peer. This is because after inserting "row", log is rolled and we look for "row2" in both the clusters and then we check for existence of "row" in both clusters. Meanwhile, Replication thread was sleeping for the second cluster and Row "row2" is not present in the second cluster from the very beginning. So, the "row2" existence check succeeds and control move on to find "row" in both clusters where it fails for the second cluster. > TestMultiSlaveReplication#testMultiSlaveReplication may fail > > > Key: HBASE-6714 > URL: https://issues.apache.org/jira/browse/HBASE-6714 > Project: HBase > Issue Type: Bug > Components: replication, test >Affects Versions: 0.92.0, 0.94.0 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > > java.lang.AssertionError: expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:472) > at org.junit.Assert.assertEquals(Assert.java:456) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.checkRow(TestMultiSlaveReplication.java:203) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.testMultiSlaveReplication(TestMultiSlaveReplication.java:188) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > TestMultiSlaveReplication->testMultiSlaveReplication failed in our local > build citing that "row" was not replicated to second peer. This is because > after inserting "row", log is rolled and we look for "row2" in both the > clusters and then we check for existence of "row" in both clusters. > Meanwhile, Replication thread was sleeping for the second cluster and Row > "row2" is not present in the second cluster from the very beginning. So, the > "row2" existence check succeeds and control move on to find "row" in both > clusters where it fails for the second cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5354) Source to standalone deployment script
[ https://issues.apache.org/jira/browse/HBASE-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450981#comment-13450981 ] Jesse Yates commented on HBASE-5354: And apparently its totally hosed on Chris Trezzo's machine... looking into it. > Source to standalone deployment script > -- > > Key: HBASE-5354 > URL: https://issues.apache.org/jira/browse/HBASE-5354 > Project: HBase > Issue Type: New Feature > Components: build, scripts >Affects Versions: 0.94.0 >Reporter: Jesse Yates >Assignee: Jesse Yates >Priority: Minor > Attachments: bash_HBASE-5354.patch, bash_HBASE-5354-v0.patch > > > Automating the testing of source code in a 'real' instance can be a bit of a > pain, even getting it into standalone mode. > Steps you need to go through: > 1) Build the project > 2) Copy it to the deployment directory > 3) Shutdown the current cluster (if it is running) > 4) Untar the tar > 5) Update the configs to point to a local data cluster > 6) Startup the new deployment > Yeah, its not super difficult, but it would be nice to just have a script to > make it button push easy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6744) Per table balancing could cause regions unbalanced overall
[ https://issues.apache.org/jira/browse/HBASE-6744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450976#comment-13450976 ] Ted Yu commented on HBASE-6744: --- bq. StochasticLoadBalancer is powerful I haven't seen concrete numbers for StochasticLoadBalancer. > Per table balancing could cause regions unbalanced overall > -- > > Key: HBASE-6744 > URL: https://issues.apache.org/jira/browse/HBASE-6744 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang > > Per table balancing just balances regions based on tables. However, overall, > regions could be seriously unbalanced. > For example, if you shutdown all most all region serves in a cluster, then > create tons of new tables (no region pre-split), then start up all region > servers. You will see the regions won't move to other region servers since > they are balanced per table (only one region for a table at this moment). > If we can make the balance algorithm sophisticated enough, we don't need the > configuration hbase.master.loadbalance.bytable. We can do the regular and > bytable balancing at the same time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5354) Source to standalone deployment script
[ https://issues.apache.org/jira/browse/HBASE-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450975#comment-13450975 ] Jesse Yates commented on HBASE-5354: Weird that it didn't work... Initially, this was a pretty small test case, though maybe it would make sense to add this check (running locally from the tarball) to the release checklist. If we add it, we should probably roll this script in to make it easier to deal with (also no real way to see usage if its just attached to the jira). > Source to standalone deployment script > -- > > Key: HBASE-5354 > URL: https://issues.apache.org/jira/browse/HBASE-5354 > Project: HBase > Issue Type: New Feature > Components: build, scripts >Affects Versions: 0.94.0 >Reporter: Jesse Yates >Assignee: Jesse Yates >Priority: Minor > Attachments: bash_HBASE-5354.patch, bash_HBASE-5354-v0.patch > > > Automating the testing of source code in a 'real' instance can be a bit of a > pain, even getting it into standalone mode. > Steps you need to go through: > 1) Build the project > 2) Copy it to the deployment directory > 3) Shutdown the current cluster (if it is running) > 4) Untar the tar > 5) Update the configs to point to a local data cluster > 6) Startup the new deployment > Yeah, its not super difficult, but it would be nice to just have a script to > make it button push easy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6744) Per table balancing could cause regions unbalanced overall
[ https://issues.apache.org/jira/browse/HBASE-6744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450971#comment-13450971 ] Jimmy Xiang commented on HBASE-6744: StochasticLoadBalancer is powerful. It looks very good to me. Why don't we deprecate the default one and make this one the new default? > Per table balancing could cause regions unbalanced overall > -- > > Key: HBASE-6744 > URL: https://issues.apache.org/jira/browse/HBASE-6744 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang > > Per table balancing just balances regions based on tables. However, overall, > regions could be seriously unbalanced. > For example, if you shutdown all most all region serves in a cluster, then > create tons of new tables (no region pre-split), then start up all region > servers. You will see the regions won't move to other region servers since > they are balanced per table (only one region for a table at this moment). > If we can make the balance algorithm sophisticated enough, we don't need the > configuration hbase.master.loadbalance.bytable. We can do the regular and > bytable balancing at the same time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5354) Source to standalone deployment script
[
https://issues.apache.org/jira/browse/HBASE-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450969#comment-13450969
]
stack commented on HBASE-5354:
--
I did this:
{code}
h-24-30:trunk stack$ pwd
/Users/Stack/checkouts/trunk
h-24-30:trunk stack$ ./dev-support/deploy.sh
{code}
Maybe we should just leave the script here attached to the issue and if folks
start looking for it, we'll point them here. If uptake, commit later?
> Source to standalone deployment script
> --
>
> Key: HBASE-5354
> URL: https://issues.apache.org/jira/browse/HBASE-5354
> Project: HBase
> Issue Type: New Feature
> Components: build, scripts
>Affects Versions: 0.94.0
>Reporter: Jesse Yates
>Assignee: Jesse Yates
>Priority: Minor
> Attachments: bash_HBASE-5354.patch, bash_HBASE-5354-v0.patch
>
>
> Automating the testing of source code in a 'real' instance can be a bit of a
> pain, even getting it into standalone mode.
> Steps you need to go through:
> 1) Build the project
> 2) Copy it to the deployment directory
> 3) Shutdown the current cluster (if it is running)
> 4) Untar the tar
> 5) Update the configs to point to a local data cluster
> 6) Startup the new deployment
> Yeah, its not super difficult, but it would be nice to just have a script to
> make it button push easy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5354) Source to standalone deployment script
[ https://issues.apache.org/jira/browse/HBASE-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450966#comment-13450966 ] Jesse Yates commented on HBASE-5354: how did you try it? I've found it works when running from /hbase, maybe this is a start directory issue (yeah, its a pretty simple script...)? > Source to standalone deployment script > -- > > Key: HBASE-5354 > URL: https://issues.apache.org/jira/browse/HBASE-5354 > Project: HBase > Issue Type: New Feature > Components: build, scripts >Affects Versions: 0.94.0 >Reporter: Jesse Yates >Assignee: Jesse Yates >Priority: Minor > Attachments: bash_HBASE-5354.patch, bash_HBASE-5354-v0.patch > > > Automating the testing of source code in a 'real' instance can be a bit of a > pain, even getting it into standalone mode. > Steps you need to go through: > 1) Build the project > 2) Copy it to the deployment directory > 3) Shutdown the current cluster (if it is running) > 4) Untar the tar > 5) Update the configs to point to a local data cluster > 6) Startup the new deployment > Yeah, its not super difficult, but it would be nice to just have a script to > make it button push easy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5354) Source to standalone deployment script
[
https://issues.apache.org/jira/browse/HBASE-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450959#comment-13450959
]
stack commented on HBASE-5354:
--
I tried it:
{code}
[INFO]
[ERROR] FATAL ERROR
[INFO]
[INFO] Could not find resource
'/Users/Stack/checkouts/trunk/hbase-common/dev-support/findbugs-exclude.xml'.
[INFO]
[INFO] Trace
org.codehaus.plexus.resource.loader.ResourceNotFoundException: Could not find
resource
'/Users/Stack/checkouts/trunk/hbase-common/dev-support/findbugs-exclude.xml'.
{code}
> Source to standalone deployment script
> --
>
> Key: HBASE-5354
> URL: https://issues.apache.org/jira/browse/HBASE-5354
> Project: HBase
> Issue Type: New Feature
> Components: build, scripts
>Affects Versions: 0.94.0
>Reporter: Jesse Yates
>Assignee: Jesse Yates
>Priority: Minor
> Attachments: bash_HBASE-5354.patch, bash_HBASE-5354-v0.patch
>
>
> Automating the testing of source code in a 'real' instance can be a bit of a
> pain, even getting it into standalone mode.
> Steps you need to go through:
> 1) Build the project
> 2) Copy it to the deployment directory
> 3) Shutdown the current cluster (if it is running)
> 4) Untar the tar
> 5) Update the configs to point to a local data cluster
> 6) Startup the new deployment
> Yeah, its not super difficult, but it would be nice to just have a script to
> make it button push easy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6746) Impacts of HBASE-6435 vs. HDFS 2.0 trunk
nkeywal created HBASE-6746: -- Summary: Impacts of HBASE-6435 vs. HDFS 2.0 trunk Key: HBASE-6746 URL: https://issues.apache.org/jira/browse/HBASE-6746 Project: HBase Issue Type: Bug Components: master, regionserver, test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal When using the trunk of HDFS branch 2, I had two errors linked to HBASE-6435: - a missing test to null - a method removed. This fixes it: - add the test - make the test case less dependant on HDFS internal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6745) Always flush region based on memstore size could hold hlog files from archiving
Jimmy Xiang created HBASE-6745: -- Summary: Always flush region based on memstore size could hold hlog files from archiving Key: HBASE-6745 URL: https://issues.apache.org/jira/browse/HBASE-6745 Project: HBase Issue Type: Bug Components: replication, wal Reporter: Jimmy Xiang Currently, memstore flusher always chooses the biggest memstore region to flush. Suppose I have two tables: one is very actively updated, while the other is periodically updated. The active one has biggest memstore all the time and is flushed all the time. But the in-active one never gets a chance to flush. Since it is not flushed, the hlog file can't be archived, although there are lots of hlog files. If the active table happens to have big updates all the time, the hlog files could cause huge disk space pressure. Other than the memstore size, periodically flushing regions based on hlog roll time is helpful in hlog archiving/replication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6744) Per table balancing could cause regions unbalanced overall
[ https://issues.apache.org/jira/browse/HBASE-6744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450952#comment-13450952 ] Elliott Clark commented on HBASE-6744: -- The StochasticLoadBalancer that was added a little while ago should give you the best of both worlds. It tries to spread all tables as much as posible without making things too unbalanced. Does that work for you? or were you thinking of something different ? > Per table balancing could cause regions unbalanced overall > -- > > Key: HBASE-6744 > URL: https://issues.apache.org/jira/browse/HBASE-6744 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang > > Per table balancing just balances regions based on tables. However, overall, > regions could be seriously unbalanced. > For example, if you shutdown all most all region serves in a cluster, then > create tons of new tables (no region pre-split), then start up all region > servers. You will see the regions won't move to other region servers since > they are balanced per table (only one region for a table at this moment). > If we can make the balance algorithm sophisticated enough, we don't need the > configuration hbase.master.loadbalance.bytable. We can do the regular and > bytable balancing at the same time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
