[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487589#comment-13487589 ] Enis Soztutar commented on HBASE-7055: -- I don't see any reason to introduce another configuration file, since it has the same property name-value format. > port HBASE-6371 tier-based compaction from 0.89-fb to trunk > --- > > Key: HBASE-7055 > URL: https://issues.apache.org/jira/browse/HBASE-7055 > Project: HBase > Issue Type: Task > Components: Compaction >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.96.0 > > Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch > > > There's divergence in the code :( > See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5500) [Coprocessors] Add hooks for bulk loading actions
[
https://issues.apache.org/jira/browse/HBASE-5500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487587#comment-13487587
]
Anoop Sam John commented on HBASE-5500:
---
This is already done as part of HBASE-6224 right?
> [Coprocessors] Add hooks for bulk loading actions
> -
>
> Key: HBASE-5500
> URL: https://issues.apache.org/jira/browse/HBASE-5500
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Andrew Purtell
>
> The API gap for bulk HFile loading was discussed on the mailing list but it
> didn't make it into a JIRA. It also came up on HBASE-5498.
> See http://search-hadoop.com/m/eEUHK1s4fo81/bulk+loading+and+RegionObservers
> The salient detail:
> {quote}
> A simple and straightforward course of action is to give the CP the
> option of rewriting the submitted store file(s) before the regionserver
> attempts to validate and move them into the store. This is similar to how CPs
> are hooked into compaction: CPs hook compaction by allowing one to wrap the
> scanner that is iterating over the store files. So the wrapper gets a chance
> to examine the KeyValues being processed and also has an opportunity to
> modify or drop them.
> Similarly for incoming HFiles for bulk load, the CP could be given a
> scanner iterating over those files, if you had a RegionObserver installed.
> You would be given the option in effect to rewrite the incoming HFiles before
> they are handed over to the RegionServer for addition to the region.
> {quote}
> I think this is a reasonable approach to interface design, because the fact
> you are given a scanner highlights the bulk nature of the input. However I
> think there should be two hooks here: one that allows for a simple yes/no
> answer as to whether the bulk load should proceed; and one that allows for a
> more expensive filtering or transformation or whatever via scanner-like
> interface. Bulk loads could be potentially very large so requiring a scan
> over them always is not a good idea.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7051) Read/Updates (increment,checkAndPut) should properly read MVCC
[
https://issues.apache.org/jira/browse/HBASE-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487581#comment-13487581
]
Hadoop QA commented on HBASE-7051:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551489/7051.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3197//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3197//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3197//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3197//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3197//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3197//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3197//console
This message is automatically generated.
> Read/Updates (increment,checkAndPut) should properly read MVCC
> --
>
> Key: HBASE-7051
> URL: https://issues.apache.org/jira/browse/HBASE-7051
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Gregory Chanan
>Assignee: Lars Hofhansl
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 7051.txt
>
>
> See, for example:
> {code}
> // TODO: Use MVCC to make this set of increments atomic to reads
> {code}
> Here's an example of what I can happen (would probably be good to write up a
> test case for each read/update):
> Concurrent update via increment and put.
> The put grabs the row lock first and updates the memstore, but releases the
> row lock before the MVCC is advanced. Then, the increment grabs the row lock
> and reads right away, reading the old value and incrementing based on that.
> There are a few options here:
> 1) Waiting for the MVCC to advance for read/updates: the downside is that you
> have to wait for updates on other rows.
> 2) Have an MVCC per-row (table configuration): this avoids the unnecessary
> contention of 1)
> 3) Transform the read/updates to write-only with rollup on read.. E.g. an
> increment would just have the number of values to increment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations
[
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487579#comment-13487579
]
Hadoop QA commented on HBASE-4583:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551483/4583-trunk-less-radical-v6.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3196//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3196//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3196//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3196//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3196//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3196//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3196//console
This message is automatically generated.
> Integrate RWCC with Append and Increment operations
> ---
>
> Key: HBASE-4583
> URL: https://issues.apache.org/jira/browse/HBASE-4583
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 4583-trunk-less-radical.txt,
> 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt,
> 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt,
> 4583-trunk-less-radical-v6.txt, 4583-trunk-radical.txt,
> 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt,
> 4583-v3.txt, 4583-v4.txt
>
>
> Currently Increment and Append operations do not work with RWCC and hence a
> client could see the results of multiple such operation mixed in the same
> Get/Scan.
> The semantics might be a bit more interesting here as upsert adds and removes
> to and from the memstore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7070) Scanner may retry forever after HBASE-5974
[ https://issues.apache.org/jira/browse/HBASE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487578#comment-13487578 ] Anoop Sam John commented on HBASE-7070: --- +1 for flag. Looks easy to understand then bq. We will go around and around if we are getting socket timeouts? If we get socket timeouts all the time, there will be 10 retries (defualt) At server side some of the calls will get OutOfOrderScannerNextException. > Scanner may retry forever after HBASE-5974 > --- > > Key: HBASE-7070 > URL: https://issues.apache.org/jira/browse/HBASE-7070 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-7070.patch, HBASE-7070v2.patch, > HBASE-7070v3.patch, HBASE-7070v4.patch, HBASE-7070v5.patch > > > After the patch of HBASE-5974 > Suppose The process is as the following: > 1.A next request is very large, so first time it is failed because of timeout > 2.The second time it is failed again because of > CallSequenceOutOfOrderException > 3.We will retry this next request again after reset scanner, so it is also > very large and failed again because of timeout > 4.CallSequenceOutOfOrderException again > 5.Repeated the above forever... > See the comment in HBASE-5974 for more > https://issues.apache.org/jira/browse/HBASE-5974?focusedCommentId=13486658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13486658 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7079) testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails
[ https://issues.apache.org/jira/browse/HBASE-7079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7079: - Status: Patch Available (was: Open) Patch looks good to me (Has formatting issues but i can fix on commit). Sending to hadoopqa to see what it thinks. > testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails > -- > > Key: HBASE-7079 > URL: https://issues.apache.org/jira/browse/HBASE-7079 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.94.2 >Reporter: Zhou wenjian >Assignee: Zhou wenjian > Attachments: HBASE-7079-trunk.patch, HBASE-7079-trunk-v2.patch > > > Regression > org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven > Failing for the past 1 build (Since #147 ) > Took 2 ms. > Error Message > -1 > Stacktrace > java.lang.ArrayIndexOutOfBoundsException: -1 > at > java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:343) > at java.util.Collections$UnmodifiableList.get(Collections.java:1154) > at > org.apache.hadoop.hbase.HBaseTestingUtility.getRSForFirstRegionInTable(HBaseTestingUtility.java:1300) > at > org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven(TestAdmin.java:1454) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) > at org.junit.rules.RunRules.evaluate(RunRules.java:18) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6793) Make hbase-examples module
[ https://issues.apache.org/jira/browse/HBASE-6793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487573#comment-13487573 ] stack commented on HBASE-6793: -- Patch looks great. Nice cleanup. One question, will we be compiling of the examples every time we build now and running the tests? It seems so going by an invocation w/ this patch applied --> we are building an examples artifiact and it is full of code: $ jar -tf hbase-examples/target/hbase-examples-0.95-SNAPSHOT.jar When we built before this change, did we build example code as part of the general build? Do you know Sergey? > Make hbase-examples module > -- > > Key: HBASE-6793 > URL: https://issues.apache.org/jira/browse/HBASE-6793 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.96.0 >Reporter: Enis Soztutar >Assignee: Sergey Shelukhin > Labels: noob > Attachments: HBASE-6793.patch, HBASE-6793-v2.patch, > HBASE-6793-v3-thrift-0.9.0.patch, HBASE-6793-v4.1-squashed.patch > > > There are some examples under /examples/, which are not compiled as a part of > the build. We can move them to an hbase-examples module. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7043) Region Server Group CLI commands
[ https://issues.apache.org/jira/browse/HBASE-7043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487572#comment-13487572 ] Francis Liu commented on HBASE-7043: What keeps it from getting into 0.94? Is it the new method in HMaster? Since we're only adding a method that should keep things binary compatible to older versions at least. > Region Server Group CLI commands > > > Key: HBASE-7043 > URL: https://issues.apache.org/jira/browse/HBASE-7043 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 0.96.0 > > Attachments: HBASE-7043_94.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487571#comment-13487571 ] Francis Liu commented on HBASE-6721: In the current implementation we handle META the same way as all the other tables. Since we just load the group information from hdfs. If the "group" information was stored on a table we won't be able to access the group information through normal means until after we assign ROOT, META and the "group" table itself. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Vandana Ayyalasomayajula > Fix For: 0.96.0 > > Attachments: HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721-DesigDoc.pdf > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7070) Scanner may retry forever after HBASE-5974
[ https://issues.apache.org/jira/browse/HBASE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487569#comment-13487569 ] stack commented on HBASE-7070: -- Should it be a flag that gets set when we are retrying because of a OutOfOrderScannerNextException? The flag, called retryAfterOutOfOrderException or some such then stays set while we retry. If we get a OutOfOrderScannerNextException again, we fail with an exception saying failed after retry. If next succeeds, flag is cleared. When you have it like this counting attempts, it is a little confusing I think. We will go around and around if we are getting socket timeouts? > Scanner may retry forever after HBASE-5974 > --- > > Key: HBASE-7070 > URL: https://issues.apache.org/jira/browse/HBASE-7070 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-7070.patch, HBASE-7070v2.patch, > HBASE-7070v3.patch, HBASE-7070v4.patch, HBASE-7070v5.patch > > > After the patch of HBASE-5974 > Suppose The process is as the following: > 1.A next request is very large, so first time it is failed because of timeout > 2.The second time it is failed again because of > CallSequenceOutOfOrderException > 3.We will retry this next request again after reset scanner, so it is also > very large and failed again because of timeout > 4.CallSequenceOutOfOrderException again > 5.Repeated the above forever... > See the comment in HBASE-5974 for more > https://issues.apache.org/jira/browse/HBASE-5974?focusedCommentId=13486658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13486658 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7066) Some HMaster coprocessor exceptions are being swallowed in try catch blocks
[
https://issues.apache.org/jira/browse/HBASE-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487567#comment-13487567
]
Hadoop QA commented on HBASE-7066:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551482/HBASE-7066_trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3195//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3195//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3195//console
This message is automatically generated.
> Some HMaster coprocessor exceptions are being swallowed in try catch blocks
> ---
>
> Key: HBASE-7066
> URL: https://issues.apache.org/jira/browse/HBASE-7066
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, security
>Affects Versions: 0.94.2, 0.96.0
>Reporter: Francis Liu
>Assignee: Francis Liu
>Priority: Critical
> Attachments: HBASE-7066_94.patch, HBASE-7066_trunk.patch,
> HBASE-7066_trunk.patch
>
>
> This is causing HMaster.shutdown() and HMaster.stopMaster() to succeed even
> when an AccessDeniedException is thrown.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7079) testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails
[ https://issues.apache.org/jira/browse/HBASE-7079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487566#comment-13487566 ] Zhou wenjian commented on HBASE-7079: - patch for trunk > testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails > -- > > Key: HBASE-7079 > URL: https://issues.apache.org/jira/browse/HBASE-7079 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.94.2 >Reporter: Zhou wenjian >Assignee: Zhou wenjian > Attachments: HBASE-7079-trunk.patch, HBASE-7079-trunk-v2.patch > > > Regression > org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven > Failing for the past 1 build (Since #147 ) > Took 2 ms. > Error Message > -1 > Stacktrace > java.lang.ArrayIndexOutOfBoundsException: -1 > at > java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:343) > at java.util.Collections$UnmodifiableList.get(Collections.java:1154) > at > org.apache.hadoop.hbase.HBaseTestingUtility.getRSForFirstRegionInTable(HBaseTestingUtility.java:1300) > at > org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven(TestAdmin.java:1454) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) > at org.junit.rules.RunRules.evaluate(RunRules.java:18) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7077) checkAndPut should properly integrate with MVCC
[ https://issues.apache.org/jira/browse/HBASE-7077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487565#comment-13487565 ] ramkrishna.s.vasudevan commented on HBASE-7077: --- I will try to write a testcase for this. That day i reproduced with debugging. > checkAndPut should properly integrate with MVCC > --- > > Key: HBASE-7077 > URL: https://issues.apache.org/jira/browse/HBASE-7077 > Project: HBase > Issue Type: Sub-task >Reporter: Gregory Chanan > Fix For: 0.96.0 > > > checkAndPut should integrate with MVCC, similar to how HBASE-4583 fixed > appends and increments. > Also need a test, here's one we could use (originally proposed in HBASE-7051): > The current value of some cell is 10. > I issue two concurrent requests: > A) a check and put where check value = 10, put value = 11 > B) a put where put value = 50 > The only result at the end of these operations that seems reasonable to me is > the value of the cell being 50. If A occurred first (ACID wise), then our > values go 10->11->50. If B occurred first, then our values go 10->50 (and the > checkAndPut fails) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7079) testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails
[ https://issues.apache.org/jira/browse/HBASE-7079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou wenjian updated HBASE-7079: Attachment: HBASE-7079-trunk-v2.patch > testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails > -- > > Key: HBASE-7079 > URL: https://issues.apache.org/jira/browse/HBASE-7079 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.94.2 >Reporter: Zhou wenjian >Assignee: Zhou wenjian > Attachments: HBASE-7079-trunk.patch, HBASE-7079-trunk-v2.patch > > > Regression > org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven > Failing for the past 1 build (Since #147 ) > Took 2 ms. > Error Message > -1 > Stacktrace > java.lang.ArrayIndexOutOfBoundsException: -1 > at > java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:343) > at java.util.Collections$UnmodifiableList.get(Collections.java:1154) > at > org.apache.hadoop.hbase.HBaseTestingUtility.getRSForFirstRegionInTable(HBaseTestingUtility.java:1300) > at > org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven(TestAdmin.java:1454) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) > at org.junit.rules.RunRules.evaluate(RunRules.java:18) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7070) Scanner may retry forever after HBASE-5974
[
https://issues.apache.org/jira/browse/HBASE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487563#comment-13487563
]
Anoop Sam John commented on HBASE-7070:
---
tries++; We can move inside instanceof OutOfOrderScannerNextException check?
{code}
+if (e instanceof OutOfOrderScannerNextException) {
+ if(tries >= 2)
+throw new DoNotRetryIOException("Failed after attempts=" +
tries
+ + ", it could be cause by rpc timeout", e);
+ tries++;
+}
{code}
Thanks Chunhui for your work on this ..
> Scanner may retry forever after HBASE-5974
> ---
>
> Key: HBASE-7070
> URL: https://issues.apache.org/jira/browse/HBASE-7070
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.96.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-7070.patch, HBASE-7070v2.patch,
> HBASE-7070v3.patch, HBASE-7070v4.patch, HBASE-7070v5.patch
>
>
> After the patch of HBASE-5974
> Suppose The process is as the following:
> 1.A next request is very large, so first time it is failed because of timeout
> 2.The second time it is failed again because of
> CallSequenceOutOfOrderException
> 3.We will retry this next request again after reset scanner, so it is also
> very large and failed again because of timeout
> 4.CallSequenceOutOfOrderException again
> 5.Repeated the above forever...
> See the comment in HBASE-5974 for more
> https://issues.apache.org/jira/browse/HBASE-5974?focusedCommentId=13486658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13486658
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7042) Master Coprocessor Endpoint
[ https://issues.apache.org/jira/browse/HBASE-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487560#comment-13487560 ] Francis Liu commented on HBASE-7042: Ted, not sure why this would break rolling restart? > Master Coprocessor Endpoint > --- > > Key: HBASE-7042 > URL: https://issues.apache.org/jira/browse/HBASE-7042 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 0.96.0 > > Attachments: HBASE-7042_94.patch > > > Having support for a master coprocessor endpoint would enable developers to > easily extended HMaster functionality/features. As is the case for region > server grouping. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7079) testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails
[ https://issues.apache.org/jira/browse/HBASE-7079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou wenjian updated HBASE-7079: Attachment: HBASE-7079-trunk.patch > testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails > -- > > Key: HBASE-7079 > URL: https://issues.apache.org/jira/browse/HBASE-7079 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.94.2 >Reporter: Zhou wenjian >Assignee: Zhou wenjian > Attachments: HBASE-7079-trunk.patch > > > Regression > org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven > Failing for the past 1 build (Since #147 ) > Took 2 ms. > Error Message > -1 > Stacktrace > java.lang.ArrayIndexOutOfBoundsException: -1 > at > java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:343) > at java.util.Collections$UnmodifiableList.get(Collections.java:1154) > at > org.apache.hadoop.hbase.HBaseTestingUtility.getRSForFirstRegionInTable(HBaseTestingUtility.java:1300) > at > org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven(TestAdmin.java:1454) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) > at org.junit.rules.RunRules.evaluate(RunRules.java:18) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7079) testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails
[ https://issues.apache.org/jira/browse/HBASE-7079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487556#comment-13487556 ] Zhou wenjian commented on HBASE-7079: - testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails. there is a window between the region adding to meta and online the region in regionserver. during that, the testcase try to get the online region provided, it will return -1, which lead to the case failture. > testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails > -- > > Key: HBASE-7079 > URL: https://issues.apache.org/jira/browse/HBASE-7079 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.94.2 >Reporter: Zhou wenjian >Assignee: Zhou wenjian > > Regression > org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven > Failing for the past 1 build (Since #147 ) > Took 2 ms. > Error Message > -1 > Stacktrace > java.lang.ArrayIndexOutOfBoundsException: -1 > at > java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:343) > at java.util.Collections$UnmodifiableList.get(Collections.java:1154) > at > org.apache.hadoop.hbase.HBaseTestingUtility.getRSForFirstRegionInTable(HBaseTestingUtility.java:1300) > at > org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven(TestAdmin.java:1454) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) > at org.junit.rules.RunRules.evaluate(RunRules.java:18) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7079) testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails
[ https://issues.apache.org/jira/browse/HBASE-7079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou wenjian updated HBASE-7079: Description: Regression org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven Failing for the past 1 build (Since #147 ) Took 2 ms. Error Message -1 Stacktrace java.lang.ArrayIndexOutOfBoundsException: -1 at java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:343) at java.util.Collections$UnmodifiableList.get(Collections.java:1154) at org.apache.hadoop.hbase.HBaseTestingUtility.getRSForFirstRegionInTable(HBaseTestingUtility.java:1300) at org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven(TestAdmin.java:1454) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) at org.junit.rules.RunRules.evaluate(RunRules.java:18) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:24) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) was: Regression org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven Failing for the past 1 build (Since #147 ) Took 2 毫秒. Error Message -1 Stacktrace java.lang.ArrayIndexOutOfBoundsException: -1 at java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:343) at java.util.Collections$UnmodifiableList.get(Collections.java:1154) at org.apache.hadoop.hbase.HBaseTestingUtility.getRSForFirstRegionInTable(HBaseTestingUtility.java:1300) at org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven(TestAdmin.java:1454) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) at or
[jira] [Updated] (HBASE-7051) Read/Updates (increment,checkAndPut) should properly read MVCC
[
https://issues.apache.org/jira/browse/HBASE-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-7051:
-
Status: Patch Available (was: Open)
> Read/Updates (increment,checkAndPut) should properly read MVCC
> --
>
> Key: HBASE-7051
> URL: https://issues.apache.org/jira/browse/HBASE-7051
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Gregory Chanan
>Assignee: Lars Hofhansl
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 7051.txt
>
>
> See, for example:
> {code}
> // TODO: Use MVCC to make this set of increments atomic to reads
> {code}
> Here's an example of what I can happen (would probably be good to write up a
> test case for each read/update):
> Concurrent update via increment and put.
> The put grabs the row lock first and updates the memstore, but releases the
> row lock before the MVCC is advanced. Then, the increment grabs the row lock
> and reads right away, reading the old value and incrementing based on that.
> There are a few options here:
> 1) Waiting for the MVCC to advance for read/updates: the downside is that you
> have to wait for updates on other rows.
> 2) Have an MVCC per-row (table configuration): this avoids the unnecessary
> contention of 1)
> 3) Transform the read/updates to write-only with rollup on read.. E.g. an
> increment would just have the number of values to increment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7051) Read/Updates (increment,checkAndPut) should properly read MVCC
[
https://issues.apache.org/jira/browse/HBASE-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-7051:
-
Attachment: 7051.txt
Super simple patch. Just wait for prior MVCC transactions to finish, before we
execute the Get.
> Read/Updates (increment,checkAndPut) should properly read MVCC
> --
>
> Key: HBASE-7051
> URL: https://issues.apache.org/jira/browse/HBASE-7051
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Gregory Chanan
>Assignee: Lars Hofhansl
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 7051.txt
>
>
> See, for example:
> {code}
> // TODO: Use MVCC to make this set of increments atomic to reads
> {code}
> Here's an example of what I can happen (would probably be good to write up a
> test case for each read/update):
> Concurrent update via increment and put.
> The put grabs the row lock first and updates the memstore, but releases the
> row lock before the MVCC is advanced. Then, the increment grabs the row lock
> and reads right away, reading the old value and incrementing based on that.
> There are a few options here:
> 1) Waiting for the MVCC to advance for read/updates: the downside is that you
> have to wait for updates on other rows.
> 2) Have an MVCC per-row (table configuration): this avoids the unnecessary
> contention of 1)
> 3) Transform the read/updates to write-only with rollup on read.. E.g. an
> increment would just have the number of values to increment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7079) testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails
Zhou wenjian created HBASE-7079: --- Summary: testCloseRegionWhenEncodedRegionNameIsNotGiven occainlly fails Key: HBASE-7079 URL: https://issues.apache.org/jira/browse/HBASE-7079 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.2 Reporter: Zhou wenjian Assignee: Zhou wenjian Regression org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven Failing for the past 1 build (Since #147 ) Took 2 毫秒. Error Message -1 Stacktrace java.lang.ArrayIndexOutOfBoundsException: -1 at java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:343) at java.util.Collections$UnmodifiableList.get(Collections.java:1154) at org.apache.hadoop.hbase.HBaseTestingUtility.getRSForFirstRegionInTable(HBaseTestingUtility.java:1300) at org.apache.hadoop.hbase.client.TestAdmin.testCloseRegionWhenEncodedRegionNameIsNotGiven(TestAdmin.java:1454) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) at org.junit.rules.RunRules.evaluate(RunRules.java:18) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:24) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7070) Scanner may retry forever after HBASE-5974
[ https://issues.apache.org/jira/browse/HBASE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-7070: Attachment: HBASE-7070v5.patch Attaching patchV5: retry only once if OutOfOrderScannerNextException as Stack's comment > Scanner may retry forever after HBASE-5974 > --- > > Key: HBASE-7070 > URL: https://issues.apache.org/jira/browse/HBASE-7070 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-7070.patch, HBASE-7070v2.patch, > HBASE-7070v3.patch, HBASE-7070v4.patch, HBASE-7070v5.patch > > > After the patch of HBASE-5974 > Suppose The process is as the following: > 1.A next request is very large, so first time it is failed because of timeout > 2.The second time it is failed again because of > CallSequenceOutOfOrderException > 3.We will retry this next request again after reset scanner, so it is also > very large and failed again because of timeout > 4.CallSequenceOutOfOrderException again > 5.Repeated the above forever... > See the comment in HBASE-5974 for more > https://issues.apache.org/jira/browse/HBASE-5974?focusedCommentId=13486658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13486658 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487549#comment-13487549
]
Hadoop QA commented on HBASE-2645:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551477/2645v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//console
This message is automatically generated.
> HLog writer can do 1-2 sync operations after lease has been recovered for
> split process.
>
>
> Key: HBASE-2645
> URL: https://issues.apache.org/jira/browse/HBASE-2645
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.90.4
>Reporter: Cosmin Lehene
>Assignee: Todd Lipcon
>Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: 2645.txt, 2645v2.txt, 2645v3.txt
>
>
> TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing.
> This test starts a thread that writes one edit to the log, syncs and counts.
> During this, a HLog.splitLog operation is started. splitLog recovers the log
> lease before reading the log, so that the original regionserver could not
> wake up and write after the split process started.
> The test compares the number of edits reported by the split process and by
> the writer thread. Writer thread (called zombie in the test) should report <=
> than the splitLog (sync() might raise after the last edit gets written and
> the edit won't get counted by zombie thread). However it appears that the
> zombie counts 1-2 more edits. So it looks like it can sync without a lease.
> This might be a hdfs-0.20 related issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7051) Read/Updates (increment,checkAndPut) should properly read MVCC
[
https://issues.apache.org/jira/browse/HBASE-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl reassigned HBASE-7051:
Assignee: Lars Hofhansl
> Read/Updates (increment,checkAndPut) should properly read MVCC
> --
>
> Key: HBASE-7051
> URL: https://issues.apache.org/jira/browse/HBASE-7051
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Gregory Chanan
>Assignee: Lars Hofhansl
> Fix For: 0.94.3, 0.96.0
>
>
> See, for example:
> {code}
> // TODO: Use MVCC to make this set of increments atomic to reads
> {code}
> Here's an example of what I can happen (would probably be good to write up a
> test case for each read/update):
> Concurrent update via increment and put.
> The put grabs the row lock first and updates the memstore, but releases the
> row lock before the MVCC is advanced. Then, the increment grabs the row lock
> and reads right away, reading the old value and incrementing based on that.
> There are a few options here:
> 1) Waiting for the MVCC to advance for read/updates: the downside is that you
> have to wait for updates on other rows.
> 2) Have an MVCC per-row (table configuration): this avoids the unnecessary
> contention of 1)
> 3) Transform the read/updates to write-only with rollup on read.. E.g. an
> increment would just have the number of values to increment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7051) Read/Updates (increment,checkAndPut) should properly read MVCC
[
https://issues.apache.org/jira/browse/HBASE-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487545#comment-13487545
]
Lars Hofhansl commented on HBASE-7051:
--
I would say this (checkAndXXX) is somewhat more critical than HBASE-4583 as it
can actually lead to incorrect, *permanent* results.
> Read/Updates (increment,checkAndPut) should properly read MVCC
> --
>
> Key: HBASE-7051
> URL: https://issues.apache.org/jira/browse/HBASE-7051
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Gregory Chanan
> Fix For: 0.94.3, 0.96.0
>
>
> See, for example:
> {code}
> // TODO: Use MVCC to make this set of increments atomic to reads
> {code}
> Here's an example of what I can happen (would probably be good to write up a
> test case for each read/update):
> Concurrent update via increment and put.
> The put grabs the row lock first and updates the memstore, but releases the
> row lock before the MVCC is advanced. Then, the increment grabs the row lock
> and reads right away, reading the old value and incrementing based on that.
> There are a few options here:
> 1) Waiting for the MVCC to advance for read/updates: the downside is that you
> have to wait for updates on other rows.
> 2) Have an MVCC per-row (table configuration): this avoids the unnecessary
> contention of 1)
> 3) Transform the read/updates to write-only with rollup on read.. E.g. an
> increment would just have the number of values to increment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6904) In the HBase shell, an error is thrown that states replication-related znodes already exist
[ https://issues.apache.org/jira/browse/HBASE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487543#comment-13487543 ] Lars Hofhansl commented on HBASE-6904: -- Looks good to me +1 > In the HBase shell, an error is thrown that states replication-related znodes > already exist > --- > > Key: HBASE-6904 > URL: https://issues.apache.org/jira/browse/HBASE-6904 > Project: HBase > Issue Type: Bug > Components: Replication, Zookeeper >Affects Versions: 0.92.0, 0.92.1 >Reporter: Aleksandr Shulman >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.94.3 > > Attachments: HBASE-6904.patch > > > On a replication-enabled cluster, querying the list_peers produces the error > lines shown below. It doesn't appear that anything is broken in terms of > functionality. > Stack trace: > hbase(main):001:0> list_peers > 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node > /hbase/replication/peers already exists and this is not a retry > 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node > /hbase/replication/rs already exists and this is not a retry > PEER ID CLUSTER KEY > 0 row(s) in 0.4650 seconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487542#comment-13487542 ] Lars Hofhansl commented on HBASE-5898: -- What should we do with this one? @Ram: you're probably right, on the other hand I cannot see anything wrong with this code (and I looked at it multiple times). The proposed patch will probably not fix the problem you saw. And from some comment earlier it is doubtful that this patch is even an improvement for the issue that Todd saw. I'd say we should table this (at least for 0.94), until we know what exactly the issue is. > Consider double-checked locking for block cache lock > > > Key: HBASE-5898 > URL: https://issues.apache.org/jira/browse/HBASE-5898 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 0.94.1 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Fix For: 0.94.3, 0.96.0 > > Attachments: 5898-TestBlocksRead.txt, HBASE-5898-0.patch, > HBASE-5898-1.patch, hbase-5898.txt > > > Running a workload with a high query rate against a dataset that fits in > cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by > HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a > lot of CPU doing lock management here. I wrote a quick patch to switch to a > double-checked locking and it improved throughput substantially for this > workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7070) Scanner may retry forever after HBASE-5974
[ https://issues.apache.org/jira/browse/HBASE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-7070: Attachment: HBASE-7070v4.patch > Scanner may retry forever after HBASE-5974 > --- > > Key: HBASE-7070 > URL: https://issues.apache.org/jira/browse/HBASE-7070 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-7070.patch, HBASE-7070v2.patch, > HBASE-7070v3.patch, HBASE-7070v4.patch > > > After the patch of HBASE-5974 > Suppose The process is as the following: > 1.A next request is very large, so first time it is failed because of timeout > 2.The second time it is failed again because of > CallSequenceOutOfOrderException > 3.We will retry this next request again after reset scanner, so it is also > very large and failed again because of timeout > 4.CallSequenceOutOfOrderException again > 5.Repeated the above forever... > See the comment in HBASE-5974 for more > https://issues.apache.org/jira/browse/HBASE-5974?focusedCommentId=13486658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13486658 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7073) OperationMetrics needs to cache the value of hbase.metrics.exposeOperationTimes
[
https://issues.apache.org/jira/browse/HBASE-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487540#comment-13487540
]
Lars Hofhansl commented on HBASE-7073:
--
nice
> OperationMetrics needs to cache the value of
> hbase.metrics.exposeOperationTimes
> ---
>
> Key: HBASE-7073
> URL: https://issues.apache.org/jira/browse/HBASE-7073
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.2
>Reporter: Jean-Daniel Cryans
>Assignee: Elliott Clark
>Priority: Minor
> Fix For: 0.94.3
>
> Attachments: HBASE-7073-0.patch
>
>
> Trying some increments on my local machine I was surprised to see this in my
> jstacks:
> {noformat}
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1061)
> - locked <7c4a26430> (a org.apache.hadoop.conf.Configuration)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:416)
> at
> org.apache.hadoop.hbase.regionserver.CompoundConfiguration$1.get(CompoundConfiguration.java:94)
> at
> org.apache.hadoop.hbase.regionserver.CompoundConfiguration.get(CompoundConfiguration.java:186)
> at
> org.apache.hadoop.hbase.regionserver.CompoundConfiguration.getBoolean(CompoundConfiguration.java:318)
> at
> org.apache.hadoop.hbase.regionserver.metrics.OperationMetrics.doSafeIncTimeVarying(OperationMetrics.java:217)
> at
> org.apache.hadoop.hbase.regionserver.metrics.OperationMetrics.doUpdateTimeVarying(OperationMetrics.java:212)
> at
> org.apache.hadoop.hbase.regionserver.metrics.OperationMetrics.updateIncrementMetrics(OperationMetrics.java:133)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:4817)
> {noformat}
> It's a pretty horrible lookup that's inline with everything else in that
> class and there's no reason why it shouldn't be a final boolean.
> Assigning this to the master of metrics since he asked for it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7070) Scanner may retry forever after HBASE-5974
[ https://issues.apache.org/jira/browse/HBASE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487539#comment-13487539 ] stack commented on HBASE-7070: -- bq. I think retrying once is same with some maximum retries number, or else we don't do the retry for OutOfOrderException.. CLIENT_RETRIES_NUMBER is overloaded. Lets not reuse it here doing yet more retries on top of retry counting. I think we retry once if OutOfOrderException given Anoop has done all the work getting it in. I'd say we only do it once because unlikely we'll succeed if we just did sockettimeout twice because it is too big. Just a suggestion Thanks for doing this Chunhui. > Scanner may retry forever after HBASE-5974 > --- > > Key: HBASE-7070 > URL: https://issues.apache.org/jira/browse/HBASE-7070 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-7070.patch, HBASE-7070v2.patch, HBASE-7070v3.patch > > > After the patch of HBASE-5974 > Suppose The process is as the following: > 1.A next request is very large, so first time it is failed because of timeout > 2.The second time it is failed again because of > CallSequenceOutOfOrderException > 3.We will retry this next request again after reset scanner, so it is also > very large and failed again because of timeout > 4.CallSequenceOutOfOrderException again > 5.Repeated the above forever... > See the comment in HBASE-5974 for more > https://issues.apache.org/jira/browse/HBASE-5974?focusedCommentId=13486658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13486658 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
[ https://issues.apache.org/jira/browse/HBASE-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487535#comment-13487535 ] stack commented on HBASE-6965: -- Yeah, you could open a new issue to rename (I think it should be JVM not JVMBean. Its not even a Bean not to mind a management Bean). Regards the ShellCommandExecutor, it is not a case of whether it has more functionality than what you have added, its that we do not want repeated code in our code base which seems to be what is going on here. Correct me if I am wrong? Thanks. > Generic MXBean Utility class to support all JDK vendors > --- > > Key: HBASE-6965 > URL: https://issues.apache.org/jira/browse/HBASE-6965 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 0.94.1 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Labels: patch > Fix For: 0.96.0, 0.94.4 > > Attachments: HBASE-6965.patch, OSMXBean_HBASE-6965-0.94.patch > > > This issue is related to JIRA > https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to > propose the use of a newly created generic > org.apache.hadoop.hbase.util.OSMXBean class that can be used by other > classes. JIRA HBASE-6945 contains a patch for the class > org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the > inclusion of this new class, HBase can be built and become functional with > JDKs and JREs other than what is provided by Oracle. > This class uses reflection to determine the JVM vendor (Sun, IBM) and the > platform (Linux or Windows), and contains other methods that return the OS > properties - 1. Number of Open File descriptors; 2. Maximum number of File > Descriptors. > This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well > as Oracle JDK 6. Junit tests (runDevTests category) completed without any > failures or errors when tested on all the three JDKs.The builds and tests > were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487530#comment-13487530 ] stack commented on HBASE-6945: -- [~kumarr] You seem to be missing the point of my review. I understand why you would want to make a patch like this. No need to keep reiterating the why. It makes sense. My comment is that your patch would seem to make a class w/ no content: i.e. OSResourceAnalyzer. Why would we ever want a class w/ no content in our code base? Just remove it? > Compilation errors when using non-Sun JDKs to build HBase-0.94 > -- > > Key: HBASE-6945 > URL: https://issues.apache.org/jira/browse/HBASE-6945 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.94.1 > Environment: RHEL 6.3, IBM Java 7 >Reporter: Kumar Ravi >Assignee: Kumar Ravi > Labels: patch > Fix For: 0.94.4 > > Attachments: ResourceCheckerJUnitListener_HBASE_6945-trunk.patch > > > When using IBM Java 7 to build HBase-0.94.1, the following comilation error > is seen. > [INFO] - > [ERROR] COMPILATION ERROR : > [INFO] - > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] > error: package com.sun.management does not exist > [ERROR] > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] > error: cannot find symbol > [ERROR] symbol: class UnixOperatingSystemMXBean > location: class ResourceAnalyzer > /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] > error: cannot find symbol > [INFO] 4 errors > [INFO] - > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > I have a patch available which should work for all JDKs including Sun. > I am in the process of testing this patch. Preliminary tests indicate the > build is working fine with this patch. I will post this patch when I am done > testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7078) Add a test that append is atomic
[ https://issues.apache.org/jira/browse/HBASE-7078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-7078. -- Resolution: Duplicate I've done this as part of HBASE-4583 > Add a test that append is atomic > > > Key: HBASE-7078 > URL: https://issues.apache.org/jira/browse/HBASE-7078 > Project: HBase > Issue Type: Sub-task >Reporter: Gregory Chanan > Fix For: 0.94.3, 0.96.0 > > > HBASE-4583 adds a test that increment is atomic; we should add one for append > as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7070) Scanner may retry forever after HBASE-5974
[ https://issues.apache.org/jira/browse/HBASE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-7070: Attachment: HBASE-7070v3.patch Attaching patchV3 bq.We have enough retrying and loops of retries already in the client Retry in ClientScanner is different with ServerCallable#withRetries, the former will reset scanner and the latter only send next request again. bq.Should we retry if OutOfOrderException only, and retry once only rather than some maximum retries number? I think retrying once is same with some maximum retries number, or else we don't do the retry for OutOfOrderException.. @ted 2 * CLIENT_RETRIES_NUMBER is only used to simulate the infinite retry number, so we expect maximal try number = CLIENT_RETRIES_NUMBER, and I has amended it in assertTrue from RegionServerWithScanTimeout.tryNumber <= 2*CLIENT_RETRIES_NUMBER to RegionServerWithScanTimeout.tryNumber <= CLIENT_RETRIES_NUMBER > Scanner may retry forever after HBASE-5974 > --- > > Key: HBASE-7070 > URL: https://issues.apache.org/jira/browse/HBASE-7070 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-7070.patch, HBASE-7070v2.patch, HBASE-7070v3.patch > > > After the patch of HBASE-5974 > Suppose The process is as the following: > 1.A next request is very large, so first time it is failed because of timeout > 2.The second time it is failed again because of > CallSequenceOutOfOrderException > 3.We will retry this next request again after reset scanner, so it is also > very large and failed again because of timeout > 4.CallSequenceOutOfOrderException again > 5.Repeated the above forever... > See the comment in HBASE-5974 for more > https://issues.apache.org/jira/browse/HBASE-5974?focusedCommentId=13486658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13486658 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4583) Integrate RWCC with Append and Increment operations
[ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4583: - Attachment: 4583-trunk-less-radical-v6.txt This is hopefully the last version here. Add a test for Append. I tried to remove updateColumnValue, but the tests are in part testing specific updateColumnValue functionality, which is hard to separate from the upsert tests. That is for another jira, I think. > Integrate RWCC with Append and Increment operations > --- > > Key: HBASE-4583 > URL: https://issues.apache.org/jira/browse/HBASE-4583 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.96.0 > > Attachments: 4583-trunk-less-radical.txt, > 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, > 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, > 4583-trunk-less-radical-v6.txt, 4583-trunk-radical.txt, > 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, > 4583-v3.txt, 4583-v4.txt > > > Currently Increment and Append operations do not work with RWCC and hence a > client could see the results of multiple such operation mixed in the same > Get/Scan. > The semantics might be a bit more interesting here as upsert adds and removes > to and from the memstore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7066) Some HMaster coprocessor exceptions are being swallowed in try catch blocks
[ https://issues.apache.org/jira/browse/HBASE-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-7066: --- Attachment: HBASE-7066_trunk.patch > Some HMaster coprocessor exceptions are being swallowed in try catch blocks > --- > > Key: HBASE-7066 > URL: https://issues.apache.org/jira/browse/HBASE-7066 > Project: HBase > Issue Type: Bug > Components: Coprocessors, security >Affects Versions: 0.94.2, 0.96.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > Attachments: HBASE-7066_94.patch, HBASE-7066_trunk.patch, > HBASE-7066_trunk.patch > > > This is causing HMaster.shutdown() and HMaster.stopMaster() to succeed even > when an AccessDeniedException is thrown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7066) Some HMaster coprocessor exceptions are being swallowed in try catch blocks
[ https://issues.apache.org/jira/browse/HBASE-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487526#comment-13487526 ] Francis Liu commented on HBASE-7066: Ted, Fixed the nit. > Some HMaster coprocessor exceptions are being swallowed in try catch blocks > --- > > Key: HBASE-7066 > URL: https://issues.apache.org/jira/browse/HBASE-7066 > Project: HBase > Issue Type: Bug > Components: Coprocessors, security >Affects Versions: 0.94.2, 0.96.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > Attachments: HBASE-7066_94.patch, HBASE-7066_trunk.patch, > HBASE-7066_trunk.patch > > > This is causing HMaster.shutdown() and HMaster.stopMaster() to succeed even > when an AccessDeniedException is thrown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7066) Some HMaster coprocessor exceptions are being swallowed in try catch blocks
[ https://issues.apache.org/jira/browse/HBASE-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487521#comment-13487521 ] ramkrishna.s.vasudevan commented on HBASE-7066: --- +1 for trunk patch. Also i would feel not to change the 0.94 balance(). Because in one of the issues we tried to change it but did not make it as it will result in api change. > Some HMaster coprocessor exceptions are being swallowed in try catch blocks > --- > > Key: HBASE-7066 > URL: https://issues.apache.org/jira/browse/HBASE-7066 > Project: HBase > Issue Type: Bug > Components: Coprocessors, security >Affects Versions: 0.94.2, 0.96.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > Attachments: HBASE-7066_94.patch, HBASE-7066_trunk.patch > > > This is causing HMaster.shutdown() and HMaster.stopMaster() to succeed even > when an AccessDeniedException is thrown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6826) [WINDOWS] TestFromClientSide failures
[
https://issues.apache.org/jira/browse/HBASE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487520#comment-13487520
]
stack commented on HBASE-6826:
--
+1 on commit
> [WINDOWS] TestFromClientSide failures
> -
>
> Key: HBASE-6826
> URL: https://issues.apache.org/jira/browse/HBASE-6826
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.3, 0.96.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Labels: windows
> Attachments: hbase-6826_v1-0.94.patch, hbase-6826_v1-trunk.patch,
> hbase-6826_v2-0.94.patch, hbase-6826_v2-trunk.patch
>
>
> The following tests fail for TestFromClientSide:
> {code}
> testPoolBehavior()
> testClientPoolRoundRobin()
> testClientPoolThreadLocal()
> {code}
> The first test fails due to the fact that the test (wrongly) assumes that
> ThredPoolExecutor can reclaim the thread immediately.
> The second and third tests seem to fail because that Put's to the table does
> not specify an explicit timestamp, but on windows, consecutive calls to put
> happen to finish in the same milisecond so that the resulting mutations have
> the same timestamp, thus there is only one version of the cell value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-2645: - Resolution: Fixed Status: Resolved (was: Patch Available) Reenabled and updated the test that was failing. It passes now. Original issue seems to have been hdfs problem since fixed by our move to hdfs 1.0+. > HLog writer can do 1-2 sync operations after lease has been recovered for > split process. > > > Key: HBASE-2645 > URL: https://issues.apache.org/jira/browse/HBASE-2645 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 0.90.4 >Reporter: Cosmin Lehene >Assignee: Todd Lipcon >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 2645.txt, 2645v2.txt, 2645v3.txt > > > TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. > This test starts a thread that writes one edit to the log, syncs and counts. > During this, a HLog.splitLog operation is started. splitLog recovers the log > lease before reading the log, so that the original regionserver could not > wake up and write after the split process started. > The test compares the number of edits reported by the split process and by > the writer thread. Writer thread (called zombie in the test) should report <= > than the splitLog (sync() might raise after the last edit gets written and > the edit won't get counted by zombie thread). However it appears that the > zombie counts 1-2 more edits. So it looks like it can sync without a lease. > This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6775) Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix
[ https://issues.apache.org/jira/browse/HBASE-6775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487517#comment-13487517 ] stack commented on HBASE-6775: -- No objection on my part after the above explanation. Would suggest that you add the above text into the description of the new config. > Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix > -- > > Key: HBASE-6775 > URL: https://issues.apache.org/jira/browse/HBASE-6775 > Project: HBase > Issue Type: Improvement > Components: Zookeeper >Affects Versions: 0.94.2 >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.94.4 > > > HBASE-6710 fixed a 0.92/0.94 compatibility issue by writing two znodes in > different formats. > If a ZK failure occurs between the writing of the two znodes, strange > behavior can result. > This issue is a reminder to change these two ZK writes to use ZK.multi when > we require ZK 3.4+. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-2645: - Attachment: 2645v3.txt Thanks Ted. This is what I'll commit. Its the fixed test minus the pollution introduced in v2 (that caused TestShell) to pass. I ran TestShell locally a few times w/ v3 and it passes. Going to commit. > HLog writer can do 1-2 sync operations after lease has been recovered for > split process. > > > Key: HBASE-2645 > URL: https://issues.apache.org/jira/browse/HBASE-2645 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 0.90.4 >Reporter: Cosmin Lehene >Assignee: Todd Lipcon >Priority: Blocker > Fix For: 0.96.0 > > Attachments: 2645.txt, 2645v2.txt, 2645v3.txt > > > TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. > This test starts a thread that writes one edit to the log, syncs and counts. > During this, a HLog.splitLog operation is started. splitLog recovers the log > lease before reading the log, so that the original regionserver could not > wake up and write after the split process started. > The test compares the number of edits reported by the split process and by > the writer thread. Writer thread (called zombie in the test) should report <= > than the splitLog (sync() might raise after the last edit gets written and > the edit won't get counted by zombie thread). However it appears that the > zombie counts 1-2 more edits. So it looks like it can sync without a lease. > This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7070) Scanner may retry forever after HBASE-5974
[
https://issues.apache.org/jira/browse/HBASE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487515#comment-13487515
]
Ted Yu commented on HBASE-7070:
---
2 * CLIENT_RETRIES_NUMBER appears twice in the patch, consider introducing a
constant for it.
{code}
+assertTrue("Expected try number=" + CLIENT_RETRIES_NUMBER + ", actual ="
++ RegionServerWithScanTimeout.tryNumber,
+RegionServerWithScanTimeout.tryNumber <= 2 * CLIENT_RETRIES_NUMBER);
{code}
Expected try number is 2 * CLIENT_RETRIES_NUMBER, not CLIENT_RETRIES_NUMBER.
{code}
+} catch (IOException ioe) {
+ // catch the exception after max retry number
{code}
Add a log in the catch block ?
> Scanner may retry forever after HBASE-5974
> ---
>
> Key: HBASE-7070
> URL: https://issues.apache.org/jira/browse/HBASE-7070
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.96.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-7070.patch, HBASE-7070v2.patch
>
>
> After the patch of HBASE-5974
> Suppose The process is as the following:
> 1.A next request is very large, so first time it is failed because of timeout
> 2.The second time it is failed again because of
> CallSequenceOutOfOrderException
> 3.We will retry this next request again after reset scanner, so it is also
> very large and failed again because of timeout
> 4.CallSequenceOutOfOrderException again
> 5.Repeated the above forever...
> See the comment in HBASE-5974 for more
> https://issues.apache.org/jira/browse/HBASE-5974?focusedCommentId=13486658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13486658
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6904) In the HBase shell, an error is thrown that states replication-related znodes already exist
[ https://issues.apache.org/jira/browse/HBASE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487511#comment-13487511 ] stack commented on HBASE-6904: -- +1 on commit > In the HBase shell, an error is thrown that states replication-related znodes > already exist > --- > > Key: HBASE-6904 > URL: https://issues.apache.org/jira/browse/HBASE-6904 > Project: HBase > Issue Type: Bug > Components: Replication, Zookeeper >Affects Versions: 0.92.0, 0.92.1 >Reporter: Aleksandr Shulman >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.94.3 > > Attachments: HBASE-6904.patch > > > On a replication-enabled cluster, querying the list_peers produces the error > lines shown below. It doesn't appear that anything is broken in terms of > functionality. > Stack trace: > hbase(main):001:0> list_peers > 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node > /hbase/replication/peers already exists and this is not a retry > 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node > /hbase/replication/rs already exists and this is not a retry > PEER ID CLUSTER KEY > 0 row(s) in 0.4650 seconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487509#comment-13487509 ] stack commented on HBASE-7055: -- [~sershe] On, someone should review, yes, I agree. Trying to. > port HBASE-6371 tier-based compaction from 0.89-fb to trunk > --- > > Key: HBASE-7055 > URL: https://issues.apache.org/jira/browse/HBASE-7055 > Project: HBase > Issue Type: Task > Components: Compaction >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.96.0 > > Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch > > > There's divergence in the code :( > See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487505#comment-13487505 ] stack commented on HBASE-7055: -- Please edit the release note and remove this sentence "A tier based compaction algorithm has been developed." I would be interested in when this new compaction should be used in place of what we have (both as prospective user and as an hbase dev) and how it relates to what we have currently. [~sershe] On 'Most of the patch (e.g. CompactionManager) is refactoring of the compaction code out of (H)Store, so the code as such as old, except for the alternative CompactionManager.' <-- This sounds good. > port HBASE-6371 tier-based compaction from 0.89-fb to trunk > --- > > Key: HBASE-7055 > URL: https://issues.apache.org/jira/browse/HBASE-7055 > Project: HBase > Issue Type: Task > Components: Compaction >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.96.0 > > Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch > > > There's divergence in the code :( > See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7070) Scanner may retry forever after HBASE-5974
[ https://issues.apache.org/jira/browse/HBASE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-7070: Attachment: HBASE-7070v2.patch As ted's comment, patchV2 modify the test case to produce the problem and rename the variable to maxRetries > Scanner may retry forever after HBASE-5974 > --- > > Key: HBASE-7070 > URL: https://issues.apache.org/jira/browse/HBASE-7070 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-7070.patch, HBASE-7070v2.patch > > > After the patch of HBASE-5974 > Suppose The process is as the following: > 1.A next request is very large, so first time it is failed because of timeout > 2.The second time it is failed again because of > CallSequenceOutOfOrderException > 3.We will retry this next request again after reset scanner, so it is also > very large and failed again because of timeout > 4.CallSequenceOutOfOrderException again > 5.Repeated the above forever... > See the comment in HBASE-5974 for more > https://issues.apache.org/jira/browse/HBASE-5974?focusedCommentId=13486658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13486658 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7073) OperationMetrics needs to cache the value of hbase.metrics.exposeOperationTimes
[
https://issues.apache.org/jira/browse/HBASE-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487503#comment-13487503
]
Hudson commented on HBASE-7073:
---
Integrated in HBase-0.94 #561 (See
[https://builds.apache.org/job/HBase-0.94/561/])
HBASE-7073 OperationMetrics needs to cache the value of
hbase.metrics.exposeOperationTimes (Revision 1403865)
Result = SUCCESS
eclark :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/OperationMetrics.java
> OperationMetrics needs to cache the value of
> hbase.metrics.exposeOperationTimes
> ---
>
> Key: HBASE-7073
> URL: https://issues.apache.org/jira/browse/HBASE-7073
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.2
>Reporter: Jean-Daniel Cryans
>Assignee: Elliott Clark
>Priority: Minor
> Fix For: 0.94.3
>
> Attachments: HBASE-7073-0.patch
>
>
> Trying some increments on my local machine I was surprised to see this in my
> jstacks:
> {noformat}
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1061)
> - locked <7c4a26430> (a org.apache.hadoop.conf.Configuration)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:416)
> at
> org.apache.hadoop.hbase.regionserver.CompoundConfiguration$1.get(CompoundConfiguration.java:94)
> at
> org.apache.hadoop.hbase.regionserver.CompoundConfiguration.get(CompoundConfiguration.java:186)
> at
> org.apache.hadoop.hbase.regionserver.CompoundConfiguration.getBoolean(CompoundConfiguration.java:318)
> at
> org.apache.hadoop.hbase.regionserver.metrics.OperationMetrics.doSafeIncTimeVarying(OperationMetrics.java:217)
> at
> org.apache.hadoop.hbase.regionserver.metrics.OperationMetrics.doUpdateTimeVarying(OperationMetrics.java:212)
> at
> org.apache.hadoop.hbase.regionserver.metrics.OperationMetrics.updateIncrementMetrics(OperationMetrics.java:133)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:4817)
> {noformat}
> It's a pretty horrible lookup that's inline with everything else in that
> class and there's no reason why it shouldn't be a final boolean.
> Assigning this to the master of metrics since he asked for it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7070) Scanner may retry forever after HBASE-5974
[ https://issues.apache.org/jira/browse/HBASE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487500#comment-13487500 ] stack commented on HBASE-7070: -- [~zjushch] Yes. Good catch. I don't like that we are adding new max retry loop up in ClientScanner. We have enough retrying and loops of retries already in the client. The case you describe is a little pathological. Is it every going to succeed? Not how you paint it. Should we retry if OutOfOrderException only, and retry once only rather than some maximum retries number? Good stuff. > Scanner may retry forever after HBASE-5974 > --- > > Key: HBASE-7070 > URL: https://issues.apache.org/jira/browse/HBASE-7070 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-7070.patch > > > After the patch of HBASE-5974 > Suppose The process is as the following: > 1.A next request is very large, so first time it is failed because of timeout > 2.The second time it is failed again because of > CallSequenceOutOfOrderException > 3.We will retry this next request again after reset scanner, so it is also > very large and failed again because of timeout > 4.CallSequenceOutOfOrderException again > 5.Repeated the above forever... > See the comment in HBASE-5974 for more > https://issues.apache.org/jira/browse/HBASE-5974?focusedCommentId=13486658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13486658 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations
[ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487499#comment-13487499 ] Lars Hofhansl commented on HBASE-4583: -- Alas, that would not work nicely with mslab. [~jdcryans] I was wondering whether you could do your same performance test with the "radical" version of this patch (4583-trunk-v3.txt)? > Integrate RWCC with Append and Increment operations > --- > > Key: HBASE-4583 > URL: https://issues.apache.org/jira/browse/HBASE-4583 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.96.0 > > Attachments: 4583-trunk-less-radical.txt, > 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, > 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, > 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, > 4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt > > > Currently Increment and Append operations do not work with RWCC and hence a > client could see the results of multiple such operation mixed in the same > Get/Scan. > The semantics might be a bit more interesting here as upsert adds and removes > to and from the memstore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487493#comment-13487493 ] Todd Lipcon commented on HBASE-7055: What I'm interested in is docs about how to _use_ the feature. Javadocs are for developers, not users. The refactoring is nice, but the new "tiered compaction" policy looks quite advanced, and introduces a new configuration file, which isn't obvious how to configure. > port HBASE-6371 tier-based compaction from 0.89-fb to trunk > --- > > Key: HBASE-7055 > URL: https://issues.apache.org/jira/browse/HBASE-7055 > Project: HBase > Issue Type: Task > Components: Compaction >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.96.0 > > Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch > > > There's divergence in the code :( > See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5974) Scanner retry behavior with RPC timeout on next() seems incorrect
[ https://issues.apache.org/jira/browse/HBASE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487494#comment-13487494 ] stack commented on HBASE-5974: -- [~ted_yu] The above makes no sense. I think you are trying to find a violation of commit protocol but there is none. Go reread what we all agreed. > Scanner retry behavior with RPC timeout on next() seems incorrect > - > > Key: HBASE-5974 > URL: https://issues.apache.org/jira/browse/HBASE-5974 > Project: HBase > Issue Type: Bug > Components: Client, regionserver >Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0 >Reporter: Todd Lipcon >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.96.0 > > Attachments: 5974_94-V4.patch, 5974_trunk.patch, 5974_trunk-V2.patch, > 5974_trunk-V3.patch, HBASE-5974_0.94.patch, HBASE-5974_94-V2.patch, > HBASE-5974_94-V3.patch > > > I'm seeing the following behavior: > - set RPC timeout to a short value > - call next() for some batch of rows, big enough so the client times out > before the result is returned > - the HConnectionManager stuff will retry the next() call to the same server. > At this point, one of two things can happen: 1) the previous next() call will > still be processing, in which case you get a LeaseException, because it was > removed from the map during the processing, or 2) the next() call will > succeed but skip the prior batch of rows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5974) Scanner retry behavior with RPC timeout on next() seems incorrect
[ https://issues.apache.org/jira/browse/HBASE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487488#comment-13487488 ] Ted Yu commented on HBASE-5974: --- On 01/Jul/12 08:25, Anoop said: bq. I had added my reasoning ... Any thoughts? There was a long period of inactivity till 18/Oct/12 when Stack picked this up. I don't see a second +1 since 18/Oct/12. Maybe I missed something. > Scanner retry behavior with RPC timeout on next() seems incorrect > - > > Key: HBASE-5974 > URL: https://issues.apache.org/jira/browse/HBASE-5974 > Project: HBase > Issue Type: Bug > Components: Client, regionserver >Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0 >Reporter: Todd Lipcon >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.96.0 > > Attachments: 5974_94-V4.patch, 5974_trunk.patch, 5974_trunk-V2.patch, > 5974_trunk-V3.patch, HBASE-5974_0.94.patch, HBASE-5974_94-V2.patch, > HBASE-5974_94-V3.patch > > > I'm seeing the following behavior: > - set RPC timeout to a short value > - call next() for some batch of rows, big enough so the client times out > before the result is returned > - the HConnectionManager stuff will retry the next() call to the same server. > At this point, one of two things can happen: 1) the previous next() call will > still be processing, in which case you get a LeaseException, because it was > removed from the map during the processing, or 2) the next() call will > succeed but skip the prior batch of rows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6410) Move RegionServer Metrics to metrics2
[ https://issues.apache.org/jira/browse/HBASE-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487478#comment-13487478 ] stack commented on HBASE-6410: -- Here are some comments on pages 8 through 11 or so: MetricsRegion needs class doc. What is it? Utility? Glue between Region Source and HRegion? Is it needed? 37 private MetricsRegionServerSource generalSource; data member name should be serverSource? Ditto on MetricsRegionServer. Is it glue between RegionServer and metrics? 35 Is it only for JMX? (See below) * Impl for exposing HRegionServer Information through JMX 36 */ 37 @InterfaceAudience.Private class MetricsRegionServerWrapperImpl Then above line is formatted strange. Fix this '246 * This is the runnable that will be executed on the executor ever PERIOD number of seconds' Just remove TestExponentiallyDecayingSample.java? And the other deprecated tests? > Move RegionServer Metrics to metrics2 > - > > Key: HBASE-6410 > URL: https://issues.apache.org/jira/browse/HBASE-6410 > Project: HBase > Issue Type: Sub-task > Components: metrics >Affects Versions: 0.96.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Blocker > Attachments: HBASE-6410-13.patch, HBASE-6410-1.patch, > HBASE-6410-2.patch, HBASE-6410-3.patch, HBASE-6410-4.patch, > HBASE-6410-5.patch, HBASE-6410-6.patch, HBASE-6410.patch > > > Move RegionServer Metrics to metrics2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487462#comment-13487462 ] Ted Yu commented on HBASE-3909: --- Excerpt from http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ConcurrentHashMap.html : bq. Like Hashtable but unlike HashMap, this class does not allow null to be used as a key or value. So the user is not able to specify null value. I think my suggestion is safe. > Add dynamic config > -- > > Key: HBASE-3909 > URL: https://issues.apache.org/jira/browse/HBASE-3909 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Subbu M Iyer > Fix For: 0.96.0 > > Attachments: 3909_090712-2.patch, 3909-102812.patch, > 3909-102912.patch, 3909.v1, 3909-v1.patch, HBase Cluster Config Details.xlsx, > patch-v2.patch, testMasterNoCluster.stack > > > I'm sure this issue exists already, at least as part of the discussion around > making online schema edits possible, but no hard this having its own issue. > Ted started a conversation on this topic up on dev and Todd suggested we > lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reopened HBASE-7055: --- @Sergey: I put in very short release notes. I think what reviewers were asking is better documentation for release notes. > port HBASE-6371 tier-based compaction from 0.89-fb to trunk > --- > > Key: HBASE-7055 > URL: https://issues.apache.org/jira/browse/HBASE-7055 > Project: HBase > Issue Type: Task > Components: Compaction >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.96.0 > > Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch > > > There's divergence in the code :( > See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6371) [89-fb] Tier based compaction
[ https://issues.apache.org/jira/browse/HBASE-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487458#comment-13487458 ] Hudson commented on HBASE-6371: --- Integrated in HBase-TRUNK #3499 (See [https://builds.apache.org/job/HBase-TRUNK/3499/]) HBASE-7055 port HBASE-6371 tier-based compaction from 0.89-fb to trunk - revert for further discussion (Revision 1403890) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseConfiguration.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionConfiguration.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TierCompactionConfiguration.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TierCompactionManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactSelection.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java * /hbase/trunk/hbase-server/src/main/resources/hbase-compactions.xml * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultCompactSelection.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTierCompactSelection.java > [89-fb] Tier based compaction > - > > Key: HBASE-6371 > URL: https://issues.apache.org/jira/browse/HBASE-6371 > Project: HBase > Issue Type: Improvement >Reporter: Akashnil >Assignee: Liyin Tang > Labels: noob > Attachments: HBASE-6371-089fb-commit.patch > > > Currently, the compaction selection is not very flexible and is not sensitive > to the hotness of the data. Very old data is likely to be accessed less, and > very recent data is likely to be in the block cache. Both of these > considerations make it inefficient to compact these files as aggressively as > other files. In some use-cases, the access-pattern is particularly obvious > even though there is no way to control the compaction algorithm in those > cases. > In the new compaction selection algorithm, we plan to divide the candidate > files into different levels according to oldness of the data that is present > in those files. For each level, parameters like compaction ratio, minimum > number of store-files in each compaction may be different. Number of levels, > time-ranges, and parameters for each level will be configurable online on a > per-column family basis. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487457#comment-13487457 ] Hudson commented on HBASE-7055: --- Integrated in HBase-TRUNK #3499 (See [https://builds.apache.org/job/HBase-TRUNK/3499/]) HBASE-7055 Revert - part 2 (Revision 1403898) HBASE-7055 port HBASE-6371 tier-based compaction from 0.89-fb to trunk - revert for further discussion (Revision 1403890) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java tedyu : Files : * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseConfiguration.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionConfiguration.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TierCompactionConfiguration.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TierCompactionManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactSelection.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java * /hbase/trunk/hbase-server/src/main/resources/hbase-compactions.xml * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultCompactSelection.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTierCompactSelection.java > port HBASE-6371 tier-based compaction from 0.89-fb to trunk > --- > > Key: HBASE-7055 > URL: https://issues.apache.org/jira/browse/HBASE-7055 > Project: HBase > Issue Type: Task > Components: Compaction >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.96.0 > > Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch > > > There's divergence in the code :( > See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations
[ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487454#comment-13487454 ] Lars Hofhansl commented on HBASE-4583: -- Another interesting thought: If in addition to counting all the KVs older than the readpoint I also count all versions of the KVs seen so far we can use this for "memstore compactions". I.e. each memstore change would be an upsert that would add the new KVs and then remove all KVs that not be flushed to disk anyway. > Integrate RWCC with Append and Increment operations > --- > > Key: HBASE-4583 > URL: https://issues.apache.org/jira/browse/HBASE-4583 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.96.0 > > Attachments: 4583-trunk-less-radical.txt, > 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, > 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, > 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, > 4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt > > > Currently Increment and Append operations do not work with RWCC and hence a > client could see the results of multiple such operation mixed in the same > Get/Scan. > The semantics might be a bit more interesting here as upsert adds and removes > to and from the memstore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7070) Scanner may retry forever after HBASE-5974
[ https://issues.apache.org/jira/browse/HBASE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487445#comment-13487445 ] chunhui shen commented on HBASE-7070: - bq.The next() call also can throw NSRE Yes, it will throw NSRE, then we will reset scanner, and then openscanner() throws NSRE, so it won't happen that next() throws infinite NSRE. > Scanner may retry forever after HBASE-5974 > --- > > Key: HBASE-7070 > URL: https://issues.apache.org/jira/browse/HBASE-7070 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-7070.patch > > > After the patch of HBASE-5974 > Suppose The process is as the following: > 1.A next request is very large, so first time it is failed because of timeout > 2.The second time it is failed again because of > CallSequenceOutOfOrderException > 3.We will retry this next request again after reset scanner, so it is also > very large and failed again because of timeout > 4.CallSequenceOutOfOrderException again > 5.Repeated the above forever... > See the comment in HBASE-5974 for more > https://issues.apache.org/jira/browse/HBASE-5974?focusedCommentId=13486658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13486658 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6966) "Compressed RPCs for HBase" (HBASE-5355) port to trunk
[
https://issues.apache.org/jira/browse/HBASE-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487441#comment-13487441
]
Hadoop QA commented on HBASE-6966:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551466/6966-v1.1.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3193//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3193//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3193//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3193//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3193//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3193//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3193//console
This message is automatically generated.
> "Compressed RPCs for HBase" (HBASE-5355) port to trunk
> --
>
> Key: HBASE-6966
> URL: https://issues.apache.org/jira/browse/HBASE-6966
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC
>Reporter: Devaraj Das
>Assignee: Devaraj Das
> Fix For: 0.96.0
>
> Attachments: 6966-1.patch, 6966-v1.1.txt, 6966-v2.txt
>
>
> This jira will address the port of the compressed RPC implementation to
> trunk. I am expecting the patch to be significantly different due to the PB
> stuff in trunk, and hence filed a separate jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations
[ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487442#comment-13487442 ] Lars Hofhansl commented on HBASE-4583: -- #1 Yeah, old comment. #2 Should refactor the tests to call Upsert on Store and Memstore directly rather than this - now defunct - method. The patch at least removes it from the public API on both cases. #3 I should add a test for Append. Bit more tricky than Increment as you get ever increasing values (in terms of size) #4 the test validates that. TestAtomicOperation would not return correct results without the Increments waiting for all previous changes. (before this patch this was handled by forcing the memTSs of all KV to 0 immediately) #5 I'm fine either here, actually. Definitely 0.96, +0 on 0.94. > Integrate RWCC with Append and Increment operations > --- > > Key: HBASE-4583 > URL: https://issues.apache.org/jira/browse/HBASE-4583 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.96.0 > > Attachments: 4583-trunk-less-radical.txt, > 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, > 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, > 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, > 4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt > > > Currently Increment and Append operations do not work with RWCC and hence a > client could see the results of multiple such operation mixed in the same > Get/Scan. > The semantics might be a bit more interesting here as upsert adds and removes > to and from the memstore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487440#comment-13487440 ] Sergey Shelukhin commented on HBASE-7055: - With regard to documentation - you mean better commit description, or something like javadocs? Most of the patch (e.g. CompactionManager) is refactoring of the compaction code out of (H)Store, so the code as such as old, except for the alternative CompactionManager. With regard to 2nd +1 - someone should review :) > port HBASE-6371 tier-based compaction from 0.89-fb to trunk > --- > > Key: HBASE-7055 > URL: https://issues.apache.org/jira/browse/HBASE-7055 > Project: HBase > Issue Type: Task > Components: Compaction >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.96.0 > > Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch > > > There's divergence in the code :( > See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations
[
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487439#comment-13487439
]
Hadoop QA commented on HBASE-4583:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551462/4583-trunk-less-radical-v5.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3192//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3192//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3192//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3192//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3192//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3192//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3192//console
This message is automatically generated.
> Integrate RWCC with Append and Increment operations
> ---
>
> Key: HBASE-4583
> URL: https://issues.apache.org/jira/browse/HBASE-4583
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 4583-trunk-less-radical.txt,
> 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt,
> 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt,
> 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt,
> 4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt
>
>
> Currently Increment and Append operations do not work with RWCC and hence a
> client could see the results of multiple such operation mixed in the same
> Get/Scan.
> The semantics might be a bit more interesting here as upsert adds and removes
> to and from the memstore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487432#comment-13487432 ] Subbu M Iyer commented on HBASE-3909: - call to containsKey(). If null is returned from get(), we delegate to super.get(). This applies to getLong(), etc. This is tricky. Basically my thought is it's quite possible that some one has overridden a configuration key with null value explicitly and in such cases we do not want to override the same with default value from config. So, if the key is present in CHM, then we must use it no matter what the value is. thoughts ? On Mon, Oct 29, 2012 at 10:12 AM, Ted Yu (JIRA) wrote: > Add dynamic config > -- > > Key: HBASE-3909 > URL: https://issues.apache.org/jira/browse/HBASE-3909 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Subbu M Iyer > Fix For: 0.96.0 > > Attachments: 3909_090712-2.patch, 3909-102812.patch, > 3909-102912.patch, 3909.v1, 3909-v1.patch, HBase Cluster Config Details.xlsx, > patch-v2.patch, testMasterNoCluster.stack > > > I'm sure this issue exists already, at least as part of the discussion around > making online schema edits possible, but no hard this having its own issue. > Ted started a conversation on this topic up on dev and Todd suggested we > lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487431#comment-13487431 ] Subbu M Iyer commented on HBASE-3909: - Ted, Addressed all your comments except the support for having more than one pair of values for the update request. My thought is we shall definitely add that as a additional feature once we commit this one. Also updated the latest patch to review board. thanks a lot for your cooperation on this. Subbu > Add dynamic config > -- > > Key: HBASE-3909 > URL: https://issues.apache.org/jira/browse/HBASE-3909 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Subbu M Iyer > Fix For: 0.96.0 > > Attachments: 3909_090712-2.patch, 3909-102812.patch, > 3909-102912.patch, 3909.v1, 3909-v1.patch, HBase Cluster Config Details.xlsx, > patch-v2.patch, testMasterNoCluster.stack > > > I'm sure this issue exists already, at least as part of the discussion around > making online schema edits possible, but no hard this having its own issue. > Ted started a conversation on this topic up on dev and Todd suggested we > lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7078) Add a test that append is atomic
Gregory Chanan created HBASE-7078: - Summary: Add a test that append is atomic Key: HBASE-7078 URL: https://issues.apache.org/jira/browse/HBASE-7078 Project: HBase Issue Type: Sub-task Reporter: Gregory Chanan HBASE-4583 adds a test that increment is atomic; we should add one for append as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7077) checkAndPut should properly integrate with MVCC
Gregory Chanan created HBASE-7077: - Summary: checkAndPut should properly integrate with MVCC Key: HBASE-7077 URL: https://issues.apache.org/jira/browse/HBASE-7077 Project: HBase Issue Type: Sub-task Reporter: Gregory Chanan Fix For: 0.96.0 checkAndPut should integrate with MVCC, similar to how HBASE-4583 fixed appends and increments. Also need a test, here's one we could use (originally proposed in HBASE-7051): The current value of some cell is 10. I issue two concurrent requests: A) a check and put where check value = 10, put value = 11 B) a put where put value = 50 The only result at the end of these operations that seems reasonable to me is the value of the cell being 50. If A occurred first (ACID wise), then our values go 10->11->50. If B occurred first, then our values go 10->50 (and the checkAndPut fails) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7076) Add test that increment/append properly integrate with MVCC
Gregory Chanan created HBASE-7076: - Summary: Add test that increment/append properly integrate with MVCC Key: HBASE-7076 URL: https://issues.apache.org/jira/browse/HBASE-7076 Project: HBase Issue Type: Sub-task Components: regionserver, Transactions/MVCC Affects Versions: 0.94.0, 0.96.0 Reporter: Gregory Chanan Fix For: 0.94.3, 0.96.0 We should test that increment/append properly integrate with MVCC. HBASE-4583 fixes the issue for increment/append. Here is a run through of the tests: increment: current cell value is 10. Issue concurrently a put of 50 and an increment by 1. Only acceptable values are 50 and 51 (not 11). append: current cell value is 'x'. Issue concurrently a put of 'y' and an append of 'x'. Only acceptable values after are 'y' and 'yx' (not 'xx'). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6904) In the HBase shell, an error is thrown that states replication-related znodes already exist
[
https://issues.apache.org/jira/browse/HBASE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487419#comment-13487419
]
Hadoop QA commented on HBASE-6904:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551450/HBASE-6904.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
org.apache.hadoop.hbase.TestRegionRebalancing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3191//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3191//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3191//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3191//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3191//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3191//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3191//console
This message is automatically generated.
> In the HBase shell, an error is thrown that states replication-related znodes
> already exist
> ---
>
> Key: HBASE-6904
> URL: https://issues.apache.org/jira/browse/HBASE-6904
> Project: HBase
> Issue Type: Bug
> Components: Replication, Zookeeper
>Affects Versions: 0.92.0, 0.92.1
>Reporter: Aleksandr Shulman
>Assignee: Gregory Chanan
>Priority: Minor
> Fix For: 0.94.3
>
> Attachments: HBASE-6904.patch
>
>
> On a replication-enabled cluster, querying the list_peers produces the error
> lines shown below. It doesn't appear that anything is broken in terms of
> functionality.
> Stack trace:
> hbase(main):001:0> list_peers
> 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node
> /hbase/replication/peers already exists and this is not a retry
> 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node
> /hbase/replication/rs already exists and this is not a retry
> PEER ID CLUSTER KEY
> 0 row(s) in 0.4650 seconds
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations
[
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487414#comment-13487414
]
Gregory Chanan commented on HBASE-4583:
---
#1
{code}// only remove Puts that concurrent scanners cannot possibly see
(readpoint - 1){code}
the "readpoint -1" stuff is out of date I think.
#2 I'm fine with updateColumnValue if it's only used in tests and the tests
pass.
#3 I'll file some JIRAs for adding a test for append (similar to your test for
increment) and for doing what you have done here for checkAndPut. checkAndPut
should be simple because I don't think it upserts.
#4 We may also need to a test to check that append/increment properly wait for
all writes to complete. Your test, as best as I can tell, checks that the
upsert behaves properly with scanners, but the waiting for all writes to
complete is sort of a separate issue. Something like what I suggested over in
HBASE-7051:
current cell value is 10. Do a put of 15 and an increment by 1. Only
acceptable answers are 15 and 16, but without this patch, we can get 11.
Again, this can be handled in a separate JIRA.
#5 I'm fine with this for 0.96. What do you think about 0.94? JD makes a good
point that we are far along to have such a big change, but on the other hand,
atomic issues are pretty serious. Wondering what your thoughts are. I'd at
least want to do some of my own performance testing before putting it in 0.94.
> Integrate RWCC with Append and Increment operations
> ---
>
> Key: HBASE-4583
> URL: https://issues.apache.org/jira/browse/HBASE-4583
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 4583-trunk-less-radical.txt,
> 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt,
> 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt,
> 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt,
> 4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt
>
>
> Currently Increment and Append operations do not work with RWCC and hence a
> client could see the results of multiple such operation mixed in the same
> Get/Scan.
> The semantics might be a bit more interesting here as upsert adds and removes
> to and from the memstore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6353) Snapshots shell
[ https://issues.apache.org/jira/browse/HBASE-6353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487409#comment-13487409 ] Jesse Yates commented on HBASE-6353: Yeah. Matteo - do you want to attach the latest version and I'll roll it in? > Snapshots shell > --- > > Key: HBASE-6353 > URL: https://issues.apache.org/jira/browse/HBASE-6353 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Attachments: HBASE-6353-v0.patch > > > h6. hbase shell with snapshot commands > * snapshot > ** Take a snapshot of the specified name with the specified name > * restore_snapshot > ** Restore specified snapshot on the original table > * mount_snapshot [readonly] > ** Load the snapshot data as specified table (optional readonly flag) > * list_snapshots [filter] > ** Show a list of snapshots > * delete_snapshot > ** Remove a specified snapshot > h6. Restore Table > Given a "snapshot name" restore override the original table with the snapshot > content. > Before restoring a new snapshot of the table is taken, just to avoid bad > situations. > (If the table is not disabled we can keep serving reads) > This allows a full and quick rollback to a previous snapshot. > h6. Mount Table (Aka Clone Table) > Given a "snapshot name" a new table is created with the content of the > specified snapshot. > This operation allows: > * To have an old version of the table in parallel with the current one. > ** Look at snapshot side-by-side with the "current" before making the > decision whether to roll back or not > * To Restore only "individual items" (only some small range of data was lost > from "current") > ** MR job that scan the cloned table and update the data in the original > one. (Partial restore of the data) > * if the table is not marked as read-only > ** To Add/Remove data from this table without affecting the original one or > the snapshot. > h6. Open points > * Add snapshot type option on take snapshot command (global, timestamp)? > * Keep separate the "restore" from "mount"? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations
[
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487408#comment-13487408
]
Hadoop QA commented on HBASE-4583:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551452/4583-trunk-less-radical-v4.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestMultiParallel
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.client.TestFromClientSide
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
org.apache.hadoop.hbase.regionserver.TestHRegion
org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3190//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3190//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3190//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3190//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3190//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3190//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3190//console
This message is automatically generated.
> Integrate RWCC with Append and Increment operations
> ---
>
> Key: HBASE-4583
> URL: https://issues.apache.org/jira/browse/HBASE-4583
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 4583-trunk-less-radical.txt,
> 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt,
> 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt,
> 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt,
> 4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt
>
>
> Currently Increment and Append operations do not work with RWCC and hence a
> client could see the results of multiple such operation mixed in the same
> Get/Scan.
> The semantics might be a bit more interesting here as upsert adds and removes
> to and from the memstore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2629) Piggyback basic "alarm" framework on RS heartbeats
[ https://issues.apache.org/jira/browse/HBASE-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HBASE-2629. Resolution: Won't Fix Upon reflection a couple years later, I think we should just make sure to emit useful WARN logs for cases like this. Existing cluster monitoring systems (eg splunk or CM) are better suited to surface these logs to users than anything w could build ourselves inside hbase > Piggyback basic "alarm" framework on RS heartbeats > -- > > Key: HBASE-2629 > URL: https://issues.apache.org/jira/browse/HBASE-2629 > Project: HBase > Issue Type: New Feature > Components: master, regionserver >Reporter: Todd Lipcon > > There are a number of system conditions that can cause HBase to perform badly > or have stability issues. For example, significant swapping activity or > overloaded ZK will result in all kinds of problems. > It would be nice to put a very lightweight "alarm" framework in place, so > that when the RS notices something is amiss, it can raise an alarm flag for > some period of time. These could be exposed by JMX to external monitoring > tools, and also displayed on the master web UI. > Some example alarms: > - "ZK read took >1000ms" > - "Long garbage collection pause detected" > - "Writes blocked on region for longer than 5 seconds" > etc etc -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations
[ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487407#comment-13487407 ] Lars Hofhansl commented on HBASE-4583: -- @Ted: I was thinking about that. It would have meant to put mvcc.beginMemstoreInsert up and out of the first try/catch block, and since we have to do this after the lock was acquired, we'd have to move that up too. > Integrate RWCC with Append and Increment operations > --- > > Key: HBASE-4583 > URL: https://issues.apache.org/jira/browse/HBASE-4583 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.96.0 > > Attachments: 4583-trunk-less-radical.txt, > 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, > 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, > 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, > 4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt > > > Currently Increment and Append operations do not work with RWCC and hence a > client could see the results of multiple such operation mixed in the same > Get/Scan. > The semantics might be a bit more interesting here as upsert adds and removes > to and from the memstore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2444) LeaseStillHeldException is overloaded for other meanings in RS management
[ https://issues.apache.org/jira/browse/HBASE-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HBASE-2444. Resolution: Not A Problem no longer a problem - this exception is now only used by scanner lease management, not overloaded in master/rs > LeaseStillHeldException is overloaded for other meanings in RS management > - > > Key: HBASE-2444 > URL: https://issues.apache.org/jira/browse/HBASE-2444 > Project: HBase > Issue Type: Improvement > Components: master, regionserver >Reporter: Todd Lipcon >Priority: Minor > > If a region server that has already been declared dead reports to the master, > the master throws LeaseStillHeldException. This is not a very descriptive > exception for this case - we should either add a new exception for this > purpose, or make a general exception like RegionServerStateException and use > a descriptive message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6353) Snapshots shell
[ https://issues.apache.org/jira/browse/HBASE-6353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487401#comment-13487401 ] Jonathan Hsieh commented on HBASE-6353: --- Hey Jesse, I believe this got all +1's on the review. Can we commit this to the snapshot branch so this isn't hanging around anymore? https://reviews.apache.org/r/7583/ > Snapshots shell > --- > > Key: HBASE-6353 > URL: https://issues.apache.org/jira/browse/HBASE-6353 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Attachments: HBASE-6353-v0.patch > > > h6. hbase shell with snapshot commands > * snapshot > ** Take a snapshot of the specified name with the specified name > * restore_snapshot > ** Restore specified snapshot on the original table > * mount_snapshot [readonly] > ** Load the snapshot data as specified table (optional readonly flag) > * list_snapshots [filter] > ** Show a list of snapshots > * delete_snapshot > ** Remove a specified snapshot > h6. Restore Table > Given a "snapshot name" restore override the original table with the snapshot > content. > Before restoring a new snapshot of the table is taken, just to avoid bad > situations. > (If the table is not disabled we can keep serving reads) > This allows a full and quick rollback to a previous snapshot. > h6. Mount Table (Aka Clone Table) > Given a "snapshot name" a new table is created with the content of the > specified snapshot. > This operation allows: > * To have an old version of the table in parallel with the current one. > ** Look at snapshot side-by-side with the "current" before making the > decision whether to roll back or not > * To Restore only "individual items" (only some small range of data was lost > from "current") > ** MR job that scan the cloned table and update the data in the original > one. (Partial restore of the data) > * if the table is not marked as read-only > ** To Add/Remove data from this table without affecting the original one or > the snapshot. > h6. Open points > * Add snapshot type option on take snapshot command (global, timestamp)? > * Keep separate the "restore" from "mount"? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2363) Add configuration for tunable dataloss sensitivity
[ https://issues.apache.org/jira/browse/HBASE-2363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HBASE-2363. Resolution: Duplicate this has been somewhat addressed by hbase.skip.errors and hbase.hlog.split.skip.errors, implemented elsewhere > Add configuration for tunable dataloss sensitivity > -- > > Key: HBASE-2363 > URL: https://issues.apache.org/jira/browse/HBASE-2363 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.20.3, 0.90.0 >Reporter: Todd Lipcon > > There are many cases in HBase when the process detects that some data is "not > quite right". We often have two choices: (a) throw up our hands and exit, or > (b) log a warning and push through the issue, potentially losing data. I > think some users are very sensitive to dataloss and would prefer that the > system become unavailable rather than continue with potentially lost data. > Other users just want the system to stay up, and if they lose a log segment > it's fine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6777) Snapshot Restore interface
[ https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487399#comment-13487399 ] Jonathan Hsieh commented on HBASE-6777: --- Hey guys, the patch up on review board has been hanging around for a few weeks and seems to be waiting for some action. Who's turn is it to update it or commit it? Jesse, Matteo, is it good to commit on the snapshot branch? > Snapshot Restore interface > -- > > Key: HBASE-6777 > URL: https://issues.apache.org/jira/browse/HBASE-6777 > Project: HBase > Issue Type: Sub-task > Components: Client, master, snapshots >Affects Versions: 0.96.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Fix For: hbase-6055 > > > Add interfaces for restoring a snapshot -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations
[
https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487393#comment-13487393
]
Ted Yu commented on HBASE-4583:
---
Putting the patch on review board, I was able to understand the changes better
:-)
completeMemstoreInsert() doesn't throw exception:
{code}
public void completeMemstoreInsert(WriteEntry e) {
{code}
I wonder if the last mvcc.completeMemstoreInsert(w) call can be latched onto
(before) closeRegionOperation():
{code}
+mvcc.completeMemstoreInsert(w);
}
- if (writeToWAL) {
-syncOrDefer(txid); // sync the transaction log outside the rowlock
- }
} finally {
closeRegionOperation();
{code}
This way there is no need to add one more try / catch block.
There are also some white spaces introduced in patch v5.
> Integrate RWCC with Append and Increment operations
> ---
>
> Key: HBASE-4583
> URL: https://issues.apache.org/jira/browse/HBASE-4583
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 4583-trunk-less-radical.txt,
> 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt,
> 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt,
> 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt,
> 4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt
>
>
> Currently Increment and Append operations do not work with RWCC and hence a
> client could see the results of multiple such operation mixed in the same
> Get/Scan.
> The semantics might be a bit more interesting here as upsert adds and removes
> to and from the memstore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6966) "Compressed RPCs for HBase" (HBASE-5355) port to trunk
[ https://issues.apache.org/jira/browse/HBASE-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-6966: --- Attachment: 6966-v1.1.txt Patch that addresses [~enis]'s comment on the exception handling. > "Compressed RPCs for HBase" (HBASE-5355) port to trunk > -- > > Key: HBASE-6966 > URL: https://issues.apache.org/jira/browse/HBASE-6966 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.96.0 > > Attachments: 6966-1.patch, 6966-v1.1.txt, 6966-v2.txt > > > This jira will address the port of the compressed RPC implementation to > trunk. I am expecting the patch to be significantly different due to the PB > stuff in trunk, and hence filed a separate jira. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4583) Integrate RWCC with Append and Increment operations
[ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4583: - Attachment: 4583-trunk-less-radical-v5.txt Ah crap... I'm tired today. Fixed. > Integrate RWCC with Append and Increment operations > --- > > Key: HBASE-4583 > URL: https://issues.apache.org/jira/browse/HBASE-4583 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.96.0 > > Attachments: 4583-trunk-less-radical.txt, > 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, > 4583-trunk-less-radical-v4.txt, 4583-trunk-less-radical-v5.txt, > 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, > 4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt > > > Currently Increment and Append operations do not work with RWCC and hence a > client could see the results of multiple such operation mixed in the same > Get/Scan. > The semantics might be a bit more interesting here as upsert adds and removes > to and from the memstore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations
[ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487382#comment-13487382 ] Gregory Chanan commented on HBASE-4583: --- append looks like it calls completeMemstoreInsert twice (once before sync, once after) > Integrate RWCC with Append and Increment operations > --- > > Key: HBASE-4583 > URL: https://issues.apache.org/jira/browse/HBASE-4583 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.96.0 > > Attachments: 4583-trunk-less-radical.txt, > 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, > 4583-trunk-less-radical-v4.txt, 4583-trunk-radical.txt, > 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, > 4583-v3.txt, 4583-v4.txt > > > Currently Increment and Append operations do not work with RWCC and hence a > client could see the results of multiple such operation mixed in the same > Get/Scan. > The semantics might be a bit more interesting here as upsert adds and removes > to and from the memstore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6775) Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix
[ https://issues.apache.org/jira/browse/HBASE-6775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487378#comment-13487378 ] Gregory Chanan commented on HBASE-6775: --- I think config is the best option. It should only be temporary (gone in 0.96), so it's not making things any more cryptic long term. The basic problem is ZOOKEEPER-1495; you can't issue a command and figure out quickly whether the cluster supports multi or not. You can try to figure this out yourself, using "srvr", but it's tricky. You need to check that every node on the cluster supports multi. Apparently just figuring out what servers are in the cluster is complicated (I was told jmx is the best way); the zk configuration in hbase-land is not sufficient because it may not list every node. It is not sufficient to just check for a quorum of servers that support it, because the servers that don't support it will be kicked (remove themselves?) from the quorum and you'll lose your fault tolerance. Even if you know every server in the cluster supports multi at some point in time, that may change by the time you actually issue your command. A new server may join or a server version may be downgraded by the time you actually send your command. So there's no good way to do this. Any objection to going with the Configuration parameter, Stack or others? (config parameter would be called hbase.zookeeper.useMulti). > Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix > -- > > Key: HBASE-6775 > URL: https://issues.apache.org/jira/browse/HBASE-6775 > Project: HBase > Issue Type: Improvement > Components: Zookeeper >Affects Versions: 0.94.2 >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.94.4 > > > HBASE-6710 fixed a 0.92/0.94 compatibility issue by writing two znodes in > different formats. > If a ZK failure occurs between the writing of the two znodes, strange > behavior can result. > This issue is a reminder to change these two ZK writes to use ZK.multi when > we require ZK 3.4+. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6826) [WINDOWS] TestFromClientSide failures
[
https://issues.apache.org/jira/browse/HBASE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487372#comment-13487372
]
Hadoop QA commented on HBASE-6826:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551427/hbase-6826_v2-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3189//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3189//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3189//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3189//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3189//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3189//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3189//console
This message is automatically generated.
> [WINDOWS] TestFromClientSide failures
> -
>
> Key: HBASE-6826
> URL: https://issues.apache.org/jira/browse/HBASE-6826
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.3, 0.96.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Labels: windows
> Attachments: hbase-6826_v1-0.94.patch, hbase-6826_v1-trunk.patch,
> hbase-6826_v2-0.94.patch, hbase-6826_v2-trunk.patch
>
>
> The following tests fail for TestFromClientSide:
> {code}
> testPoolBehavior()
> testClientPoolRoundRobin()
> testClientPoolThreadLocal()
> {code}
> The first test fails due to the fact that the test (wrongly) assumes that
> ThredPoolExecutor can reclaim the thread immediately.
> The second and third tests seem to fail because that Put's to the table does
> not specify an explicit timestamp, but on windows, consecutive calls to put
> happen to finish in the same milisecond so that the resulting mutations have
> the same timestamp, thus there is only one version of the cell value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4583) Integrate RWCC with Append and Increment operations
[ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4583: - Fix Version/s: (was: 0.94.3) > Integrate RWCC with Append and Increment operations > --- > > Key: HBASE-4583 > URL: https://issues.apache.org/jira/browse/HBASE-4583 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.96.0 > > Attachments: 4583-trunk-less-radical.txt, > 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, > 4583-trunk-less-radical-v4.txt, 4583-trunk-radical.txt, > 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, > 4583-v3.txt, 4583-v4.txt > > > Currently Increment and Append operations do not work with RWCC and hence a > client could see the results of multiple such operation mixed in the same > Get/Scan. > The semantics might be a bit more interesting here as upsert adds and removes > to and from the memstore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6829) [WINDOWS] Tests should ensure that HLog is closed
[
https://issues.apache.org/jira/browse/HBASE-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487367#comment-13487367
]
Hadoop QA commented on HBASE-6829:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551432/hbase-6829_v3-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3187//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3187//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3187//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3187//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3187//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3187//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3187//console
This message is automatically generated.
> [WINDOWS] Tests should ensure that HLog is closed
> -
>
> Key: HBASE-6829
> URL: https://issues.apache.org/jira/browse/HBASE-6829
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.3, 0.96.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Labels: windows
> Attachments: hbase-6829_v1-0.94.patch, hbase-6829_v1-trunk.patch,
> hbase-6829_v2-0.94.patch, hbase-6829_v2-trunk.patch,
> hbase-6829_v3-0.94.patch, hbase-6829_v3-trunk.patch
>
>
> TestCacheOnWriteInSchema and TestCompactSelection fails with
> {code}
> java.io.IOException: Target HLog directory already exists:
> ./target/test-data/2d814e66-75d3-4c1b-92c7-a49d9972e8fd/TestCacheOnWriteInSchema/logs
> at org.apache.hadoop.hbase.regionserver.wal.HLog.(HLog.java:385)
> at org.apache.hadoop.hbase.regionserver.wal.HLog.(HLog.java:316)
> at
> org.apache.hadoop.hbase.regionserver.TestCacheOnWriteInSchema.setUp(TestCacheOnWriteInSchema.java:162)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4583) Integrate RWCC with Append and Increment operations
[ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4583: - Attachment: 4583-trunk-less-radical-v4.txt Does completeMemstoreInsert after the WAL sync. (good catch Gregory). Unfortunately the only way to do this without more restructuring is another try/catch wrapper. As for updateColumnValue... All the test there make the call directly, there is no region to derive a readpoint from. No not actually deleting anything is the best approach. > Integrate RWCC with Append and Increment operations > --- > > Key: HBASE-4583 > URL: https://issues.apache.org/jira/browse/HBASE-4583 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.96.0 > > Attachments: 4583-trunk-less-radical.txt, > 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, > 4583-trunk-less-radical-v4.txt, 4583-trunk-radical.txt, > 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, 4583.txt, 4583-v2.txt, > 4583-v3.txt, 4583-v4.txt > > > Currently Increment and Append operations do not work with RWCC and hence a > client could see the results of multiple such operation mixed in the same > Get/Scan. > The semantics might be a bit more interesting here as upsert adds and removes > to and from the memstore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6823) [WINDOWS] TestSplitTransaction fails due to the Log handle not released by a call to DaughterOpener.start()
[
https://issues.apache.org/jira/browse/HBASE-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487364#comment-13487364
]
Hadoop QA commented on HBASE-6823:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551430/hbase-6823_v2-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3188//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3188//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3188//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3188//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3188//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3188//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3188//console
This message is automatically generated.
> [WINDOWS] TestSplitTransaction fails due to the Log handle not released by a
> call to DaughterOpener.start()
> ---
>
> Key: HBASE-6823
> URL: https://issues.apache.org/jira/browse/HBASE-6823
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.3, 0.96.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Labels: windows
> Fix For: 0.96.0
>
> Attachments: hbase-6823_v1-0.94.patch, hbase-6823_v1-trunk.patch,
> hbase-6823_v2-0.94.patch, hbase-6823_v2-trunk.patch
>
>
> There are two unit test cases in HBase RegionServer test failed in the clean
> up stage that failed to delete the files/folders created in the test.
> testWholesomeSplit(org.apache.hadoop.hbase.regionserver.TestSplitTransaction):
> Failed delete of ./target/test-
> data/1c386abc-f159-492e-b21f-e89fab24d85b/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/a588d813fd26280c2b42e93565ed960c
> testRollback(org.apache.hadoop.hbase.regionserver.TestSplitTransaction):
> Failed delete of ./target/test-data/6
> 1a1a14b-0cc9-4dd6-93fd-4dc021e2bfcc/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/8090abc89528461fa284288c257662cd
> The root cause is triggered by ta call to the DaughterOpener.start() in
> \src\hbase\src\main\java\org\apache\hadoop\hbase\regionserver\SplitTransactopn.Java
> (openDaughters() function). It left handles to the splited folder/file and
> causing deleting of the file/folder failed in the Windows OS.
> Windows does not allow to delete a file, while there are open file handlers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4583) Integrate RWCC with Append and Increment operations
[ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487359#comment-13487359 ] Lars Hofhansl commented on HBASE-4583: -- @Gregory: updateColumnValue is only used in tests... It should just be removed, but there're tests using it still. And you're right the memstore transaction should not be completed before the WAL is sync'ed. Update coming. > Integrate RWCC with Append and Increment operations > --- > > Key: HBASE-4583 > URL: https://issues.apache.org/jira/browse/HBASE-4583 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.3, 0.96.0 > > Attachments: 4583-trunk-less-radical.txt, > 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, > 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, > 4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt > > > Currently Increment and Append operations do not work with RWCC and hence a > client could see the results of multiple such operation mixed in the same > Get/Scan. > The semantics might be a bit more interesting here as upsert adds and removes > to and from the memstore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6814) [WINDOWS] HBase on Windows
[ https://issues.apache.org/jira/browse/HBASE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487360#comment-13487360 ] Enis Soztutar commented on HBASE-6814: -- We were able to get HBase working locally and the test passing (other than some flaky ones) for 0.94 and trunk under windows. I have uploaded the recent versions of the patches for most of the issues. I think at this point we can go on an start committing the patches to trunk once they are reviewed. There will be other patches later on, but we can work with them one-by-one. I think, once trunk is in a relatively good shape, and we have a final list of patches, we can then decide on whether to merge back into 0.94. For convenience here is a breakdown of the issues: Patch availables: HBASE-6816 (NR) HBASE-6826 (R) ->v1 reviewed, v2 is a rebase HBASE-6822 (R) HBASE-6820 (R) HBASE-6823 (R) -> v1 reviewed, v2 is a rebase HBASE-6827 (R) HBASE-6828 (R) HBASE-6829 (NR) -> v3 not reviewed HBASE-6831 (R) HBASE-6832 (NR) -> v4 (timing issue) HBASE-6830 (NR) -> v5 (pathing issue) HBASE-6821 (NR) -> v2 HBASE-6825 (NR) -> v3 Open issues with patches coming shortly: HBASE-6815 (NR) -> there is a patch (not attached yet) for 0.94, to be ported to trunk. HBASE-6818 (NR) -> (depends on scripts porting, trunk patch not tested yet) Where (R) indicates reviewed, (NR) is not reviewed. I'll apply all the PA patches above, and run the full suite of tests on my linux box to make sure that they are not breaking anything. It would be really helpful if you guys can take a look at (NR) issues, and leave a comment. Once we get all reviewed, I will commit them to trunk. Any suggestions are always welcome of course. > [WINDOWS] HBase on Windows > -- > > Key: HBASE-6814 > URL: https://issues.apache.org/jira/browse/HBASE-6814 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.3, 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Labels: windows > > This is an umbrella jira to support windows as a first class citizen. > The short term goals are: > - Get unit tests passing > - Scripts converted to windows > - Runtime does not depend on cgywin (build can still depend on cygwin for > now) > - Hbase on windows will consume hadoop branch-1-win artifacts. > - Get nightly jenkins build on windows -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6904) In the HBase shell, an error is thrown that states replication-related znodes already exist
[ https://issues.apache.org/jira/browse/HBASE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-6904: -- Fix Version/s: 0.94.3 Assignee: Gregory Chanan Target for 0.94.3 > In the HBase shell, an error is thrown that states replication-related znodes > already exist > --- > > Key: HBASE-6904 > URL: https://issues.apache.org/jira/browse/HBASE-6904 > Project: HBase > Issue Type: Bug > Components: Replication, Zookeeper >Affects Versions: 0.92.0, 0.92.1 >Reporter: Aleksandr Shulman >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.94.3 > > Attachments: HBASE-6904.patch > > > On a replication-enabled cluster, querying the list_peers produces the error > lines shown below. It doesn't appear that anything is broken in terms of > functionality. > Stack trace: > hbase(main):001:0> list_peers > 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node > /hbase/replication/peers already exists and this is not a retry > 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node > /hbase/replication/rs already exists and this is not a retry > PEER ID CLUSTER KEY > 0 row(s) in 0.4650 seconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6823) [WINDOWS] TestSplitTransaction fails due to the Log handle not released by a call to DaughterOpener.start()
[ https://issues.apache.org/jira/browse/HBASE-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6823: -- Fix Version/s: 0.96.0 Hadoop Flags: Reviewed Summary: [WINDOWS] TestSplitTransaction fails due to the Log handle not released by a call to DaughterOpener.start() (was: [WINDOWS] TestSplitTransaction fails due the Log handle is not released by a call to the DaughterOpener.start()) > [WINDOWS] TestSplitTransaction fails due to the Log handle not released by a > call to DaughterOpener.start() > --- > > Key: HBASE-6823 > URL: https://issues.apache.org/jira/browse/HBASE-6823 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.3, 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Labels: windows > Fix For: 0.96.0 > > Attachments: hbase-6823_v1-0.94.patch, hbase-6823_v1-trunk.patch, > hbase-6823_v2-0.94.patch, hbase-6823_v2-trunk.patch > > > There are two unit test cases in HBase RegionServer test failed in the clean > up stage that failed to delete the files/folders created in the test. > testWholesomeSplit(org.apache.hadoop.hbase.regionserver.TestSplitTransaction): > Failed delete of ./target/test- > data/1c386abc-f159-492e-b21f-e89fab24d85b/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/a588d813fd26280c2b42e93565ed960c > testRollback(org.apache.hadoop.hbase.regionserver.TestSplitTransaction): > Failed delete of ./target/test-data/6 > 1a1a14b-0cc9-4dd6-93fd-4dc021e2bfcc/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/8090abc89528461fa284288c257662cd > The root cause is triggered by ta call to the DaughterOpener.start() in > \src\hbase\src\main\java\org\apache\hadoop\hbase\regionserver\SplitTransactopn.Java > (openDaughters() function). It left handles to the splited folder/file and > causing deleting of the file/folder failed in the Windows OS. > Windows does not allow to delete a file, while there are open file handlers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6338) Cache Method in RPC handler
[ https://issues.apache.org/jira/browse/HBASE-6338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487354#comment-13487354 ] Sergey Shelukhin commented on HBASE-6338: - Hi. Is there plan to commit this to .94? > Cache Method in RPC handler > --- > > Key: HBASE-6338 > URL: https://issues.apache.org/jira/browse/HBASE-6338 > Project: HBase > Issue Type: Improvement >Reporter: binlijin > Attachments: HBASE-6338-90-2.patch, HBASE-6338-90.patch, > HBASE-6338-92-2.patch, HBASE-6338-92.patch, HBASE-6338-94-2.patch, > HBASE-6338-94.patch, HBASE-6338-trunk-2.patch, HBASE-6338-trunk.patch > > > Every call in rpc handler a Method will be created, if we cache the method > will improve a little. > I test with 0.90, Average Class.getMethod(String name, Class... > parameterTypes) cost 4780 ns , if we cache it cost 2620 ns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6904) In the HBase shell, an error is thrown that states replication-related znodes already exist
[ https://issues.apache.org/jira/browse/HBASE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-6904: -- Attachment: HBASE-6904.patch Here's a patch that bumps the log message down to info. I tested in the shell and it fixes the issue. > In the HBase shell, an error is thrown that states replication-related znodes > already exist > --- > > Key: HBASE-6904 > URL: https://issues.apache.org/jira/browse/HBASE-6904 > Project: HBase > Issue Type: Bug > Components: Replication, Zookeeper >Affects Versions: 0.92.0, 0.92.1 >Reporter: Aleksandr Shulman >Priority: Minor > Attachments: HBASE-6904.patch > > > On a replication-enabled cluster, querying the list_peers produces the error > lines shown below. It doesn't appear that anything is broken in terms of > functionality. > Stack trace: > hbase(main):001:0> list_peers > 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node > /hbase/replication/peers already exists and this is not a retry > 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node > /hbase/replication/rs already exists and this is not a retry > PEER ID CLUSTER KEY > 0 row(s) in 0.4650 seconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6904) In the HBase shell, an error is thrown that states replication-related znodes already exist
[ https://issues.apache.org/jira/browse/HBASE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-6904: -- Status: Patch Available (was: Open) > In the HBase shell, an error is thrown that states replication-related znodes > already exist > --- > > Key: HBASE-6904 > URL: https://issues.apache.org/jira/browse/HBASE-6904 > Project: HBase > Issue Type: Bug > Components: Replication, Zookeeper >Affects Versions: 0.92.1, 0.92.0 >Reporter: Aleksandr Shulman >Priority: Minor > Attachments: HBASE-6904.patch > > > On a replication-enabled cluster, querying the list_peers produces the error > lines shown below. It doesn't appear that anything is broken in terms of > functionality. > Stack trace: > hbase(main):001:0> list_peers > 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node > /hbase/replication/peers already exists and this is not a retry > 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node > /hbase/replication/rs already exists and this is not a retry > PEER ID CLUSTER KEY > 0 row(s) in 0.4650 seconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6904) In the HBase shell, an error is thrown that states replication-related znodes already exist
[
https://issues.apache.org/jira/browse/HBASE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487353#comment-13487353
]
Gregory Chanan commented on HBASE-6904:
---
The issue is in RecoverableZooKeeper.createNonSequential handling of
NodeExistsException:
{code}
LOG.error("Node " + path + " already exists and this is not a " +
"retry");
throw e;
{code}
The hbase shell reprints log messages at error+ level. This error log is
unnecessary, because the code already throws an exception. The code that calls
it can just decide if it wants to reprint the error message or not by
catching/rethrowing the exception. I looked through the calls, which all
originate in ZKUtil, and I think it's fine to just bump this down to "info"
level. All the calls in ZKUtil either rethrow the exception or have semantics
where getting a NodeExists exception is not a problem.
> In the HBase shell, an error is thrown that states replication-related znodes
> already exist
> ---
>
> Key: HBASE-6904
> URL: https://issues.apache.org/jira/browse/HBASE-6904
> Project: HBase
> Issue Type: Bug
> Components: Replication, Zookeeper
>Affects Versions: 0.92.0, 0.92.1
>Reporter: Aleksandr Shulman
>Priority: Minor
>
> On a replication-enabled cluster, querying the list_peers produces the error
> lines shown below. It doesn't appear that anything is broken in terms of
> functionality.
> Stack trace:
> hbase(main):001:0> list_peers
> 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node
> /hbase/replication/peers already exists and this is not a retry
> 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node
> /hbase/replication/rs already exists and this is not a retry
> PEER ID CLUSTER KEY
> 0 row(s) in 0.4650 seconds
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6829) [WINDOWS] Tests should ensure that HLog is closed
[
https://issues.apache.org/jira/browse/HBASE-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487352#comment-13487352
]
Ted Yu commented on HBASE-6829:
---
{code}
+ public void tearDown() throws IOException {
+region.close();
+hlog.close();
{code}
Should region.close() be enclosed in try / catch block ?
If IOE comes out of region.close(), hlog wouldn't be closed, right ?
> [WINDOWS] Tests should ensure that HLog is closed
> -
>
> Key: HBASE-6829
> URL: https://issues.apache.org/jira/browse/HBASE-6829
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.3, 0.96.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Labels: windows
> Attachments: hbase-6829_v1-0.94.patch, hbase-6829_v1-trunk.patch,
> hbase-6829_v2-0.94.patch, hbase-6829_v2-trunk.patch,
> hbase-6829_v3-0.94.patch, hbase-6829_v3-trunk.patch
>
>
> TestCacheOnWriteInSchema and TestCompactSelection fails with
> {code}
> java.io.IOException: Target HLog directory already exists:
> ./target/test-data/2d814e66-75d3-4c1b-92c7-a49d9972e8fd/TestCacheOnWriteInSchema/logs
> at org.apache.hadoop.hbase.regionserver.wal.HLog.(HLog.java:385)
> at org.apache.hadoop.hbase.regionserver.wal.HLog.(HLog.java:316)
> at
> org.apache.hadoop.hbase.regionserver.TestCacheOnWriteInSchema.setUp(TestCacheOnWriteInSchema.java:162)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4583) Integrate RWCC with Append and Increment operations
[ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4583: - Attachment: 4583-trunk-less-radical-v3.txt This patch is doing the right thing (TM). A KV is removable if it is not the newest one older than that smallest readpoint. I.e. no current scanner sees that KV. > Integrate RWCC with Append and Increment operations > --- > > Key: HBASE-4583 > URL: https://issues.apache.org/jira/browse/HBASE-4583 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.3, 0.96.0 > > Attachments: 4583-trunk-less-radical.txt, > 4583-trunk-less-radical-v2.txt, 4583-trunk-less-radical-v3.txt, > 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt, > 4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt > > > Currently Increment and Append operations do not work with RWCC and hence a > client could see the results of multiple such operation mixed in the same > Get/Scan. > The semantics might be a bit more interesting here as upsert adds and removes > to and from the memstore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
