[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786919#comment-13786919
]
Hudson commented on HBASE-9612:
---
SUCCESS: Integrated in HBase-TRUNK #4593 (See
[https://builds.apache.org/job/HBase-TRUNK/4593/])
HBASE-9612 Ability to batch edits destined to different regions (stack: rev
1529348)
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallScanner.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/PayloadCarryingRpcController.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
*
/hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java
*
/hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java
* /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
* /hbase/trunk/hbase-protocol/README.txt
*
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
*
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java
* /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/RPC.proto
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AnnotationReadingPriorityFunction.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestIPC.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestSnapshotFromMaster.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionBusyWait.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQosFunction.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/rest/TestResourceFilter.java
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612-nico.v14.096.txt, 9612-nico.v14.096.txt, 9612-nico.v14.096.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch,
> 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt,
> 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt,
> broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (
[jira] [Commented] (HBASE-9714) add scan logging to IntegrationTestLoadAndVerify
[
https://issues.apache.org/jira/browse/HBASE-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786917#comment-13786917
]
Hadoop QA commented on HBASE-9714:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12606955/HBASE-9714.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7476//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7476//console
This message is automatically generated.
> add scan logging to IntegrationTestLoadAndVerify
>
>
> Key: HBASE-9714
> URL: https://issues.apache.org/jira/browse/HBASE-9714
> Project: HBase
> Issue Type: Improvement
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Minor
> Attachments: HBASE-9714.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[ https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786906#comment-13786906 ] Hudson commented on HBASE-9612: --- SUCCESS: Integrated in hbase-0.96 #125 (See [https://builds.apache.org/job/hbase-0.96/125/]) HBASE-9612 Ability to batch edits destined to different regions (stack: rev 1529355) * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallScanner.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/PayloadCarryingRpcController.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java * /hbase/branches/0.96/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java * /hbase/branches/0.96/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java * /hbase/branches/0.96/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java * /hbase/branches/0.96/hbase-protocol/README.txt * /hbase/branches/0.96/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/branches/0.96/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/branches/0.96/hbase-protocol/src/main/protobuf/Client.proto * /hbase/branches/0.96/hbase-protocol/src/main/protobuf/RPC.proto * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/QosFunction.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestIPC.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestSnapshotFromMaster.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionBusyWait.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQosFunction.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/rest/TestResourceFilter.java > Ability to batch edits destined to different regions > > > Key: HBASE-9612 > URL: https://issues.apache.org/jira/browse/HBASE-9612 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0 >Reporter: Benoit Sigoure >Assignee: stack >Priority: Blocker > Fix For: 0.98.0, 0.96.0 > > Attachments: > 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt, > 9612-nico.v14.096.txt, 9612-nico.v14.096.txt, 9612-nico.v14.096.txt, > 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch, > 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch, > 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt, > 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt, > broken.patch > > > The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits > destined to different regions. Se
[jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks
[ https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786872#comment-13786872 ] Sergey Shelukhin commented on HBASE-5487: - *of events > Generic framework for Master-coordinated tasks > -- > > Key: HBASE-5487 > URL: https://issues.apache.org/jira/browse/HBASE-5487 > Project: HBase > Issue Type: New Feature > Components: master, regionserver, Zookeeper >Affects Versions: 0.94.0 >Reporter: Mubarak Seyed >Priority: Critical > Attachments: Region management in Master.pdf > > > Need a framework to execute master-coordinated tasks in a fault-tolerant > manner. > Master-coordinated tasks such as online-scheme change and delete-range > (deleting region(s) based on start/end key) can make use of this framework. > The advantages of framework are > 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for > master-coordinated tasks > 2. Ability to abstract the common functions across Master -> ZK and RS -> ZK > 3. Easy to plugin new master-coordinated tasks without adding code to core > components -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks
[ https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786870#comment-13786870 ] Sergey Shelukhin commented on HBASE-5487: - I would like to resurrect this issue (and rename it). It seems that every fix to assignmentmanager introduces 1-3 more maps or lists around it, makes it even more impossible to comprehend and may add more bugs. We have talked a little bit here, and agreed on 2 key points. 1) Region state should be managed in one permanent place with one state machine; no separate and/or transient state machines, no operation-based state machines. 2) New assignment manager should be easily testable by simulating sequences in events. I think my doc above is still reasonably good approximation, but of course we might need to discuss flesh out the details. Ahem. I said new assignment manager, that is because I would like to rename this jira "rewrite assignment manager". Wdyt? [~enis] [~jxiang] [~saint@gmail.com] [~ndimiduk] [~jmhsieh] > Generic framework for Master-coordinated tasks > -- > > Key: HBASE-5487 > URL: https://issues.apache.org/jira/browse/HBASE-5487 > Project: HBase > Issue Type: New Feature > Components: master, regionserver, Zookeeper >Affects Versions: 0.94.0 >Reporter: Mubarak Seyed >Priority: Critical > Attachments: Region management in Master.pdf > > > Need a framework to execute master-coordinated tasks in a fault-tolerant > manner. > Master-coordinated tasks such as online-scheme change and delete-range > (deleting region(s) based on start/end key) can make use of this framework. > The advantages of framework are > 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for > master-coordinated tasks > 2. Ability to abstract the common functions across Master -> ZK and RS -> ZK > 3. Easy to plugin new master-coordinated tasks without adding code to core > components -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9272) A simple parallel, unordered scanner
[
https://issues.apache.org/jira/browse/HBASE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786862#comment-13786862
]
Hadoop QA commented on HBASE-9272:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12606939/9272-trunk.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the trunk's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7475//console
This message is automatically generated.
> A simple parallel, unordered scanner
>
>
> Key: HBASE-9272
> URL: https://issues.apache.org/jira/browse/HBASE-9272
> Project: HBase
> Issue Type: New Feature
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Attachments: 9272-0.94.txt, 9272-0.94-v2.txt, 9272-0.94-v3.txt,
> 9272-0.94-v4.txt, 9272-trunk.txt, ParallelClientScanner.java,
> ParallelClientScanner.java
>
>
> The contract of ClientScanner is to return rows in sort order. That limits
> the order in which region can be scanned.
> I propose a simple ParallelScanner that does not have this requirement and
> queries regions in parallel, return whatever gets returned first.
> This is generally useful for scans that filter a lot of data on the server,
> or in cases where the client can very quickly react to the returned data.
> I have a simple prototype (doesn't do error handling right, and might be a
> bit heavy on the synchronization side - it used a BlockingQueue to hand data
> between the client using the scanner and the threads doing the scanning, it
> also could potentially starve some scanners long enugh to time out at the
> server).
> On the plus side, it's only a 130 lines of code. :)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786852#comment-13786852
]
Hudson commented on HBASE-9612:
---
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #778 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/778/])
HBASE-9612 Ability to batch edits destined to different regions (stack: rev
1529348)
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallScanner.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/PayloadCarryingRpcController.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
*
/hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java
*
/hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java
* /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
* /hbase/trunk/hbase-protocol/README.txt
*
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
*
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java
* /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/RPC.proto
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AnnotationReadingPriorityFunction.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestIPC.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestSnapshotFromMaster.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionBusyWait.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQosFunction.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/rest/TestResourceFilter.java
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612-nico.v14.096.txt, 9612-nico.v14.096.txt, 9612-nico.v14.096.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch,
> 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt,
> 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt,
> broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786853#comment-13786853
]
Hudson commented on HBASE-9687:
---
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #778 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/778/])
HBASE-9687 Addendum shuts down the MiniMapReduceCluster (tedyu: rev 1529304)
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
HBASE-9687 ClassNotFoundException is thrown when ExportSnapshot runs against
hadoop cluster where HBase is not installed on the same node as resourcemanager
(tedyu: rev 1529288)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687.add, 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/
[jira] [Commented] (HBASE-9709) LogReplay throws NPE when no KVs to be replayed in a WALEdit
[
https://issues.apache.org/jira/browse/HBASE-9709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786851#comment-13786851
]
Hudson commented on HBASE-9709:
---
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #778 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/778/])
HBASE-9709: LogReplay throws NPE when no KVs to be replayed in a WALEdit
(jeffreyz: rev 1529247)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
> LogReplay throws NPE when no KVs to be replayed in a WALEdit
>
>
> Key: HBASE-9709
> URL: https://issues.apache.org/jira/browse/HBASE-9709
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9709.patch
>
>
> This is regression from my recent checkin from hbase-9390 below is the
> exception stack:
> {code}
> 2013-10-03 09:34:32,735 ERROR [WriterThread-1] wal.HLogSplitter: Exiting
> thread
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogReplayOutputSink.groupEditsByServer(HLogSplitter.java:1489)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogReplayOutputSink.append(HLogSplitter.java:1368)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.writeBuffer(HLogSplitter.java:847)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.doRun(HLogSplitter.java:839)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.run(HLogSplitter.java:809)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9714) add scan logging to IntegrationTestLoadAndVerify
[ https://issues.apache.org/jira/browse/HBASE-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786848#comment-13786848 ] Ted Yu commented on HBASE-9714: --- +1 > add scan logging to IntegrationTestLoadAndVerify > > > Key: HBASE-9714 > URL: https://issues.apache.org/jira/browse/HBASE-9714 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Minor > Attachments: HBASE-9714.patch > > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HBASE-9714) add scan logging to IntegrationTestLoadAndVerify
Sergey Shelukhin created HBASE-9714: --- Summary: add scan logging to IntegrationTestLoadAndVerify Key: HBASE-9714 URL: https://issues.apache.org/jira/browse/HBASE-9714 Project: HBase Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Attachments: HBASE-9714.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9714) add scan logging to IntegrationTestLoadAndVerify
[ https://issues.apache.org/jira/browse/HBASE-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-9714: Status: Patch Available (was: Open) > add scan logging to IntegrationTestLoadAndVerify > > > Key: HBASE-9714 > URL: https://issues.apache.org/jira/browse/HBASE-9714 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Minor > Attachments: HBASE-9714.patch > > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9714) add scan logging to IntegrationTestLoadAndVerify
[ https://issues.apache.org/jira/browse/HBASE-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-9714: Attachment: HBASE-9714.patch This patch is for 96 branch > add scan logging to IntegrationTestLoadAndVerify > > > Key: HBASE-9714 > URL: https://issues.apache.org/jira/browse/HBASE-9714 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Minor > Attachments: HBASE-9714.patch > > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9713) create integration or unit test for scanner lease expiration
[ https://issues.apache.org/jira/browse/HBASE-9713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786840#comment-13786840 ] Nick Dimiduk commented on HBASE-9713: - Think this is related to HBASE-9582? > create integration or unit test for scanner lease expiration > > > Key: HBASE-9713 > URL: https://issues.apache.org/jira/browse/HBASE-9713 > Project: HBase > Issue Type: Test >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin > > See HBASE-8935. > I see similar issue again in 96 in IntegrationTestLoadAndVerify, it seems > that when scanner lease expires, MR HBase reader may occasionally skip a > bunch of rows. > We need some sort of test to verify it works correctly, some CP or hook may > be necessary to create well-defined lease expiration. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HBASE-9713) create integration or unit test for scanner lease expiration
Sergey Shelukhin created HBASE-9713: --- Summary: create integration or unit test for scanner lease expiration Key: HBASE-9713 URL: https://issues.apache.org/jira/browse/HBASE-9713 Project: HBase Issue Type: Test Affects Versions: 0.96.0 Reporter: Sergey Shelukhin See HBASE-8935. I see similar issue again in 96 in IntegrationTestLoadAndVerify, it seems that when scanner lease expires, MR HBase reader may occasionally skip a bunch of rows. We need some sort of test to verify it works correctly, some CP or hook may be necessary to create well-defined lease expiration. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (HBASE-8589) add "on-demand-only" integration tests
[ https://issues.apache.org/jira/browse/HBASE-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin resolved HBASE-8589. - Resolution: Invalid place for such tests is probably just perf benchmakr > add "on-demand-only" integration tests > -- > > Key: HBASE-8589 > URL: https://issues.apache.org/jira/browse/HBASE-8589 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Priority: Minor > > Sometimes there's need to write an integration test that only makes sense for > large amount of data, cluster, etc. It doesn't make sense to run such test on > minicluster on single machine. We need to add an attribute or something like > that to indicate that, and make sure mvn verify and IntegrationTestDriver do > not run such tests by default, unless name is explicitly specified -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786834#comment-13786834
]
Hudson commented on HBASE-9687:
---
SUCCESS: Integrated in HBase-TRUNK #4592 (See
[https://builds.apache.org/job/HBase-TRUNK/4592/])
HBASE-9687 Addendum shuts down the MiniMapReduceCluster (tedyu: rev 1529304)
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
HBASE-9687 ClassNotFoundException is thrown when ExportSnapshot runs against
hadoop cluster where HBase is not installed on the same node as resourcemanager
(tedyu: rev 1529288)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687.add, 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/09/30 17:23:58 INFO mapreduce.
[jira] [Commented] (HBASE-9711) Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary
[
https://issues.apache.org/jira/browse/HBASE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786822#comment-13786822
]
Hadoop QA commented on HBASE-9711:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12606930/9711-trunk.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestHRegionBusyWait
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7474//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7474//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7474//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7474//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7474//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7474//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7474//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7474//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7474//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7474//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7474//console
This message is automatically generated.
> Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary
> -
>
> Key: HBASE-9711
> URL: https://issues.apache.org/jira/browse/HBASE-9711
> Project: HBase
> Issue Type: Sub-task
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Trivial
> Fix For: 0.98.0, 0.94.13, 0.96.1
>
> Attachments: 9711-trunk.txt
>
>
> Parent patch copies input for RegexFilter unconditionally.
> We should only do this if the KV portion into the passed byte[] is < 1/2 of
> the passed byte[]. Otherwise we waste cycles.
> Patch is trivial and will be coming momentarily.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9688) Fix javadoc warning in HConnectionManager class javadoc
[
https://issues.apache.org/jira/browse/HBASE-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9688:
--
Fix Version/s: 0.96.0
0.98.0
> Fix javadoc warning in HConnectionManager class javadoc
> ---
>
> Key: HBASE-9688
> URL: https://issues.apache.org/jira/browse/HBASE-9688
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9688.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/7422/artifact/trunk/patchprocess/patchJavadocWarnings.txt
> :
> {code}
> [WARNING] Javadoc Warnings
> [WARNING]
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java:193:
> warning - End Delimiter } missing for possible See Tag in comment string: "A
> non-instantiable class that manages creation of {@link HConnection}s.
> [WARNING] The simplest way to use this class is by using {@link
> #createConnection(Configuration)}.
> [WARNING] This creates a new {@link HConnection} to the cluster that is
> managed by the caller.
> [WARNING] From this {@link HConnection} {@link HTableInterface}
> implementations are retrieved
> [WARNING] with {@link HConnection#getTable(byte[])}. Example:
> [WARNING]
> [WARNING] {@code
> [WARNING] HConnection connection =
> HConnectionManager.createConnection(config);
> [WARNING] HTableInterface table = connection.getTable("table1");
> [WARNING] try {
> [WARNING] // Use the table as needed, for a single operation and a single
> thread
> [WARNING] } finally {
> [WARNING] table.close();
> [WARNING] connection.close();
> [WARNING] }
> [WARNING]
> [WARNING] The following logic and API will be removed in the future:
> [WARNING] This class has a static Map of {@link HConnection} instances
> keyed by
> {code}
> The @code misses right brace
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786788#comment-13786788
]
stack commented on HBASE-9612:
--
Thanks @nkeywal for finishing the patch (dev-around-the-clock).
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612-nico.v14.096.txt, 9612-nico.v14.096.txt, 9612-nico.v14.096.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch,
> 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt,
> 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt,
> broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[ https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786789#comment-13786789 ] Hudson commented on HBASE-9612: --- FAILURE: Integrated in hbase-0.96-hadoop2 #76 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/76/]) HBASE-9612 Ability to batch edits destined to different regions (stack: rev 1529355) * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallScanner.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/PayloadCarryingRpcController.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java * /hbase/branches/0.96/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java * /hbase/branches/0.96/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java * /hbase/branches/0.96/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java * /hbase/branches/0.96/hbase-protocol/README.txt * /hbase/branches/0.96/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/branches/0.96/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/branches/0.96/hbase-protocol/src/main/protobuf/Client.proto * /hbase/branches/0.96/hbase-protocol/src/main/protobuf/RPC.proto * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/QosFunction.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestIPC.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestSnapshotFromMaster.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionBusyWait.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQosFunction.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/rest/TestResourceFilter.java > Ability to batch edits destined to different regions > > > Key: HBASE-9612 > URL: https://issues.apache.org/jira/browse/HBASE-9612 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0 >Reporter: Benoit Sigoure >Assignee: stack >Priority: Blocker > Fix For: 0.98.0, 0.96.0 > > Attachments: > 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt, > 9612-nico.v14.096.txt, 9612-nico.v14.096.txt, 9612-nico.v14.096.txt, > 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch, > 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch, > 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt, > 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt, > broken.patch > > > The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits > destined to differen
[jira] [Commented] (HBASE-9688) Fix javadoc warning in HConnectionManager class javadoc
[
https://issues.apache.org/jira/browse/HBASE-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786790#comment-13786790
]
Hudson commented on HBASE-9688:
---
FAILURE: Integrated in hbase-0.96-hadoop2 #76 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/76/])
HBASE-9688 Fix javadoc warning in HConnectionManager class javadoc (stack: rev
1529312)
*
/hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
> Fix javadoc warning in HConnectionManager class javadoc
> ---
>
> Key: HBASE-9688
> URL: https://issues.apache.org/jira/browse/HBASE-9688
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 9688.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/7422/artifact/trunk/patchprocess/patchJavadocWarnings.txt
> :
> {code}
> [WARNING] Javadoc Warnings
> [WARNING]
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java:193:
> warning - End Delimiter } missing for possible See Tag in comment string: "A
> non-instantiable class that manages creation of {@link HConnection}s.
> [WARNING] The simplest way to use this class is by using {@link
> #createConnection(Configuration)}.
> [WARNING] This creates a new {@link HConnection} to the cluster that is
> managed by the caller.
> [WARNING] From this {@link HConnection} {@link HTableInterface}
> implementations are retrieved
> [WARNING] with {@link HConnection#getTable(byte[])}. Example:
> [WARNING]
> [WARNING] {@code
> [WARNING] HConnection connection =
> HConnectionManager.createConnection(config);
> [WARNING] HTableInterface table = connection.getTable("table1");
> [WARNING] try {
> [WARNING] // Use the table as needed, for a single operation and a single
> thread
> [WARNING] } finally {
> [WARNING] table.close();
> [WARNING] connection.close();
> [WARNING] }
> [WARNING]
> [WARNING] The following logic and API will be removed in the future:
> [WARNING] This class has a static Map of {@link HConnection} instances
> keyed by
> {code}
> The @code misses right brace
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9272) A simple parallel, unordered scanner
[
https://issues.apache.org/jira/browse/HBASE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786783#comment-13786783
]
Ted Yu commented on HBASE-9272:
---
License is missing for ParallelClientScanner.java
Please add annotation for audience.
{code}
+ // reader interface
+ public static interface ResultReader {
...
+ // writer interface
+ public static interface ResultWriter {
{code}
Looks like the above classes can be private.
{code}
+ } catch (InterruptedException ix) {
+// ignore
{code}
Restore interrupt status ?
> A simple parallel, unordered scanner
>
>
> Key: HBASE-9272
> URL: https://issues.apache.org/jira/browse/HBASE-9272
> Project: HBase
> Issue Type: New Feature
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Attachments: 9272-0.94.txt, 9272-0.94-v2.txt, 9272-0.94-v3.txt,
> 9272-0.94-v4.txt, 9272-trunk.txt, ParallelClientScanner.java,
> ParallelClientScanner.java
>
>
> The contract of ClientScanner is to return rows in sort order. That limits
> the order in which region can be scanned.
> I propose a simple ParallelScanner that does not have this requirement and
> queries regions in parallel, return whatever gets returned first.
> This is generally useful for scans that filter a lot of data on the server,
> or in cases where the client can very quickly react to the returned data.
> I have a simple prototype (doesn't do error handling right, and might be a
> bit heavy on the synchronization side - it used a BlockingQueue to hand data
> between the client using the scanner and the threads doing the scanning, it
> also could potentially starve some scanners long enugh to time out at the
> server).
> On the plus side, it's only a 130 lines of code. :)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9612:
-
Attachment: 9612-nico.v14.096.txt
... and an import (good thing I haven't committed to 0.96 yet)
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612-nico.v14.096.txt, 9612-nico.v14.096.txt, 9612-nico.v14.096.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch,
> 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt,
> 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt,
> broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9612:
-
Attachment: 9612-nico.v14.096.txt
Missed a paren.
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612-nico.v14.096.txt, 9612-nico.v14.096.txt, 9612revert.txt,
> 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch, 9612v11.txt,
> 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch, 9612v2.txt,
> 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt, 9612v7.txt,
> 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt, broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9612:
-
Resolution: Fixed
Release Note: Restores being able to send many regions's worth of edits to
a regionserver when running the multi call (previous it was doing a single
region's worth at a time). Required redo of the multi proto model. Removed
multiGet since it had same problem in that it only did a region's worth at a
time; use the multi call instead to do multiGets. All should be transparent to
the user but those trying to do clients against hbase will see changes (flags
moved from GetRequest to Get and the above mentioned refactor of the
MultiRequest proto).
Status: Resolved (was: Patch Available)
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612-nico.v14.096.txt, 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt,
> 9612-v11-nico.patch, 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch,
> 9612.v14-nico.patch, 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt,
> 9612v5.txt, 9612v5.txt, 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt,
> 9612v9.txt, 9612.wip.txt, broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9612:
-
Attachment: 9612-nico.v14.096.txt
What I applied to 0.96. Needed a little fixup; filename is diferent for the
qos function and wouldn't apply in HRegionServer.
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612-nico.v14.096.txt, 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt,
> 9612-v11-nico.patch, 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch,
> 9612.v14-nico.patch, 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt,
> 9612v5.txt, 9612v5.txt, 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt,
> 9612v9.txt, 9612.wip.txt, broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9272) A simple parallel, unordered scanner
[ https://issues.apache.org/jira/browse/HBASE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9272: - Status: Patch Available (was: Open) > A simple parallel, unordered scanner > > > Key: HBASE-9272 > URL: https://issues.apache.org/jira/browse/HBASE-9272 > Project: HBase > Issue Type: New Feature >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Attachments: 9272-0.94.txt, 9272-0.94-v2.txt, 9272-0.94-v3.txt, > 9272-0.94-v4.txt, 9272-trunk.txt, ParallelClientScanner.java, > ParallelClientScanner.java > > > The contract of ClientScanner is to return rows in sort order. That limits > the order in which region can be scanned. > I propose a simple ParallelScanner that does not have this requirement and > queries regions in parallel, return whatever gets returned first. > This is generally useful for scans that filter a lot of data on the server, > or in cases where the client can very quickly react to the returned data. > I have a simple prototype (doesn't do error handling right, and might be a > bit heavy on the synchronization side - it used a BlockingQueue to hand data > between the client using the scanner and the threads doing the scanning, it > also could potentially starve some scanners long enugh to time out at the > server). > On the plus side, it's only a 130 lines of code. :) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9272) A simple parallel, unordered scanner
[ https://issues.apache.org/jira/browse/HBASE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9272: - Attachment: 9272-trunk.txt Here finally is a trunk version. > A simple parallel, unordered scanner > > > Key: HBASE-9272 > URL: https://issues.apache.org/jira/browse/HBASE-9272 > Project: HBase > Issue Type: New Feature >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Attachments: 9272-0.94.txt, 9272-0.94-v2.txt, 9272-0.94-v3.txt, > 9272-0.94-v4.txt, 9272-trunk.txt, ParallelClientScanner.java, > ParallelClientScanner.java > > > The contract of ClientScanner is to return rows in sort order. That limits > the order in which region can be scanned. > I propose a simple ParallelScanner that does not have this requirement and > queries regions in parallel, return whatever gets returned first. > This is generally useful for scans that filter a lot of data on the server, > or in cases where the client can very quickly react to the returned data. > I have a simple prototype (doesn't do error handling right, and might be a > bit heavy on the synchronization side - it used a BlockingQueue to hand data > between the client using the scanner and the threads doing the scanning, it > also could potentially starve some scanners long enugh to time out at the > server). > On the plus side, it's only a 130 lines of code. :) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786753#comment-13786753
]
Hudson commented on HBASE-9687:
---
SUCCESS: Integrated in hbase-0.96 #124 (See
[https://builds.apache.org/job/hbase-0.96/124/])
HBASE-9687 Addendum shuts down the MiniMapReduceCluster (tedyu: rev 1529303)
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
HBASE-9687 ClassNotFoundException is thrown when ExportSnapshot runs against
hadoop cluster where HBase is not installed on the same node as resourcemanager
(tedyu: rev 1529286)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687.add, 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/09/30 17:2
[jira] [Updated] (HBASE-9711) Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary
[ https://issues.apache.org/jira/browse/HBASE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9711: - Status: Patch Available (was: Open) > Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary > - > > Key: HBASE-9711 > URL: https://issues.apache.org/jira/browse/HBASE-9711 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Trivial > Fix For: 0.98.0, 0.94.13, 0.96.1 > > Attachments: 9711-trunk.txt > > > Parent patch copies input for RegexFilter unconditionally. > We should only do this if the KV portion into the passed byte[] is < 1/2 of > the passed byte[]. Otherwise we waste cycles. > Patch is trivial and will be coming momentarily. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9688) Fix javadoc warning in HConnectionManager class javadoc
[
https://issues.apache.org/jira/browse/HBASE-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786752#comment-13786752
]
Hudson commented on HBASE-9688:
---
SUCCESS: Integrated in hbase-0.96 #124 (See
[https://builds.apache.org/job/hbase-0.96/124/])
HBASE-9688 Fix javadoc warning in HConnectionManager class javadoc (stack: rev
1529312)
*
/hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
> Fix javadoc warning in HConnectionManager class javadoc
> ---
>
> Key: HBASE-9688
> URL: https://issues.apache.org/jira/browse/HBASE-9688
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 9688.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/7422/artifact/trunk/patchprocess/patchJavadocWarnings.txt
> :
> {code}
> [WARNING] Javadoc Warnings
> [WARNING]
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java:193:
> warning - End Delimiter } missing for possible See Tag in comment string: "A
> non-instantiable class that manages creation of {@link HConnection}s.
> [WARNING] The simplest way to use this class is by using {@link
> #createConnection(Configuration)}.
> [WARNING] This creates a new {@link HConnection} to the cluster that is
> managed by the caller.
> [WARNING] From this {@link HConnection} {@link HTableInterface}
> implementations are retrieved
> [WARNING] with {@link HConnection#getTable(byte[])}. Example:
> [WARNING]
> [WARNING] {@code
> [WARNING] HConnection connection =
> HConnectionManager.createConnection(config);
> [WARNING] HTableInterface table = connection.getTable("table1");
> [WARNING] try {
> [WARNING] // Use the table as needed, for a single operation and a single
> thread
> [WARNING] } finally {
> [WARNING] table.close();
> [WARNING] connection.close();
> [WARNING] }
> [WARNING]
> [WARNING] The following logic and API will be removed in the future:
> [WARNING] This class has a static Map of {@link HConnection} instances
> keyed by
> {code}
> The @code misses right brace
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9711) Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary
[ https://issues.apache.org/jira/browse/HBASE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9711: - Attachment: 9711-trunk.txt Whoops. Here's the right patch. > Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary > - > > Key: HBASE-9711 > URL: https://issues.apache.org/jira/browse/HBASE-9711 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Trivial > Fix For: 0.98.0, 0.94.13, 0.96.1 > > Attachments: 9711-trunk.txt > > > Parent patch copies input for RegexFilter unconditionally. > We should only do this if the KV portion into the passed byte[] is < 1/2 of > the passed byte[]. Otherwise we waste cycles. > Patch is trivial and will be coming momentarily. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786746#comment-13786746
]
Enis Soztutar commented on HBASE-9612:
--
Cool, hadoopqa also +1'ed the patch. Let's get this in to RC. We will run the
full suite of tests on the RC anyway, so the patch will be tested even more at
the RC.
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch,
> 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt,
> 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt,
> broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9711) Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary
[ https://issues.apache.org/jira/browse/HBASE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9711: - Attachment: (was: 9711-trunk.txt) > Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary > - > > Key: HBASE-9711 > URL: https://issues.apache.org/jira/browse/HBASE-9711 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Trivial > Fix For: 0.98.0, 0.94.13, 0.96.1 > > > Parent patch copies input for RegexFilter unconditionally. > We should only do this if the KV portion into the passed byte[] is < 1/2 of > the passed byte[]. Otherwise we waste cycles. > Patch is trivial and will be coming momentarily. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9711) Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary
[ https://issues.apache.org/jira/browse/HBASE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9711: - Priority: Trivial (was: Major) > Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary > - > > Key: HBASE-9711 > URL: https://issues.apache.org/jira/browse/HBASE-9711 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Trivial > Fix For: 0.98.0, 0.94.13, 0.96.1 > > Attachments: 9711-trunk.txt > > > Parent patch copies input for RegexFilter unconditionally. > We should only do this if the KV portion into the passed byte[] is < 1/2 of > the passed byte[]. Otherwise we waste cycles. > Patch is trivial and will be coming momentarily. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786736#comment-13786736
]
Hadoop QA commented on HBASE-9612:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12606882/9612.v14-nico.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 33 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7473//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7473//console
This message is automatically generated.
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch,
> 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt,
> 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt,
> broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{Mutati
[jira] [Updated] (HBASE-9711) Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary
[ https://issues.apache.org/jira/browse/HBASE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9711: - Attachment: 9711-trunk.txt Simple patch. Copying the bytes ahead of the String(...) only makes sense if the compared portion is less than 1/2 of the passed byte[]. This is a simple patch. If I do not hear any objections I will commit this to 0.94+. > Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary > - > > Key: HBASE-9711 > URL: https://issues.apache.org/jira/browse/HBASE-9711 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.13, 0.96.1 > > Attachments: 9711-trunk.txt > > > Parent patch copies input for RegexFilter unconditionally. > We should only do this if the KV portion into the passed byte[] is < 1/2 of > the passed byte[]. Otherwise we waste cycles. > Patch is trivial and will be coming momentarily. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (HBASE-9711) Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary
[ https://issues.apache.org/jira/browse/HBASE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl reassigned HBASE-9711: Assignee: Lars Hofhansl > Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary > - > > Key: HBASE-9711 > URL: https://issues.apache.org/jira/browse/HBASE-9711 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.13, 0.96.1 > > > Parent patch copies input for RegexFilter unconditionally. > We should only do this if the KV portion into the passed byte[] is < 1/2 of > the passed byte[]. Otherwise we waste cycles. > Patch is trivial and will be coming momentarily. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9709) LogReplay throws NPE when no KVs to be replayed in a WALEdit
[
https://issues.apache.org/jira/browse/HBASE-9709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786680#comment-13786680
]
Hudson commented on HBASE-9709:
---
SUCCESS: Integrated in hbase-0.96-hadoop2 #75 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/75/])
HBASE-9709: LogReplay throws NPE when no KVs to be replayed in a WALEdit
(jeffreyz: rev 1529252)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
> LogReplay throws NPE when no KVs to be replayed in a WALEdit
>
>
> Key: HBASE-9709
> URL: https://issues.apache.org/jira/browse/HBASE-9709
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9709.patch
>
>
> This is regression from my recent checkin from hbase-9390 below is the
> exception stack:
> {code}
> 2013-10-03 09:34:32,735 ERROR [WriterThread-1] wal.HLogSplitter: Exiting
> thread
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogReplayOutputSink.groupEditsByServer(HLogSplitter.java:1489)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogReplayOutputSink.append(HLogSplitter.java:1368)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.writeBuffer(HLogSplitter.java:847)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.doRun(HLogSplitter.java:839)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.run(HLogSplitter.java:809)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786681#comment-13786681
]
Hudson commented on HBASE-9687:
---
SUCCESS: Integrated in hbase-0.96-hadoop2 #75 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/75/])
HBASE-9687 Addendum shuts down the MiniMapReduceCluster (tedyu: rev 1529303)
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
HBASE-9687 ClassNotFoundException is thrown when ExportSnapshot runs against
hadoop cluster where HBase is not installed on the same node as resourcemanager
(tedyu: rev 1529286)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687.add, 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
>
[jira] [Created] (HBASE-9712) TestSplitLogManager still fails on occasion
stack created HBASE-9712:
Summary: TestSplitLogManager still fails on occasion
Key: HBASE-9712
URL: https://issues.apache.org/jira/browse/HBASE-9712
Project: HBase
Issue Type: Bug
Reporter: stack
Opening this issue to keep account of failures. It failed for me locally just
now.
Failed tests:
testTaskResigned(org.apache.hadoop.hbase.master.TestSplitLogManager):
version1=2, version=2
{code}
durruti:hbase stack$ more
hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.TestSplitLogManager.txt
---
Test set: org.apache.hadoop.hbase.master.TestSplitLogManager
---
Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 86.697 sec <<<
FAILURE!
testTaskResigned(org.apache.hadoop.hbase.master.TestSplitLogManager) Time
elapsed: 0.004 sec <<< FAILURE!
java.lang.AssertionError: version1=2, version=2
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.hbase.master.TestSplitLogManager.testTaskResigned(TestSplitLogManager.java:387)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
{code}
Let me attach the log
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-4364) Filters applied to columns not in the selected column list are ignored
[ https://issues.apache.org/jira/browse/HBASE-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4364: - Attachment: org.apache.hadoop.hbase.master.TestSplitLogManager-output.txt Log of failed test. > Filters applied to columns not in the selected column list are ignored > -- > > Key: HBASE-4364 > URL: https://issues.apache.org/jira/browse/HBASE-4364 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 0.90.4, 0.92.0, 0.94.0 >Reporter: Todd Lipcon >Priority: Critical > Attachments: > HBASE-4364-failing-test-with-simplest-custom-filter.patch, HBASE-4364.patch, > hbase-4364_trunk.patch, hbase-4364_trunk-v2.patch, > org.apache.hadoop.hbase.master.TestSplitLogManager-output.txt > > > For a scan, if you select some set of columns using addColumns(), and then > apply a SingleColumnValueFilter that restricts the results based on some > other columns which aren't selected, then those filter conditions are ignored. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786672#comment-13786672
]
Hadoop QA commented on HBASE-9687:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12606873/9687-v12.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7469//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7469//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7469//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7469//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7469//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7469//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7469//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7469//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7469//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7469//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7469//console
This message is automatically generated.
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687.add, 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/3
[jira] [Commented] (HBASE-9706) Improve detection of secure ZooKeeper
[
https://issues.apache.org/jira/browse/HBASE-9706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786660#comment-13786660
]
stack commented on HBASE-9706:
--
[~eyang] Does that tmp file get cleaned up after test is done? +File
saslConfFile = File.createTempFile("tmp", "fakeJaas.conf");
[~mbertozzi] You want more changes in this patch or you good w/ it as is?
Thanks.
> Improve detection of secure ZooKeeper
> -
>
> Key: HBASE-9706
> URL: https://issues.apache.org/jira/browse/HBASE-9706
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.94.4, 0.95.0
> Environment: Java 6, RHEL, HBase 0.94.9
>Reporter: Eric Yang
>Assignee: Eric Yang
> Attachments: JIRA-9706-0.94.patch, JIRA-9706-0.95-trunk.patch
>
>
> HBase client code assumes ZooKeeper is secured, as long as there is a
> java.security.auth.login.config property being set. When HBase client is
> embedded in other java program with other security configuration, it can
> produce wrong assumption that ZooKeeper is secured. Ideally,
> isSecureZooKeeper method should detect Jaas configuration specifically for
> ZooKeeper to ensure that client program doesn't have a false positive
> detection.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9709) LogReplay throws NPE when no KVs to be replayed in a WALEdit
[
https://issues.apache.org/jira/browse/HBASE-9709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786658#comment-13786658
]
Hudson commented on HBASE-9709:
---
FAILURE: Integrated in HBase-TRUNK #4591 (See
[https://builds.apache.org/job/HBase-TRUNK/4591/])
HBASE-9709: LogReplay throws NPE when no KVs to be replayed in a WALEdit
(jeffreyz: rev 1529247)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
> LogReplay throws NPE when no KVs to be replayed in a WALEdit
>
>
> Key: HBASE-9709
> URL: https://issues.apache.org/jira/browse/HBASE-9709
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9709.patch
>
>
> This is regression from my recent checkin from hbase-9390 below is the
> exception stack:
> {code}
> 2013-10-03 09:34:32,735 ERROR [WriterThread-1] wal.HLogSplitter: Exiting
> thread
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogReplayOutputSink.groupEditsByServer(HLogSplitter.java:1489)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogReplayOutputSink.append(HLogSplitter.java:1368)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.writeBuffer(HLogSplitter.java:847)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.doRun(HLogSplitter.java:839)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.run(HLogSplitter.java:809)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786641#comment-13786641
]
stack commented on HBASE-9687:
--
Just wondering if it broke unit tests. Sounds like we are good. No worries.
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687.add, 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/09/30 17:23:58 INFO mapreduce.Job: Running job: job_1380335340676_0003
> 13/09/30 17:24:03 INFO mapreduce.Job: Job job_1380335340676_0003 running in
> uber mode : false
> 13/09/30 17:24:03 INFO mapreduce.Job: map 0% reduce 0%
> 13/09/30 17:24:06 INFO mapreduce.Job: Task Id :
> attempt_1380335340676_0003_m_00_0, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.TableName
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at j
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786638#comment-13786638
]
Matteo Bertozzi commented on HBASE-9687:
[~saint@gmail.com] yeah no broken build, and the test is marked as Large...
so no problem anyway.. but I missed the shutdown of the mr cluster on review..
my bad
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687.add, 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/09/30 17:23:58 INFO mapreduce.Job: Running job: job_1380335340676_0003
> 13/09/30 17:24:03 INFO mapreduce.Job: Job job_1380335340676_0003 running in
> uber mode : false
> 13/09/30 17:24:03 INFO mapreduce.Job: map 0% reduce 0%
> 13/09/30 17:24:06 INFO mapreduce.Job: Task Id :
> attempt_1380335340676_0003_m_00_0, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.TableName
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.Acces
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786634#comment-13786634
]
Ted Yu commented on HBASE-9687:
---
I ran TestExportSnapshot for 0.96 based on hadoop-2 profile and it passed.
Where did you see the broken build ?
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687.add, 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/09/30 17:23:58 INFO mapreduce.Job: Running job: job_1380335340676_0003
> 13/09/30 17:24:03 INFO mapreduce.Job: Job job_1380335340676_0003 running in
> uber mode : false
> 13/09/30 17:24:03 INFO mapreduce.Job: map 0% reduce 0%
> 13/09/30 17:24:06 INFO mapreduce.Job: Task Id :
> attempt_1380335340676_0003_m_00_0, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.TableName
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.find
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786628#comment-13786628
]
stack commented on HBASE-9687:
--
Does this patch break 0.96 build?
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687.add, 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/09/30 17:23:58 INFO mapreduce.Job: Running job: job_1380335340676_0003
> 13/09/30 17:24:03 INFO mapreduce.Job: Job job_1380335340676_0003 running in
> uber mode : false
> 13/09/30 17:24:03 INFO mapreduce.Job: map 0% reduce 0%
> 13/09/30 17:24:06 INFO mapreduce.Job: Task Id :
> attempt_1380335340676_0003_m_00_0, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.TableName
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.j
[jira] [Commented] (HBASE-9706) Improve detection of secure ZooKeeper
[ https://issues.apache.org/jira/browse/HBASE-9706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786625#comment-13786625 ] Gary Helmling commented on HBASE-9706: -- +1, patch looks good to me. > Improve detection of secure ZooKeeper > - > > Key: HBASE-9706 > URL: https://issues.apache.org/jira/browse/HBASE-9706 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.4, 0.95.0 > Environment: Java 6, RHEL, HBase 0.94.9 >Reporter: Eric Yang >Assignee: Eric Yang > Attachments: JIRA-9706-0.94.patch, JIRA-9706-0.95-trunk.patch > > > HBase client code assumes ZooKeeper is secured, as long as there is a > java.security.auth.login.config property being set. When HBase client is > embedded in other java program with other security configuration, it can > produce wrong assumption that ZooKeeper is secured. Ideally, > isSecureZooKeeper method should detect Jaas configuration specifically for > ZooKeeper to ensure that client program doesn't have a false positive > detection. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9687:
--
Attachment: 9687.add
Addendum shuts down the MiniMapReduceCluster
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687.add, 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/09/30 17:23:58 INFO mapreduce.Job: Running job: job_1380335340676_0003
> 13/09/30 17:24:03 INFO mapreduce.Job: Job job_1380335340676_0003 running in
> uber mode : false
> 13/09/30 17:24:03 INFO mapreduce.Job: map 0% reduce 0%
> 13/09/30 17:24:06 INFO mapreduce.Job: Task Id :
> attempt_1380335340676_0003_m_00_0, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.TableName
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.
[jira] [Commented] (HBASE-9709) LogReplay throws NPE when no KVs to be replayed in a WALEdit
[
https://issues.apache.org/jira/browse/HBASE-9709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786619#comment-13786619
]
Hudson commented on HBASE-9709:
---
SUCCESS: Integrated in hbase-0.96 #123 (See
[https://builds.apache.org/job/hbase-0.96/123/])
HBASE-9709: LogReplay throws NPE when no KVs to be replayed in a WALEdit
(jeffreyz: rev 1529252)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
> LogReplay throws NPE when no KVs to be replayed in a WALEdit
>
>
> Key: HBASE-9709
> URL: https://issues.apache.org/jira/browse/HBASE-9709
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9709.patch
>
>
> This is regression from my recent checkin from hbase-9390 below is the
> exception stack:
> {code}
> 2013-10-03 09:34:32,735 ERROR [WriterThread-1] wal.HLogSplitter: Exiting
> thread
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogReplayOutputSink.groupEditsByServer(HLogSplitter.java:1489)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogReplayOutputSink.append(HLogSplitter.java:1368)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.writeBuffer(HLogSplitter.java:847)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.doRun(HLogSplitter.java:839)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.run(HLogSplitter.java:809)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (HBASE-9711) Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary
Lars Hofhansl created HBASE-9711: Summary: Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary Key: HBASE-9711 URL: https://issues.apache.org/jira/browse/HBASE-9711 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Parent patch copies input for RegexFilter unconditionally. We should only do this if the KV portion into the passed byte[] is < 1/2 of the passed byte[]. Otherwise we waste cycles. Patch is trivial and will be coming momentarily. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9711) Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary
[ https://issues.apache.org/jira/browse/HBASE-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9711: - Fix Version/s: (was: 0.96.0) (was: 0.94.12) 0.96.1 0.94.13 > Improve HBASE-9428 - avoid copying bytes for RegexFilter unless necessary > - > > Key: HBASE-9711 > URL: https://issues.apache.org/jira/browse/HBASE-9711 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl > Fix For: 0.98.0, 0.94.13, 0.96.1 > > > Parent patch copies input for RegexFilter unconditionally. > We should only do this if the KV portion into the passed byte[] is < 1/2 of > the passed byte[]. Otherwise we waste cycles. > Patch is trivial and will be coming momentarily. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786614#comment-13786614
]
stack commented on HBASE-9612:
--
v13 patch passed all tests when run locally for me.
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch,
> 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt,
> 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt,
> broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786609#comment-13786609
]
Hadoop QA commented on HBASE-9687:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12606873/9687-v12.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7470//console
This message is automatically generated.
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/09/30 17:23:58 INFO mapreduce.Job: Running job: job_1380335340676_0003
> 13/09/30 17:24:03 INFO mapreduce.Job:
[jira] [Commented] (HBASE-9706) Improve detection of secure ZooKeeper
[ https://issues.apache.org/jira/browse/HBASE-9706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786595#comment-13786595 ] Matteo Bertozzi commented on HBASE-9706: patch looks good to me, and also the client should be able to connect to zookeeper even without jaas.conf since all znodes accessed by the client should be WORLD_READABLE (set by ZKUtil.createACL()) [~apurtell] [~ghelmling] thoughts? > Improve detection of secure ZooKeeper > - > > Key: HBASE-9706 > URL: https://issues.apache.org/jira/browse/HBASE-9706 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.4, 0.95.0 > Environment: Java 6, RHEL, HBase 0.94.9 >Reporter: Eric Yang >Assignee: Eric Yang > Attachments: JIRA-9706-0.94.patch, JIRA-9706-0.95-trunk.patch > > > HBase client code assumes ZooKeeper is secured, as long as there is a > java.security.auth.login.config property being set. When HBase client is > embedded in other java program with other security configuration, it can > produce wrong assumption that ZooKeeper is secured. Ideally, > isSecureZooKeeper method should detect Jaas configuration specifically for > ZooKeeper to ensure that client program doesn't have a false positive > detection. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-8803) region_mover.rb should move multiple regions at a time
[ https://issues.apache.org/jira/browse/HBASE-8803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8803: - Priority: Critical (was: Major) Fix Version/s: (was: 0.96.0) 0.96.1 > region_mover.rb should move multiple regions at a time > -- > > Key: HBASE-8803 > URL: https://issues.apache.org/jira/browse/HBASE-8803 > Project: HBase > Issue Type: Bug > Components: Usability >Affects Versions: 0.98.0, 0.94.8, 0.95.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Critical > Fix For: 0.98.0, 0.94.13, 0.96.1 > > Attachments: 8803v5.txt, HBASE-8803-v0-trunk.patch, > HBASE-8803-v1-0.94.patch, HBASE-8803-v1-trunk.patch, > HBASE-8803-v2-0.94.patch, HBASE-8803-v2-0.94.patch, HBASE-8803-v3-0.94.patch, > HBASE-8803-v4-0.94.patch, HBASE-8803-v4-trunk.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When there is many regions in a cluster, rolling_restart can take hours > because region_mover is moving the regions one by one. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786592#comment-13786592
]
Ted Yu commented on HBASE-9687:
---
Thanks for the quick review, Matteo.
Integrated to 0.96 and trunk.
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/09/30 17:23:58 INFO mapreduce.Job: Running job: job_1380335340676_0003
> 13/09/30 17:24:03 INFO mapreduce.Job: Job job_1380335340676_0003 running in
> uber mode : false
> 13/09/30 17:24:03 INFO mapreduce.Job: map 0% reduce 0%
> 13/09/30 17:24:06 INFO mapreduce.Job: Task Id :
> attempt_1380335340676_0003_m_00_0, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.TableName
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoad
[jira] [Commented] (HBASE-9322) Contention on the SoftValueSortedMap with multiple clients
[
https://issues.apache.org/jira/browse/HBASE-9322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786591#comment-13786591
]
stack commented on HBASE-9322:
--
Close as 'not a problem' or lower priority since root issue was thousands of
threads?
> Contention on the SoftValueSortedMap with multiple clients
> --
>
> Key: HBASE-9322
> URL: https://issues.apache.org/jira/browse/HBASE-9322
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Jean-Daniel Cryans
> Fix For: 0.98.0, 0.96.0
>
>
> I've been running tests on clusters with "lots" of regions, about 400, and
> I'm seeing weird contention in the client.
> This one shows up a lot around the SoftValueSortedMap.
> First I have this blocked thread on I'm not sure what:
> {noformat}
> "TestClient-12" prio=10 tid=0x7fb268872000 nid=0x3add waiting for monitor
> entry [0x7fb251416000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.hbase.util.SoftValueSortedMap.isEmpty(SoftValueSortedMap.java:210)
> - locked <0xc1b70318> (a java.util.TreeMap)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:1263)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1103)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1036)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:993)
> {noformat}
> This client waits on it:
> {noformat}
> "TestClient-14" prio=10 tid=0x7fb268876000 nid=0x3adf waiting for monitor
> entry [0x7fb251214000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.hbase.util.SoftValueSortedMap.lowerValueByKey(SoftValueSortedMap.java:189)
> - waiting to lock <0xc1b70318> (a java.util.TreeMap)
> - locked <0xc1b82120> (a
> org.apache.hadoop.hbase.util.SoftValueSortedMap)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:1272)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1103)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1036)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:993)
> {noformat}
> As you can see it's also holding a lock, which I have 11 other clients
> waiting on:
> {noformat}
> "TestClient-13" prio=10 tid=0x7fb268874000 nid=0x3ade waiting for monitor
> entry [0x7fb251315000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.hbase.util.SoftValueSortedMap.lowerValueByKey(SoftValueSortedMap.java:189)
> - waiting to lock <0xc1b82120> (a
> org.apache.hadoop.hbase.util.SoftValueSortedMap)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:1272)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1103)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1036)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:993)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9664) ArrayIndexOutOfBoundsException may be thrown in TestZKSecretWatcher
[
https://issues.apache.org/jira/browse/HBASE-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9664:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed a while back. Resolving.
> ArrayIndexOutOfBoundsException may be thrown in TestZKSecretWatcher
> ---
>
> Key: HBASE-9664
> URL: https://issues.apache.org/jira/browse/HBASE-9664
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9664.txt
>
>
> In our internal Jenkins build, I saw failure in TestZKSecretWatcher:
> {code}
> java.lang.ArrayIndexOutOfBoundsException: 2
> at
> org.apache.hadoop.hbase.security.token.TestZKSecretWatcher.setupBeforeClass(TestZKSecretWatcher.java:87)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> {code}
> This was due to i being 1, resulting in index of 2 being used in the
> following statement:
> {code}
> KEY_SLAVE = tmp[ i+1 % 2 ];
> {code}
> See http://docs.oracle.com/javase/tutorial/java/nutsandbolts/operators.html
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9670) Client pause should be 100ms everywhere
[ https://issues.apache.org/jira/browse/HBASE-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9670: - Resolution: Fixed Status: Resolved (was: Patch Available) This was committed a while back. Resolving. > Client pause should be 100ms everywhere > --- > > Key: HBASE-9670 > URL: https://issues.apache.org/jira/browse/HBASE-9670 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.96.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.98.0, 0.96.0 > > Attachments: 9670.v1.patch > > > It was changed to 100ms, but kept as 1000ms in a lot of places... -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
[
https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786583#comment-13786583
]
Matteo Bertozzi commented on HBASE-9687:
+1 for me
> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop
> cluster where HBase is not installed on the same node as resourcemanager
> -
>
> Key: HBASE-9687
> URL: https://issues.apache.org/jira/browse/HBASE-9687
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9687-v12.txt, 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://hor17n25:8020/
> -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot
> snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under
> directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to
> override final parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at
> hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application
> application_1380335340676_0003 to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job:
> http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/09/30 17:23:58 INFO mapreduce.Job: Running job: job_1380335340676_0003
> 13/09/30 17:24:03 INFO mapreduce.Job: Job job_1380335340676_0003 running in
> uber mode : false
> 13/09/30 17:24:03 INFO mapreduce.Job: map 0% reduce 0%
> 13/09/30 17:24:06 INFO mapreduce.Job: Task Id :
> attempt_1380335340676_0003_m_00_0, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.TableName
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at
[jira] [Updated] (HBASE-9428) Regex filters are at least an order of magnitude slower since 0.94.3
[
https://issues.apache.org/jira/browse/HBASE-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9428:
-
Fix Version/s: (was: 0.96.1)
0.96.0
> Regex filters are at least an order of magnitude slower since 0.94.3
>
>
> Key: HBASE-9428
> URL: https://issues.apache.org/jira/browse/HBASE-9428
> Project: HBase
> Issue Type: Bug
>Reporter: Jean-Daniel Cryans
>Assignee: Lars Hofhansl
> Fix For: 0.98.0, 0.94.12, 0.96.0
>
> Attachments: 9428-0.94.txt, 9428-trunk.txt
>
>
> I found this issue after debugging a performance problem on an OpenTSDB
> cluster, it was basically unusable after an upgrade from 0.94.2 to 0.94.6. It
> was caused by HBASE-7279 (ping [~lhofhansl]).
> The easiest way to see it is to run a simple 1 client PE:
> {noformat}
> $ ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite 1
> {noformat}
> Then in the shell do a filter scan (flush the table first and make sure if
> fits in your blockcache if you want stable numbers).
> Pre HBASE-7279:
> {noformat}
> hbase(main):028:0> scan 'TestTable', {FILTER => "(RowFilter (=,
> 'regexstring:055872') )"}
> ROW COLUMN+CELL
>
>
> 055872 column=info:data,
> timestamp=1378248850191, value=(blanked)
>
>
> 1 row(s) in 1.2780 seconds
> {noformat}
> Post HBASE-7279
> {noformat}
> hbase(main):037:0* scan 'TestTable', {FILTER => "(RowFilter (=,
> 'regexstring:055872') )"}
> ROW COLUMN+CELL
>
>
> 055872 column=info:data,
> timestamp=1378248850191, value=(blanked)
>
>
> 1 row(s) in 24.2940 seconds
> {noformat}
> I tried a bunch of 0.94, up to 0.94.11, and the tip of 0.96. They are all
> slow like this.
> It seems that since that jira went in we do a lot more row matching, and
> running the regex gets super expensive.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9301) Default hbase.dynamic.jars.dir to hbase.rootdir/jars
[ https://issues.apache.org/jira/browse/HBASE-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9301: - Fix Version/s: (was: 0.96.1) 0.96.0 > Default hbase.dynamic.jars.dir to hbase.rootdir/jars > > > Key: HBASE-9301 > URL: https://issues.apache.org/jira/browse/HBASE-9301 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.2, 0.94.12, 0.96.0 >Reporter: James Taylor >Assignee: Vasu Mariyala > Fix For: 0.98.0, 0.94.12, 0.96.0 > > Attachments: 0.94-HBASE-9301.patch, 0.94-HBASE-9301-rev1.patch, > HBASE-9301.patch, HBASE-9301-rev1.patch, HBASE-9301-rev2.patch > > > A reasonable default for hbase.dynamic.jars.dir would be hbase.rootdir/jars > so that folks aren't forced to edit their hbase-sites.xml to take advantage > of the new, cool feature to load coprocessor/custom filter jars out of HDFS. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-8930) Filter evaluates KVs outside requested columns
[
https://issues.apache.org/jira/browse/HBASE-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8930:
-
Fix Version/s: (was: 0.96.1)
0.96.0
> Filter evaluates KVs outside requested columns
> --
>
> Key: HBASE-8930
> URL: https://issues.apache.org/jira/browse/HBASE-8930
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.94.7
>Reporter: Federico Gaule
>Assignee: Vasu Mariyala
>Priority: Critical
> Labels: filters, hbase, keyvalue
> Fix For: 0.98.0, 0.94.12, 0.96.0
>
> Attachments: 0.94-HBASE-8930.patch, 0.94-HBASE-8930-rev1.patch,
> 0.94-HBASE-8930-rev2.patch, 0.94-Independent-Test-Execution.patch,
> 0.95-HBASE-8930.patch, 0.95-HBASE-8930-rev1.patch,
> 0.96-HBASE-8930-rev2.patch, 0.96-HBASE-8930-rev3.patch,
> 0.96-trunk-Independent-Test-Execution.patch, 8930-0.94.txt, HBASE-8930.patch,
> HBASE-8930-rev1.patch, HBASE-8930-rev2.patch, HBASE-8930-rev3.patch,
> HBASE-8930-rev4.patch, HBASE-8930-rev5.patch, HBASE-8930-rev6.patch
>
>
> 1- Fill row with some columns
> 2- Get row with some columns less than universe - Use filter to print kvs
> 3- Filter prints not requested columns
> Filter (AllwaysNextColFilter) always return ReturnCode.INCLUDE_AND_NEXT_COL
> and prints KV's qualifier
> SUFFIX_0 = 0
> SUFFIX_1 = 1
> SUFFIX_4 = 4
> SUFFIX_6 = 6
> P= Persisted
> R= Requested
> E= Evaluated
> X= Returned
> | 5580 | 5581 | 5584 | 5586 | 5590 | 5591 | 5594 | 5596 | 5600 | 5601 | 5604
> | 5606 |...
> | | P | P | | | P | P | | | P | P
> | |...
> | | R | R | R | | R | R | R | | |
> | |...
> | | E | E | | | E | E | | |
> {color:red}E{color} | | |...
> | | X | X | | | X | X | | | |
> | |
> {code:title=ExtraColumnTest.java|borderStyle=solid}
> @Test
> public void testFilter() throws Exception {
> Configuration config = HBaseConfiguration.create();
> config.set("hbase.zookeeper.quorum", "myZK");
> HTable hTable = new HTable(config, "testTable");
> byte[] cf = Bytes.toBytes("cf");
> byte[] row = Bytes.toBytes("row");
> byte[] col1 = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 558, (byte) SUFFIX_1));
> byte[] col2 = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 559, (byte) SUFFIX_1));
> byte[] col3 = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 560, (byte) SUFFIX_1));
> byte[] col4 = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 561, (byte) SUFFIX_1));
> byte[] col5 = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 562, (byte) SUFFIX_1));
> byte[] col6 = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 563, (byte) SUFFIX_1));
> byte[] col1g = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 558, (byte) SUFFIX_6));
> byte[] col2g = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 559, (byte) SUFFIX_6));
> byte[] col1v = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 558, (byte) SUFFIX_4));
> byte[] col2v = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 559, (byte) SUFFIX_4));
> byte[] col3v = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 560, (byte) SUFFIX_4));
> byte[] col4v = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 561, (byte) SUFFIX_4));
> byte[] col5v = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 562, (byte) SUFFIX_4));
> byte[] col6v = new QualifierConverter().objectToByteArray(new
> Qualifier((short) 563, (byte) SUFFIX_4));
> // === INSERTION =//
> Put put = new Put(row);
> put.add(cf, col1, Bytes.toBytes((short) 1));
> put.add(cf, col2, Bytes.toBytes((short) 1));
> put.add(cf, col3, Bytes.toBytes((short) 3));
> put.add(cf, col4, Bytes.toBytes((short) 3));
> put.add(cf, col5, Bytes.toBytes((short) 3));
> put.add(cf, col6, Bytes.toBytes((short) 3));
> hTable.put(put);
> put = new Put(row);
> put.add(cf, col1v, Bytes.toBytes((short) 10));
> put.add(cf, col2v, Bytes.toBytes((short) 10));
> put.add(cf, col3v, Bytes.toBytes((short) 10));
> put.add(cf, col4v, Bytes.toBytes((short) 10));
> put.add(cf, col5v, Bytes.toBytes((short) 10));
> put.add(cf, col6v, Bytes.toBytes((short) 10));
> hTable.put(put);
> hTable.flushCommi
[jira] [Updated] (HBASE-9414) start-hbase.cmd doesn't need the execute flag.
[ https://issues.apache.org/jira/browse/HBASE-9414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9414: - Fix Version/s: (was: 0.96.1) 0.96.0 > start-hbase.cmd doesn't need the execute flag. > -- > > Key: HBASE-9414 > URL: https://issues.apache.org/jira/browse/HBASE-9414 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.0 >Reporter: Jean-Marc Spaggiari >Assignee: stack >Priority: Trivial > Fix For: 0.98.0, 0.96.0 > > Attachments: 9414.txt > > > When you do start- and tabulation, since there start-hbase.cmd has the > execution flag, completion go only up to start-hbase. > We should remove the execution flag for this script. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9701) Make SLEEP_TIME in IntegrationTestMTTR configurable
[ https://issues.apache.org/jira/browse/HBASE-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9701: - Fix Version/s: (was: 0.96.1) 0.96.0 > Make SLEEP_TIME in IntegrationTestMTTR configurable > --- > > Key: HBASE-9701 > URL: https://issues.apache.org/jira/browse/HBASE-9701 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.98.0, 0.96.1 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon >Priority: Minor > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-9701.v1.patch > > > This will allow to try to shorten the testing time. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9412) Startup scripts create 2 .out files.
[
https://issues.apache.org/jira/browse/HBASE-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9412:
-
Fix Version/s: (was: 0.96.1)
0.96.0
> Startup scripts create 2 .out files.
>
>
> Key: HBASE-9412
> URL: https://issues.apache.org/jira/browse/HBASE-9412
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.96.0
>Reporter: Jean-Marc Spaggiari
>Assignee: stack
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9412.txt
>
>
> When start HBase with bin/start-hbase.sh, script creates 2 out files.
> {code}
> -rw-r--r-- 1 jmspaggiari jmspaggiari 0 Aug 31 15:38
> hbase-jmspaggiari-master-t430s.out
> -rw-r--r-- 1 jmspaggiari jmspaggiari 0 Aug 31 15:38
> hbase-jmspaggiari-master-t430s.out.1
> {code}
> Should create only one.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-7462) TestDrainingServer is an integration test. It should be a unit test instead
[ https://issues.apache.org/jira/browse/HBASE-7462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7462: - Fix Version/s: (was: 0.96.1) 0.96.0 > TestDrainingServer is an integration test. It should be a unit test instead > --- > > Key: HBASE-7462 > URL: https://issues.apache.org/jira/browse/HBASE-7462 > Project: HBase > Issue Type: Wish > Components: test >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Gustavo Anatoly >Priority: Trivial > Labels: noob > Fix For: 0.98.0, 0.96.0 > > Attachments: 7462.v3.patch, HBASE-7462-v2.patch > > > TestDrainingServer tests the function that allows to say that a regionserver > should not get new regions. > As it is written today, it's an integration test: it starts & stops a cluster. > The test would be more efficient if it would just check that the > AssignmentManager does not use the drained region server; whatever the > circumstances (bulk assign or not for example). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9446) saveVersion.sh spit pushd: not found popd: not found
[
https://issues.apache.org/jira/browse/HBASE-9446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9446:
-
Fix Version/s: (was: 0.96.1)
0.96.0
> saveVersion.sh spit pushd: not found popd: not found
>
>
> Key: HBASE-9446
> URL: https://issues.apache.org/jira/browse/HBASE-9446
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Jimmy Xiang
>Assignee: Elliott Clark
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9446-0.patch
>
>
> I got the following in building 0.96.
> {noformat}
> [exec]
> /home/jxiang/git-repos/apache/tmp/0.96/hbase-common/src/saveVersion.sh: 26:
> /home/jxiang/git-repos/apache/tmp/0.96/hbase-common/src/saveVersion.sh:
> pushd: not found
> [exec]
> /home/jxiang/git-repos/apache/tmp/0.96/hbase-common/src/saveVersion.sh: 43:
> /home/jxiang/git-repos/apache/tmp/0.96/hbase-common/src/saveVersion.sh: popd:
> not found
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9558) PerformanceEvaluation is in hbase-server, and create a dependency to MiniDFSCluster
[ https://issues.apache.org/jira/browse/HBASE-9558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9558: - Fix Version/s: (was: 0.96.1) 0.96.0 > PerformanceEvaluation is in hbase-server, and create a dependency to > MiniDFSCluster > --- > > Key: HBASE-9558 > URL: https://issues.apache.org/jira/browse/HBASE-9558 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon >Priority: Minor > Fix For: 0.98.0, 0.96.0 > > Attachments: 9558.addendum.v1.patch, 9958.v1.patch > > > It's the only dependency that is not in the tests package. > I'm not clear on how to fix it. Any idea? -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9571) hbase-client does not depend on map reduce, but the pom.xml includes it
[ https://issues.apache.org/jira/browse/HBASE-9571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9571: - Fix Version/s: (was: 0.96.1) > hbase-client does not depend on map reduce, but the pom.xml includes it > --- > > Key: HBASE-9571 > URL: https://issues.apache.org/jira/browse/HBASE-9571 > Project: HBase > Issue Type: Bug > Components: build, Client >Affects Versions: 0.98.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon >Priority: Minor > Attachments: 9571.v1.patch > > > mvn dependency:analyze is strange on this one. You can remove or add the > dependency, it won't complain about "used but undeclared" or "undeclared but > used" dependencies. But it does a difference in dependency:tree or for the > client application. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9511) LZ4 codec retrieval executes redundant code
[
https://issues.apache.org/jira/browse/HBASE-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9511:
-
Fix Version/s: (was: 0.96.1)
0.96.0
> LZ4 codec retrieval executes redundant code
> ---
>
> Key: HBASE-9511
> URL: https://issues.apache.org/jira/browse/HBASE-9511
> Project: HBase
> Issue Type: Bug
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9511.00.patch
>
>
> the LZ4 implementation of {{Compression.Algorithm#getCodec(Configuration)}}
> makes an extra call to {{buildCodec}}.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9453) make dev-support/generate-hadoopX-poms.sh have exec perms.
[
https://issues.apache.org/jira/browse/HBASE-9453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9453:
-
Fix Version/s: (was: 0.96.1)
0.96.0
> make dev-support/generate-hadoopX-poms.sh have exec perms.
> --
>
> Key: HBASE-9453
> URL: https://issues.apache.org/jira/browse/HBASE-9453
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.95.2
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9453.patch
>
>
> currently:
> {code}
> jon@swoop:~/proj/hbase-trunk$ ls -la dev-support/generate-hadoopX-poms.sh
> -rw-r--r-- 1 jon jon 5216 2013-09-06 10:45
> dev-support/generate-hadoopX-poms.sh
> {code}
> after patch:
> {code}
> jon@swoop:~/proj/hbase-trunk$ ls -la dev-support/generate-hadoopX-poms.sh
> -rwxr-xr-x 1 jon jon 5216 2013-08-07 18:05
> dev-support/generate-hadoopX-poms.sh
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9430) Memstore heapSize calculation - DEEP_OVERHEAD is incorrect
[
https://issues.apache.org/jira/browse/HBASE-9430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9430:
-
Fix Version/s: (was: 0.96.1)
0.96.0
> Memstore heapSize calculation - DEEP_OVERHEAD is incorrect
> --
>
> Key: HBASE-9430
> URL: https://issues.apache.org/jira/browse/HBASE-9430
> Project: HBase
> Issue Type: Bug
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 0.98.0, 0.96.0, 0.94.13
>
> Attachments: HBASE-9430_94.patch, HBASE-9430.patch
>
>
> {code}
> expected += ClassSize.estimateBase(ConcurrentSkipListMap.class, false);
> expected += ClassSize.estimateBase(ConcurrentSkipListMap.class, false);
> expected += ClassSize.estimateBase(CopyOnWriteArraySet.class, false);
> expected += ClassSize.estimateBase(CopyOnWriteArrayList.class, false);
> {code}
> We need to consider the heap requirement for KeyValueSkipListSet.
> From where CopyOnWriteArraySet & CopyOnWriteArrayList is coming into picture?
> I am not able to follow.
> Also we need to consider the heap for TimeRangeTracker
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9609) AsyncProcess doesn't increase all the counters when trying to limit the per region flow.
[ https://issues.apache.org/jira/browse/HBASE-9609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9609: - Fix Version/s: (was: 0.96.1) 0.96.0 > AsyncProcess doesn't increase all the counters when trying to limit the per > region flow. > > > Key: HBASE-9609 > URL: https://issues.apache.org/jira/browse/HBASE-9609 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.98.0, 0.96.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.98.0, 0.96.0 > > Attachments: 9609.v1.patch > > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786575#comment-13786575
]
Nicolas Liochon commented on HBASE-9612:
v14 is v13 w/o the change on HLogSplitter
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch,
> 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt,
> 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt,
> broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9390) coprocessors observers are not called during a recovery with the new log replay algorithm
[ https://issues.apache.org/jira/browse/HBASE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9390: - Fix Version/s: (was: 0.96.1) 0.98.0 > coprocessors observers are not called during a recovery with the new log > replay algorithm > - > > Key: HBASE-9390 > URL: https://issues.apache.org/jira/browse/HBASE-9390 > Project: HBase > Issue Type: Bug > Components: Coprocessors, MTTR >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.96.0 > > Attachments: copro.patch, hbase-9390-part2.patch, > hbase-9390-part2-v2.patch, hbase-9390.patch, > hbase-9390-review-addendum.patch, hbase-9390-v2.patch > > > See the patch to reproduce the issue: If we activate log replay we don't have > the events on WAL restore. > Pinging [~jeffreyz], we discussed this offline. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-9612:
---
Status: Patch Available (was: Open)
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2, 0.95.1, 0.95.0, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch,
> 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt,
> 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt,
> broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9632) Put the shell in a maven sub module (hbase-shell) instead of hbase-server
[ https://issues.apache.org/jira/browse/HBASE-9632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9632: - Fix Version/s: (was: 0.96.1) 0.96.0 > Put the shell in a maven sub module (hbase-shell) instead of hbase-server > - > > Key: HBASE-9632 > URL: https://issues.apache.org/jira/browse/HBASE-9632 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.98.0, 0.96.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.98.0, 0.96.0 > > Attachments: 9632.v1.patch > > > This will remove the dependency from hbase-server to jruby. jruby is huge and > contains many sub dependencies. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9557) strange dependencies for hbase-client
[ https://issues.apache.org/jira/browse/HBASE-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9557: - Fix Version/s: (was: 0.96.1) 0.96.0 > strange dependencies for hbase-client > - > > Key: HBASE-9557 > URL: https://issues.apache.org/jira/browse/HBASE-9557 > Project: HBase > Issue Type: Bug >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.98.0, 0.96.0 > > Attachments: 9557.v1.patch > > > Here is what we have with hadoop 2. > In our plate we have > - junit (should be test, it's not because we use it in the integration test > runner > - log4j (we have direct dependencies in RegionPlacementMaintainer, > RESTServlet, and LogMonitoring, not counting the dependencies in the tests) > The others are in hadoop. I marked the ones that were strange to me. Do we > need all of them? > mvn dependency:tree -pl hbase-client -Dhadoop.profile=2.0 > [INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ hbase-client --- > [INFO] org.apache.hbase:hbase-client:jar:0.97.0-SNAPSHOT > [INFO] +- org.apache.hbase:hbase-common:jar:0.97.0-SNAPSHOT:compile > [INFO] | \- commons-collections:commons-collections:jar:3.2.1:compile > [INFO] +- org.apache.hbase:hbase-common:test-jar:tests:0.97.0-SNAPSHOT:test > [INFO] +- org.apache.hbase:hbase-protocol:jar:0.97.0-SNAPSHOT:compile > [INFO] +- commons-codec:commons-codec:jar:1.7:compile > [INFO] +- commons-io:commons-io:jar:2.4:compile > [INFO] +- commons-lang:commons-lang:jar:2.6:compile > [INFO] +- commons-logging:commons-logging:jar:1.1.1:compile > [INFO] +- com.google.guava:guava:jar:12.0.1:compile > [INFO] | \- com.google.code.findbugs:jsr305:jar:1.3.9:compile > [INFO] +- com.google.protobuf:protobuf-java:jar:2.5.0:compile > [INFO] +- org.apache.zookeeper:zookeeper:jar:3.4.5:compile > [INFO] | +- org.slf4j:slf4j-api:jar:1.6.4:compile > [INFO] | \- org.slf4j:slf4j-log4j12:jar:1.6.1:compile > [INFO] +- org.cloudera.htrace:htrace-core:jar:2.01:compile > *[INFO] | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile > <=== why?* > [INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile > [INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile > [INFO] +- io.netty:netty:jar:3.5.9.Final:compile > [INFO] +- log4j:log4j:jar:1.2.17:test (scope not updated to compile) > [INFO] +- org.apache.hadoop:hadoop-common:jar:2.1.0-beta:compile > [INFO] | +- commons-cli:commons-cli:jar:1.2:compile > [INFO] | +- org.apache.commons:commons-math:jar:2.2:compile (version managed > from 2.1) > [INFO] | +- xmlenc:xmlenc:jar:0.52:compile > *[INFO] | +- commons-httpclient:commons-httpclient:jar:3.0.1:compile > (version managed from 3.1) <= decrease the version. dangerous. But why > hadoop does this?* > [INFO] | +- commons-net:commons-net:jar:3.1:compile > *[INFO] | +- javax.servlet:servlet-api:jar:2.5:compile < why a servlet > api in hbase-client or hadoop common?* > [INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile > [INFO] | +- com.sun.jersey:jersey-core:jar:1.8:compile > [INFO] | +- com.sun.jersey:jersey-json:jar:1.8:compile > *[INFO] | | +- org.codehaus.jettison:jettison:jar:1.3.1:compile (version > managed from 1.1)* > [INFO] | | | \- stax:stax-api:jar:1.0.1:compile > [INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile > *[INFO] | | | \- javax.xml.bind:jaxb-api:jar:2.1:compile (version managed > from 2.2.2) <= decrease the version. dangerous* > [INFO] | | | \- javax.activation:activation:jar:1.1:compile > [INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.8:compile (version > managed from 1.7.1) > [INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.8.8:compile (version > managed from 1.7.1) > *[INFO] | +- com.sun.jersey:jersey-server:jar:1.8:compile <= Why a > server in a common piece of code? could we exclude it from hbase client?* > [INFO] | | \- asm:asm:jar:3.1:compile > *[INFO] | +- tomcat:jasper-compiler:jar:5.5.23:runtime <=== ??? why * > *[INFO] | +- tomcat:jasper-runtime:jar:5.5.23:runtime <=== ??? why* > *[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime < Why? could we > exclude it from hbase client?* > [INFO] | +- commons-el:commons-el:jar:1.0:runtime > [INFO] | +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile > [INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile > [INFO] | | +- commons-digester:commons-digester:jar:1.8:compile > [INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile > [INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile > [INFO] | +- org.apache.avro:avro:jar:1.5.3:compile > [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile > [INFO] | | \- org.xerial.snappy:snappy-java:jar:1.0.3.2:compile > [IN
[jira] [Updated] (HBASE-9443) ReplicationProtbufUtil.toHLogEntries needs to be either removed or deprecated
[ https://issues.apache.org/jira/browse/HBASE-9443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9443: - Fix Version/s: (was: 0.96.1) 0.96.0 > ReplicationProtbufUtil.toHLogEntries needs to be either removed or deprecated > - > > Key: HBASE-9443 > URL: https://issues.apache.org/jira/browse/HBASE-9443 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 0.96.0 >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > Fix For: 0.98.0, 0.96.0 > > Attachments: 9443.txt > > > Currently, due to HBASE-9442 ReplicationProtbufUtil.toHLogEntries is rendered > useless. > Given that it isn't really used by HBase itself, perhaps it needs to be > removed/deprecated? -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-9612:
---
Status: Open (was: Patch Available)
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2, 0.95.1, 0.95.0, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch,
> 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt,
> 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt,
> broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-9612:
---
Attachment: 9612.v14-nico.patch
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612.v14-nico.patch,
> 9612v2.txt, 9612v3.txt, 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt,
> 9612v7.txt, 9612v8.096.txt, 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt,
> broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9633) Partial reverse of HBASE-9533
[
https://issues.apache.org/jira/browse/HBASE-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9633:
-
Fix Version/s: (was: 0.96.1)
0.96.0
> Partial reverse of HBASE-9533
> -
>
> Key: HBASE-9633
> URL: https://issues.apache.org/jira/browse/HBASE-9633
> Project: HBase
> Issue Type: Bug
> Components: build, Client
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9633.v1.patch
>
>
> I don't understand the solution in HBASE-9533
> In netty 3.3, they changed the group id, if I understand well for legal
> reasons see https://github.com/netty/netty/issues/103). But they have not
> changed the java package name: it's still org.jboss.netty. So we should not
> have to remove our dependency to netty 3.6.
> So:
> - this comment is wrong imho: the explicit load is not related to the package
> name but to how mapreduce load work.
> {code}
> + // This is ugly. Our zk3.4.5 depends on the org.jboss.netty, not
> hadoops io.netty
> + // so need to load it up explicitly while on 3.4.5 zk
> {code}
> - We do use Netty (for the multicast message), so now we're have a missing
> dependency, as maven says:
> {code}
> [INFO] — maven-dependency-plugin:2.1:analyze (default-cli) @ hbase-client —
> [WARNING] Used undeclared dependencies found:
> [WARNING] org.jboss.netty:netty:jar:3.2.2.Final:compile
> {code}
> So I propose a partial reverse. [~saint@gmail.com], [@Aleksandr Shulman]
> would it work for you?
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9652) Remove HServerLoad092
[ https://issues.apache.org/jira/browse/HBASE-9652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9652: - Fix Version/s: (was: 0.96.1) 0.96.0 > Remove HServerLoad092 > - > > Key: HBASE-9652 > URL: https://issues.apache.org/jira/browse/HBASE-9652 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.98.0, 0.96.0 > > Attachments: hbase-9652.patch > > > Say your goodbyes. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9667) NullOutputStream removed from Guava 15
[
https://issues.apache.org/jira/browse/HBASE-9667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9667:
-
Fix Version/s: (was: 0.96.1)
0.96.0
> NullOutputStream removed from Guava 15
> --
>
> Key: HBASE-9667
> URL: https://issues.apache.org/jira/browse/HBASE-9667
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.98.0, 0.94.12, 0.96.1
>Reporter: Matt Greenfield
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9667.v1.patch, 9667.v2.patch, 9667.v3.patch
>
>
> {{com.google.common.io.NullOutputStream}} was dropped in Guava 15.0 in favor
> of {{com.google.common.io.ByteStreams.nullOutputStream()}} which prevents
> projects on this artifact from upgrading from Guava 14 to Guava 15.
> {noformat}
> ERROR 2013-09-26 17:46:12,229 [hbase.master.MasterFileSystem] bootstrap
> org.apache.hadoop.hbase.DroppedSnapshotException: region: -ROOT-,,0
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1608)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1482)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1011)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:959)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:930)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:447)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:387)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:134)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:119)
> at
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:536)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:395)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.NoClassDefFoundError:
> com/google/common/io/NullOutputStream
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.close(HFileWriterV2.java:374)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.close(StoreFile.java:1283)
> at
> org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:836)
> at
> org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:747)
> at
> org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:2229)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1583)
> ... 11 more
> Caused by: java.lang.ClassNotFoundException:
> com.google.common.io.NullOutputStream
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> ... 17 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9685) IntegrationTestMTTR should stop on RetriesExhaustedException
[ https://issues.apache.org/jira/browse/HBASE-9685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9685: - Fix Version/s: (was: 0.96.1) 0.96.0 > IntegrationTestMTTR should stop on RetriesExhaustedException > > > Key: HBASE-9685 > URL: https://issues.apache.org/jira/browse/HBASE-9685 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.98.0, 0.96.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.98.0, 0.96.0 > > Attachments: 9685.v1.patch > > > If it happens, it means that the server is in an unexpected state, to the > next iterations are likely to fail as well. As the client retries for around > 5 minutes, the sooner we stop the better -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9592) Downstream projects cannot use hadoop.profile in their maven builds
[
https://issues.apache.org/jira/browse/HBASE-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9592:
-
Fix Version/s: (was: 0.96.1)
> Downstream projects cannot use hadoop.profile in their maven builds
>
>
> Key: HBASE-9592
> URL: https://issues.apache.org/jira/browse/HBASE-9592
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-9592_v1.patch
>
>
> When a downstream project depends on hbase, and uses hadoop.profile in it's
> build, then maven craps out because the pom file generated by
> generate-hadoopX-poms.sh will not activate the relevant hadoop profile:
> {code}
> mvn -Dhadoop.profile=2 dependency:tree
> [INFO] Scanning for projects...
> [INFO]
>
> [INFO]
>
> [INFO] Building foo-app 1.0-SNAPSHOT
> [INFO]
>
> [WARNING] The POM for org.apache.hbase:hbase-common:jar:0.96.0 is invalid,
> transitive dependencies (if any) will not be available, enable debug logging
> for more details
> [WARNING] The POM for org.apache.hbase:hbase-common:jar:tests:0.96.0 is
> invalid, transitive dependencies (if any) will not be available, enable debug
> logging for more details
> [WARNING] The POM for org.apache.hbase:hbase-client:jar:0.96.0 is invalid,
> transitive dependencies (if any) will not be available, enable debug logging
> for more details
> [WARNING] The POM for org.apache.hbase:hbase-server:jar:0.96.0 is invalid,
> transitive dependencies (if any) will not be available, enable debug logging
> for more details
> [WARNING] The POM for org.apache.hbase:hbase-server:jar:tests:0.96.0 is
> invalid, transitive dependencies (if any) will not be available, enable debug
> logging for more details
> [WARNING] The POM for org.apache.hbase:hbase-hadoop-compat:jar:tests:0.96.0
> is invalid, transitive dependencies (if any) will not be available, enable
> debug logging for more details
> [WARNING] The POM for org.apache.hbase:hbase-hadoop2-compat:jar:tests:0.96.0
> is invalid, transitive dependencies (if any) will not be available, enable
> debug logging for more details
> {code}
> Note that a lot of downstream components like flume, mahout, sqoop use this
> property.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9647) Add a test in TestAsyncProcess to check the number of threads created
[ https://issues.apache.org/jira/browse/HBASE-9647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9647: - Fix Version/s: (was: 0.96.1) 0.96.0 > Add a test in TestAsyncProcess to check the number of threads created > - > > Key: HBASE-9647 > URL: https://issues.apache.org/jira/browse/HBASE-9647 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.98.0, 0.96.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.98.0, 0.96.0 > > Attachments: 9333.v1.patch > > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9676) AsyncProcess can create more tasks than hbase.client.max.total.tasks
[ https://issues.apache.org/jira/browse/HBASE-9676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9676: - Fix Version/s: (was: 0.96.1) 0.96.0 > AsyncProcess can create more tasks than hbase.client.max.total.tasks > > > Key: HBASE-9676 > URL: https://issues.apache.org/jira/browse/HBASE-9676 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.98.0, 0.96.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon >Priority: Critical > Fix For: 0.98.0, 0.96.0 > > Attachments: 9676.v2.patch, 9676.v3.patch, 9676.v4.patch > > > hbase.client.max.total.tasks allows to control the number of tasks in > progress. But when adding tasks, we take only into account the number of > tasks currently running, not the one we're creating. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9612:
-
Priority: Blocker (was: Critical)
This is a blocker. Incompatible changes to wire protocol.
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612v2.txt, 9612v3.txt,
> 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt, 9612v7.txt, 9612v8.096.txt,
> 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt, broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786565#comment-13786565
]
Nicolas Liochon commented on HBASE-9612:
I pulled the trunk & forgot to merge. Will fix.
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612v2.txt, 9612v3.txt,
> 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt, 9612v7.txt, 9612v8.096.txt,
> 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt, broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786563#comment-13786563
]
stack commented on HBASE-9612:
--
Is this change supposed to be in HLogSplitter?
16 // skip the edit
17 -if (loc == null || needSkip) continue;
18 +if (needSkip) continue;
+1 on v13 for trunk and 0.96
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612v2.txt, 9612v3.txt,
> 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt, 9612v7.txt, 9612v8.096.txt,
> 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt, broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9710) Use the region name, not the encoded name, when region is not on current server
[
https://issues.apache.org/jira/browse/HBASE-9710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9710:
-
Attachment: 9710v3.txt
> Use the region name, not the encoded name, when region is not on current
> server
> ---
>
> Key: HBASE-9710
> URL: https://issues.apache.org/jira/browse/HBASE-9710
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: Benoit Sigoure
>Priority: Minor
> Attachments:
> 0001-Log-the-region-name-instead-of-the-encoded-region-na.patch, 9710v2.txt,
> 9710v2.txt, 9710v3.txt
>
>
> When we throw a {{RegionOpeningException}} or a {{NotServingRegionException}}
> we put the encoded region name in the exception, which isn't super useful. I
> propose putting the region name instead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-9612:
---
Status: Open (was: Patch Available)
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2, 0.95.1, 0.95.0, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612v2.txt, 9612v3.txt,
> 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt, 9612v7.txt, 9612v8.096.txt,
> 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt, broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-9612:
---
Status: Patch Available (was: Open)
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2, 0.95.1, 0.95.0, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612v2.txt, 9612v3.txt,
> 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt, 9612v7.txt, 9612v8.096.txt,
> 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt, broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-9612:
---
Attachment: 9612.v13-nico.patch
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments:
> 0001-fix-packaging-by-region-in-MultiServerCallable.patch, 9612.096.v5.txt,
> 9612revert.txt, 9612-v10-nico.patch, 9612v10.txt, 9612-v11-nico.patch,
> 9612v11.txt, 9612v12.txt, 9612.v13-nico.patch, 9612v2.txt, 9612v3.txt,
> 9612v4.txt, 9612v5.txt, 9612v5.txt, 9612v5.txt, 9612v7.txt, 9612v8.096.txt,
> 9612v8.txt, 9612v9.txt, 9612v9.txt, 9612.wip.txt, broken.patch
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
